2024-09-10T10:16:39.310 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T10:16:39.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T10:16:39.426 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029 branch: wip-rishabh-testing-20240909.165402-squid description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} email: null first_in_suite: false job_id: '7899029' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - but it is still running - objects unfound and apparently lost - MDS_SLOW_METADATA_IO - MDS_TRIM sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOE4Q1fM/wqQOf9QV1j/xnQNLV/x1GN7k7t9K3Z12mdTp3KYRictUhGXKeLjLXnD4Ghegb/wveqP0HuZG4Zxnmw= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKoFGj0/6EhtvQ0kieazDWYYHO55D7S4rQ2Gfc2u8WY8c0ziZELLsmkvW4mMDpiNT9qqDhzNsQVqPXUp5Azdjks= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - thrashosds: null - workunit: clients: all: - fs/misc/trivial_sync.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T10:16:39.426 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T10:16:39.427 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T10:16:39.427 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T10:16:39.429 INFO:teuthology.task.internal:Checking packages... 2024-09-10T10:16:39.447 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T10:16:39.447 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T10:16:39.448 INFO:teuthology.packaging:ref: None 2024-09-10T10:16:39.448 INFO:teuthology.packaging:tag: None 2024-09-10T10:16:39.448 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:16:39.448 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:16:39.448 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:16:39.586 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T10:16:39.587 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T10:16:39.594 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T10:16:39.594 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T10:16:39.609 INFO:teuthology.task.internal:Saving configuration 2024-09-10T10:16:39.626 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T10:16:39.633 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T10:16:39.658 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 10:11:39.576162', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOE4Q1fM/wqQOf9QV1j/xnQNLV/x1GN7k7t9K3Z12mdTp3KYRictUhGXKeLjLXnD4Ghegb/wveqP0HuZG4Zxnmw='} 2024-09-10T10:16:39.680 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 10:11:39.577900', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKoFGj0/6EhtvQ0kieazDWYYHO55D7S4rQ2Gfc2u8WY8c0ziZELLsmkvW4mMDpiNT9qqDhzNsQVqPXUp5Azdjks='} 2024-09-10T10:16:39.681 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T10:16:39.686 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-10T10:16:39.686 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-09-10T10:16:39.687 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T10:16:39.747 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa89d4db520>, signals=[15]) 2024-09-10T10:16:39.747 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T10:16:39.753 INFO:teuthology.task.internal:Opening connections... 2024-09-10T10:16:39.753 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:16:39.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:16:39.829 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:16:39.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:16:39.902 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T10:16:39.908 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-09-10T10:16:39.926 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-09-10T10:16:39.926 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-09-10T10:16:39.983 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-09-10T10:16:39.983 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-09-10T10:16:39.983 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-09-10T10:16:39.983 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-09-10T10:16:39.983 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-09-10T10:16:39.984 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-09-10T10:16:39.984 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T10:16:39.984 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-09-10T10:16:39.984 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-09-10T10:16:39.984 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T10:16:39.984 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-09-10T10:16:39.984 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T10:16:39.984 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T10:16:39.984 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T10:16:39.985 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-09-10T10:16:40.006 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-09-10T10:16:40.024 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-09-10T10:16:40.025 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-09-10T10:16:40.082 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-09-10T10:16:40.082 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-09-10T10:16:40.082 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-09-10T10:16:40.082 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-09-10T10:16:40.082 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-09-10T10:16:40.082 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-09-10T10:16:40.082 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T10:16:40.082 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-09-10T10:16:40.083 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-09-10T10:16:40.083 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T10:16:40.083 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-09-10T10:16:40.083 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T10:16:40.083 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T10:16:40.083 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T10:16:40.083 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-09-10T10:16:40.106 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T10:16:40.115 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T10:16:40.145 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T10:16:40.145 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-09-10T10:16:40.147 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-09-10T10:16:40.164 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T10:16:40.171 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T10:16:40.171 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-09-10T10:16:40.205 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-09-10T10:16:40.320 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T10:16:40.368 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T10:16:40.381 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T10:16:40.381 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-10T10:16:40.381 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-10T10:16:40.381 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T10:16:40.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:16:40.398 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-09-10T10:16:40.457 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-503.el9.x86_64 2024-09-10T10:16:40.457 INFO:teuthology.task.kernel:Running kernel on smithi120: 5.14.0-503.el9.x86_64 2024-09-10T10:16:40.457 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum install -y kernel 2024-09-10T10:16:45.378 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.2 MB 00:03 2024-09-10T10:16:47.163 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-10T10:16:52.708 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-10T10:16:54.206 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-10T10:16:55.110 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-10T10:17:00.558 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T10:17:02.523 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T10:17:02.523 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T10:17:02.623 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-10T10:17:02.632 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-10T10:17:02.632 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2024-09-10T10:17:02.632 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-10T10:17:02.632 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-10T10:17:02.632 INFO:teuthology.orchestra.run.smithi120.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T10:17:02.633 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-10T10:17:02.633 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T10:17:02.633 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T10:17:02.633 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T10:17:02.633 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:17:02.633 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-10T10:17:02.633 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-10T10:17:02.634 INFO:teuthology.orchestra.run.smithi120.stdout:Install 4 Packages 2024-09-10T10:17:02.634 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:17:02.635 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 79 M 2024-09-10T10:17:02.635 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 126 M 2024-09-10T10:17:02.635 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-10T10:17:02.768 INFO:teuthology.orchestra.run.smithi120.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 200 kB/s | 23 kB 00:00 2024-09-10T10:17:03.518 INFO:teuthology.orchestra.run.smithi120.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-10T10:17:04.085 INFO:teuthology.orchestra.run.smithi120.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 24 MB/s | 35 MB 00:01 2024-09-10T10:17:04.644 INFO:teuthology.orchestra.run.smithi120.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 15 MB/s | 29 MB 00:01 2024-09-10T10:17:04.644 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:17:04.644 INFO:teuthology.orchestra.run.smithi120.stdout:Total 39 MB/s | 79 MB 00:02 2024-09-10T10:17:05.322 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-10T10:17:05.672 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-10T10:17:05.673 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-10T10:17:06.612 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-10T10:17:06.612 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-10T10:17:09.041 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-10T10:17:09.689 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T10:17:09.771 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:17:12.571 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:17:12.697 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:17:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:17:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:17:22.503 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:18:00.618 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:18:00.662 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:18:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:18:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T10:18:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:18:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:18:01.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:18:01.783 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:18:01.783 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-10T10:18:01.783 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T10:18:01.783 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T10:18:01.783 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T10:18:01.784 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T10:18:01.784 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:18:01.784 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-10T10:18:02.142 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T10:18:02.225 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T10:18:02.225 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T10:18:02.225 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T10:18:02.225 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T10:18:02.225 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T10:18:02.226 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-10T10:18:02.226 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi120.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T10:18:02.226 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T10:18:02.284 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T10:18:02.284 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T10:18:02.284 DEBUG:teuthology.orchestra.run.smithi120:> sudo rpm -qi grub2-tools 2024-09-10T10:18:02.320 INFO:teuthology.orchestra.run.smithi120.stdout:Name : grub2-tools 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:Epoch : 1 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:Version : 2.06 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:Release : 82.el9 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: x86_64 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:Group : Unspecified 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:Size : 8274670 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:License : GPLv3+ 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T10:18:02.321 INFO:teuthology.orchestra.run.smithi120.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout:Packager : builder@centos.org 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout:Vendor : CentOS 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout:Summary : Support tools for GRUB. 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout:Description : 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout:hardware devices. 2024-09-10T10:18:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:18:02.323 INFO:teuthology.orchestra.run.smithi120.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T10:18:02.323 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T10:18:02.323 INFO:teuthology.task.kernel:Updating grub on smithi120 to boot 5.14.0-505.el9.x86_64 2024-09-10T10:18:02.324 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T10:18:02.997 INFO:teuthology.orchestra.run.smithi120.stderr:Generating grub configuration file ... 2024-09-10T10:18:04.291 INFO:teuthology.orchestra.run.smithi120.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T10:18:04.326 INFO:teuthology.orchestra.run.smithi120.stderr:done 2024-09-10T10:18:04.328 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-09-10T10:18:04.344 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.mn8QyPcvHK 2024-09-10T10:18:04.345 DEBUG:teuthology.orchestra.run.smithi120:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mn8QyPcvHK 2024-09-10T10:18:04.412 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.mn8QyPcvHK 2024-09-10T10:18:04.554 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.mn8QyPcvHK is 6KB 2024-09-10T10:18:04.603 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.mn8QyPcvHK 2024-09-10T10:18:04.617 DEBUG:teuthology.orchestra.run.smithi120:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T10:18:04.680 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T10:18:04.680 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T10:18:04.680 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T10:18:04.680 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T10:18:04.681 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T10:18:04.837 DEBUG:teuthology.orchestra.run.smithi120:> sudo shutdown -r now 2024-09-10T10:18:34.840 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T10:18:34.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T10:18:34.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:18:53.373 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-09-10T10:19:02.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T10:19:02.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:19:05.437 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-09-10T10:19:17.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T10:19:17.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:19:49.294 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-09-10T10:19:49.774 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T10:19:49.774 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-10T10:19:49.774 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-09-10T10:19:49.831 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-505.el9.x86_64 2024-09-10T10:19:49.833 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T10:19:49.833 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T10:19:49.833 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T10:19:50.833 DEBUG:teuthology.parallel:result is None 2024-09-10T10:19:50.834 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T10:19:50.840 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T10:19:50.841 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T10:19:50.843 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T10:19:50.861 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T10:19:50.867 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T10:19:50.872 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T10:19:50.873 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T10:19:50.899 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T10:19:51.018 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T10:19:51.027 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T10:19:51.027 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T10:19:51.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:19:51.045 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T10:19:51.048 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T10:19:51.118 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:19:51.134 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:19:51.155 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:19:51.166 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:19:51.167 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T10:19:51.174 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T10:19:51.175 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T10:19:51.177 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T10:19:51.204 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T10:19:51.214 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T10:19:51.215 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T10:19:51.243 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T10:19:51.262 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:19:51.336 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:19:51.390 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:19:51.474 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:19:51.512 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:19:51.512 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T10:19:51.576 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:19:51.623 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:19:51.669 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:19:51.754 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:19:51.797 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:19:51.797 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T10:19:51.863 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-09-10T10:19:51.865 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-09-10T10:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:19:51.946 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:19:52.299 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T10:19:52.306 INFO:teuthology.task.internal:Starting timer... 2024-09-10T10:19:52.306 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T10:19:52.369 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T10:19:52.379 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T10:19:52.379 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-09-10T10:19:52.448 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-09-10T10:19:52.450 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-09-10T10:19:52.511 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-09-10T10:19:52.513 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T10:19:52.513 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-09-10T10:19:52.535 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-09-10T10:19:52.535 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-09-10T10:19:52.561 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-09-10T10:19:52.561 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi120.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-09-10T10:19:52.562 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:19:52.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:19:52.601 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:19:52.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:19:52.629 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T10:19:52.630 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-09-10T10:19:52.684 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-09-10T10:19:52.722 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T10:19:52.733 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T10:19:52.745 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T10:19:52.746 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi120.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T10:23:48.656 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi120.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-09-10T10:23:48.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T10:23:48.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:23:48.733 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-09-10T10:23:48.812 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T10:23:48.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-10T10:23:48.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:23:48.888 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-10T10:23:48.976 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-10T10:23:48.976 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T10:23:48.986 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T10:23:48.986 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T10:23:48.987 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:23:48.989 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T10:23:48.989 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:23:49.015 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T10:23:49.028 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T10:23:49.044 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T10:23:49.053 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-09-10T10:23:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T10:23:49.063 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-09-10T10:23:49.077 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T10:23:49.082 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-09-10T10:23:49.091 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T10:23:49.093 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-09-10T10:23:49.107 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T10:23:49.121 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T10:23:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:23:49.154 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:23:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-10T10:23:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:23:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:23:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:23:49.155 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:23:49.176 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:23:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:23:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-10T10:23:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:23:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:23:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:23:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:23:49.180 INFO:teuthology.run_tasks:Running task install... 2024-09-10T10:23:49.188 DEBUG:teuthology.task.install:project ceph 2024-09-10T10:23:49.189 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T10:23:49.189 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T10:23:49.189 INFO:teuthology.task.install:Using flavor: default 2024-09-10T10:23:49.194 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T10:23:49.194 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T10:23:49.194 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T10:23:49.195 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:23:49.197 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T10:23:49.197 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:23:49.342 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T10:23:49.342 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T10:23:49.351 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T10:23:49.351 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T10:23:49.453 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T10:23:49.454 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:23:49.454 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T10:23:49.462 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T10:23:49.463 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:23:49.463 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T10:23:49.479 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T10:23:49.480 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T10:23:49.488 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T10:23:49.488 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T10:23:49.545 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T10:23:49.553 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T10:23:49.620 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T10:23:49.627 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T10:23:49.684 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-09-10T10:23:49.685 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-09-10T10:23:49.691 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-09-10T10:23:49.693 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-09-10T10:23:49.917 INFO:teuthology.orchestra.run.smithi134.stdout:45 files removed 2024-09-10T10:23:49.918 INFO:teuthology.orchestra.run.smithi120.stdout:45 files removed 2024-09-10T10:23:49.944 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T10:23:49.945 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T10:23:50.491 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-10T10:23:50.507 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 298 kB/s | 84 kB 00:00 2024-09-10T10:23:50.775 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T10:23:50.783 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-10T10:23:51.043 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-10T10:23:51.051 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T10:23:52.768 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-10T10:23:54.142 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.2 MB 00:03 2024-09-10T10:23:57.952 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 5.0 MB/s | 20 MB 00:04 2024-09-10T10:23:59.835 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-09-10T10:24:02.419 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-10T10:24:04.136 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-10T10:24:04.270 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-10T10:24:05.249 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-10T10:24:05.620 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-10T10:24:06.399 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T10:24:10.714 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T10:24:11.930 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-10T10:24:13.100 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:24:13.100 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:24:13.112 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T10:24:13.113 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T10:24:13.113 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T10:24:13.115 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T10:24:13.116 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T10:24:13.118 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T10:24:13.119 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T10:24:13.181 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-10T10:24:13.191 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-10T10:24:13.191 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-10T10:24:13.191 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-10T10:24:13.191 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T10:24:13.192 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T10:24:13.193 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T10:24:13.194 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T10:24:13.194 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T10:24:13.194 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T10:24:13.194 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T10:24:13.194 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T10:24:13.194 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T10:24:13.194 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T10:24:13.194 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T10:24:13.194 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T10:24:13.194 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T10:24:13.195 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T10:24:13.195 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T10:24:13.195 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-09-10T10:24:13.195 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T10:24:13.195 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T10:24:13.195 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T10:24:13.195 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T10:24:13.195 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T10:24:13.195 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T10:24:13.195 INFO:teuthology.orchestra.run.smithi120.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T10:24:13.196 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T10:24:13.196 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-10T10:24:13.196 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T10:24:13.196 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T10:24:13.196 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T10:24:13.196 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T10:24:13.196 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T10:24:13.196 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T10:24:13.196 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T10:24:13.197 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T10:24:13.197 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T10:24:13.197 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T10:24:13.197 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T10:24:13.197 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T10:24:13.197 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T10:24:13.197 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T10:24:13.197 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T10:24:13.197 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T10:24:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T10:24:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T10:24:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T10:24:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T10:24:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T10:24:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T10:24:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T10:24:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T10:24:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T10:24:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T10:24:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T10:24:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T10:24:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T10:24:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T10:24:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T10:24:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T10:24:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T10:24:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T10:24:13.200 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T10:24:13.200 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T10:24:13.200 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T10:24:13.200 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T10:24:13.200 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T10:24:13.200 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T10:24:13.200 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T10:24:13.200 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T10:24:13.200 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T10:24:13.200 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T10:24:13.201 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T10:24:13.201 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T10:24:13.201 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T10:24:13.201 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T10:24:13.201 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T10:24:13.201 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T10:24:13.201 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T10:24:13.201 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T10:24:13.201 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T10:24:13.201 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T10:24:13.202 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T10:24:13.202 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T10:24:13.202 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T10:24:13.202 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T10:24:13.202 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T10:24:13.202 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T10:24:13.202 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T10:24:13.202 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T10:24:13.202 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T10:24:13.202 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T10:24:13.203 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T10:24:13.203 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T10:24:13.203 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T10:24:13.203 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T10:24:13.203 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T10:24:13.203 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T10:24:13.203 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T10:24:13.203 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T10:24:13.203 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T10:24:13.204 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T10:24:13.204 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T10:24:13.204 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T10:24:13.204 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T10:24:13.204 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T10:24:13.204 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T10:24:13.204 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T10:24:13.204 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T10:24:13.204 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T10:24:13.205 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T10:24:13.205 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T10:24:13.205 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T10:24:13.205 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T10:24:13.205 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T10:24:13.205 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T10:24:13.205 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T10:24:13.205 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T10:24:13.205 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T10:24:13.205 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T10:24:13.206 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T10:24:13.206 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T10:24:13.206 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T10:24:13.206 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T10:24:13.206 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T10:24:13.206 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T10:24:13.206 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T10:24:13.206 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T10:24:13.206 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:24:13.207 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-10T10:24:13.208 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-10T10:24:13.208 INFO:teuthology.orchestra.run.smithi120.stdout:Install 148 Packages 2024-09-10T10:24:13.208 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 8 Packages 2024-09-10T10:24:13.208 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:24:13.208 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 226 M 2024-09-10T10:24:13.208 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-10T10:24:13.640 INFO:teuthology.orchestra.run.smithi120.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 32 kB/s | 6.5 kB 00:00 2024-09-10T10:24:14.057 INFO:teuthology.orchestra.run.smithi120.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 2.1 MB/s | 897 kB 00:00 2024-09-10T10:24:14.149 INFO:teuthology.orchestra.run.smithi120.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.5 MB/s | 145 kB 00:00 2024-09-10T10:24:14.265 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:24:14.265 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:24:14.277 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T10:24:14.277 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T10:24:14.278 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T10:24:14.279 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T10:24:14.280 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T10:24:14.283 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T10:24:14.284 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T10:24:14.346 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-10T10:24:14.364 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-10T10:24:14.364 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-10T10:24:14.364 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-10T10:24:14.365 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-10T10:24:14.365 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T10:24:14.365 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T10:24:14.365 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T10:24:14.365 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T10:24:14.365 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T10:24:14.365 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T10:24:14.365 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T10:24:14.366 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T10:24:14.366 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T10:24:14.366 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T10:24:14.366 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T10:24:14.366 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T10:24:14.366 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T10:24:14.366 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T10:24:14.366 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T10:24:14.367 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T10:24:14.368 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T10:24:14.368 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T10:24:14.368 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T10:24:14.368 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T10:24:14.368 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T10:24:14.368 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T10:24:14.368 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T10:24:14.368 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T10:24:14.368 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T10:24:14.368 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T10:24:14.369 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-09-10T10:24:14.369 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T10:24:14.369 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T10:24:14.369 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T10:24:14.369 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T10:24:14.369 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T10:24:14.369 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T10:24:14.369 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T10:24:14.369 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T10:24:14.369 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-10T10:24:14.370 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T10:24:14.370 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T10:24:14.370 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T10:24:14.370 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T10:24:14.370 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T10:24:14.370 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T10:24:14.370 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T10:24:14.370 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T10:24:14.370 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T10:24:14.370 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T10:24:14.371 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T10:24:14.372 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T10:24:14.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T10:24:14.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T10:24:14.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T10:24:14.376 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T10:24:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T10:24:14.378 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout:Install 148 Packages 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 8 Packages 2024-09-10T10:24:14.379 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:24:14.380 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 226 M 2024-09-10T10:24:14.380 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-10T10:24:14.399 INFO:teuthology.orchestra.run.smithi120.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 5.6 MB/s | 5.4 MB 00:00 2024-09-10T10:24:14.550 INFO:teuthology.orchestra.run.smithi120.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 10 MB/s | 1.5 MB 00:00 2024-09-10T10:24:14.758 INFO:teuthology.orchestra.run.smithi120.stdout:(6/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 3.9 MB/s | 2.4 MB 00:00 2024-09-10T10:24:15.000 INFO:teuthology.orchestra.run.smithi120.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-10T10:24:15.448 INFO:teuthology.orchestra.run.smithi134.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 30 kB/s | 6.5 kB 00:00 2024-09-10T10:24:15.525 INFO:teuthology.orchestra.run.smithi120.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 10 MB/s | 21 MB 00:02 2024-09-10T10:24:15.584 INFO:teuthology.orchestra.run.smithi120.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 433 kB/s | 25 kB 00:00 2024-09-10T10:24:15.818 INFO:teuthology.orchestra.run.smithi120.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 13 MB/s | 11 MB 00:00 2024-09-10T10:24:15.873 INFO:teuthology.orchestra.run.smithi134.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 2.1 MB/s | 897 kB 00:00 2024-09-10T10:24:15.910 INFO:teuthology.orchestra.run.smithi120.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 2.4 MB/s | 228 kB 00:00 2024-09-10T10:24:15.966 INFO:teuthology.orchestra.run.smithi134.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.5 MB/s | 145 kB 00:00 2024-09-10T10:24:15.994 INFO:teuthology.orchestra.run.smithi120.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 324 kB/s | 27 kB 00:00 2024-09-10T10:24:16.086 INFO:teuthology.orchestra.run.smithi120.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 346 kB/s | 32 kB 00:00 2024-09-10T10:24:16.090 INFO:teuthology.orchestra.run.smithi134.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 6.3 MB/s | 5.4 MB 00:00 2024-09-10T10:24:16.707 INFO:teuthology.orchestra.run.smithi134.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 2.5 MB/s | 1.5 MB 00:00 2024-09-10T10:24:17.094 INFO:teuthology.orchestra.run.smithi120.stdout:(14/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 7.2 MB/s | 17 MB 00:02 2024-09-10T10:24:17.119 INFO:teuthology.orchestra.run.smithi120.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 727 kB/s | 751 kB 00:01 2024-09-10T10:24:17.124 INFO:teuthology.orchestra.run.smithi134.stdout:(6/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 2.1 MB/s | 2.4 MB 00:01 2024-09-10T10:24:17.153 INFO:teuthology.orchestra.run.smithi120.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.7 MB/s | 164 kB 00:00 2024-09-10T10:24:17.178 INFO:teuthology.orchestra.run.smithi120.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 2.1 MB/s | 127 kB 00:00 2024-09-10T10:24:17.228 INFO:teuthology.orchestra.run.smithi120.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 6.6 MB/s | 506 kB 00:00 2024-09-10T10:24:17.241 INFO:teuthology.orchestra.run.smithi134.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 8.8 MB/s | 4.7 MB 00:00 2024-09-10T10:24:17.303 INFO:teuthology.orchestra.run.smithi120.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 601 kB/s | 45 kB 00:00 2024-09-10T10:24:17.404 INFO:teuthology.orchestra.run.smithi120.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.3 MB/s | 136 kB 00:00 2024-09-10T10:24:17.504 INFO:teuthology.orchestra.run.smithi120.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 1.6 MB/s | 161 kB 00:00 2024-09-10T10:24:17.613 INFO:teuthology.orchestra.run.smithi120.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 2.9 MB/s | 320 kB 00:00 2024-09-10T10:24:17.755 INFO:teuthology.orchestra.run.smithi120.stdout:(23/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 9.3 MB/s | 5.3 MB 00:00 2024-09-10T10:24:17.780 INFO:teuthology.orchestra.run.smithi120.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 1.8 MB/s | 301 kB 00:00 2024-09-10T10:24:17.839 INFO:teuthology.orchestra.run.smithi120.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.2 MB/s | 99 kB 00:00 2024-09-10T10:24:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-10T10:24:17.964 INFO:teuthology.orchestra.run.smithi120.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 1.7 MB/s | 170 kB 00:00 2024-09-10T10:24:18.064 INFO:teuthology.orchestra.run.smithi120.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 318 kB/s | 32 kB 00:00 2024-09-10T10:24:18.182 INFO:teuthology.orchestra.run.smithi120.stdout:(29/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 9.0 MB/s | 3.1 MB 00:00 2024-09-10T10:24:18.209 INFO:teuthology.orchestra.run.smithi120.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 1.0 MB/s | 146 kB 00:00 2024-09-10T10:24:18.441 INFO:teuthology.orchestra.run.smithi134.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 6.6 MB/s | 21 MB 00:03 2024-09-10T10:24:18.490 INFO:teuthology.orchestra.run.smithi120.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 12 MB/s | 3.6 MB 00:00 2024-09-10T10:24:18.491 INFO:teuthology.orchestra.run.smithi134.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 507 kB/s | 25 kB 00:00 2024-09-10T10:24:18.608 INFO:teuthology.orchestra.run.smithi134.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 7.8 MB/s | 11 MB 00:01 2024-09-10T10:24:18.658 INFO:teuthology.orchestra.run.smithi134.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 4.5 MB/s | 228 kB 00:00 2024-09-10T10:24:18.708 INFO:teuthology.orchestra.run.smithi134.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 541 kB/s | 27 kB 00:00 2024-09-10T10:24:18.759 INFO:teuthology.orchestra.run.smithi134.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 635 kB/s | 32 kB 00:00 2024-09-10T10:24:18.842 INFO:teuthology.orchestra.run.smithi134.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 8.8 MB/s | 751 kB 00:00 2024-09-10T10:24:18.893 INFO:teuthology.orchestra.run.smithi134.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 3.2 MB/s | 164 kB 00:00 2024-09-10T10:24:18.933 INFO:teuthology.orchestra.run.smithi120.stdout:(32/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 15 MB/s | 49 MB 00:03 2024-09-10T10:24:18.943 INFO:teuthology.orchestra.run.smithi134.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T10:24:18.958 INFO:teuthology.orchestra.run.smithi120.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 534 kB/s | 249 kB 00:00 2024-09-10T10:24:18.983 INFO:teuthology.orchestra.run.smithi120.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 990 kB/s | 49 kB 00:00 2024-09-10T10:24:19.008 INFO:teuthology.orchestra.run.smithi120.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 329 kB/s | 16 kB 00:00 2024-09-10T10:24:19.010 INFO:teuthology.orchestra.run.smithi134.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 7.4 MB/s | 506 kB 00:00 2024-09-10T10:24:19.034 INFO:teuthology.orchestra.run.smithi120.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 5.5 MB/s | 285 kB 00:00 2024-09-10T10:24:19.093 INFO:teuthology.orchestra.run.smithi120.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 8.9 MB/s | 766 kB 00:00 2024-09-10T10:24:19.117 INFO:teuthology.orchestra.run.smithi120.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 218 kB/s | 18 kB 00:00 2024-09-10T10:24:19.243 INFO:teuthology.orchestra.run.smithi120.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 7.1 MB/s | 7.4 MB 00:01 2024-09-10T10:24:19.268 INFO:teuthology.orchestra.run.smithi120.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 455 kB/s | 80 kB 00:00 2024-09-10T10:24:19.402 INFO:teuthology.orchestra.run.smithi120.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-10T10:24:19.402 INFO:teuthology.orchestra.run.smithi134.stdout:(18/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-10T10:24:19.427 INFO:teuthology.orchestra.run.smithi120.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-10T10:24:19.452 INFO:teuthology.orchestra.run.smithi134.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 908 kB/s | 45 kB 00:00 2024-09-10T10:24:19.477 INFO:teuthology.orchestra.run.smithi120.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T10:24:19.502 INFO:teuthology.orchestra.run.smithi134.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 2.7 MB/s | 136 kB 00:00 2024-09-10T10:24:19.528 INFO:teuthology.orchestra.run.smithi120.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T10:24:19.553 INFO:teuthology.orchestra.run.smithi134.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 3.1 MB/s | 161 kB 00:00 2024-09-10T10:24:19.553 INFO:teuthology.orchestra.run.smithi120.stdout:(45/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.8 MB/s | 801 kB 00:00 2024-09-10T10:24:19.603 INFO:teuthology.orchestra.run.smithi134.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 6.3 MB/s | 320 kB 00:00 2024-09-10T10:24:19.604 INFO:teuthology.orchestra.run.smithi120.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-10T10:24:19.637 INFO:teuthology.orchestra.run.smithi120.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-10T10:24:19.662 INFO:teuthology.orchestra.run.smithi134.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.0 MB/s | 301 kB 00:00 2024-09-10T10:24:19.712 INFO:teuthology.orchestra.run.smithi134.stdout:(24/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T10:24:19.754 INFO:teuthology.orchestra.run.smithi120.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-09-10T10:24:19.762 INFO:teuthology.orchestra.run.smithi134.stdout:(25/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T10:24:19.963 INFO:teuthology.orchestra.run.smithi134.stdout:(26/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-10T10:24:20.013 INFO:teuthology.orchestra.run.smithi134.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-10T10:24:20.063 INFO:teuthology.orchestra.run.smithi134.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 637 kB/s | 32 kB 00:00 2024-09-10T10:24:20.113 INFO:teuthology.orchestra.run.smithi134.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T10:24:20.347 INFO:teuthology.orchestra.run.smithi134.stdout:(30/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 15 MB/s | 3.6 MB 00:00 2024-09-10T10:24:20.421 INFO:teuthology.orchestra.run.smithi120.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 160 kB/s | 107 kB 00:00 2024-09-10T10:24:20.504 INFO:teuthology.orchestra.run.smithi120.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 765 kB/s | 689 kB 00:00 2024-09-10T10:24:20.563 INFO:teuthology.orchestra.run.smithi120.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-10T10:24:20.772 INFO:teuthology.orchestra.run.smithi134.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-10T10:24:20.948 INFO:teuthology.orchestra.run.smithi134.stdout:(32/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 4.4 MB/s | 17 MB 00:03 2024-09-10T10:24:20.973 INFO:teuthology.orchestra.run.smithi134.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 1.2 MB/s | 249 kB 00:00 2024-09-10T10:24:20.989 INFO:teuthology.orchestra.run.smithi120.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 38 kB/s | 22 kB 00:00 2024-09-10T10:24:21.006 INFO:teuthology.orchestra.run.smithi134.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 846 kB/s | 49 kB 00:00 2024-09-10T10:24:21.031 INFO:teuthology.orchestra.run.smithi134.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 281 kB/s | 16 kB 00:00 2024-09-10T10:24:21.132 INFO:teuthology.orchestra.run.smithi134.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 2.2 MB/s | 285 kB 00:00 2024-09-10T10:24:21.139 INFO:teuthology.orchestra.run.smithi120.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 57 kB/s | 33 kB 00:00 2024-09-10T10:24:21.166 INFO:teuthology.orchestra.run.smithi134.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 5.6 MB/s | 766 kB 00:00 2024-09-10T10:24:21.199 INFO:teuthology.orchestra.run.smithi134.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 272 kB/s | 18 kB 00:00 2024-09-10T10:24:21.325 INFO:teuthology.orchestra.run.smithi134.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 504 kB/s | 80 kB 00:00 2024-09-10T10:24:21.458 INFO:teuthology.orchestra.run.smithi120.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 362 kB/s | 313 kB 00:00 2024-09-10T10:24:21.459 INFO:teuthology.orchestra.run.smithi134.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-10T10:24:21.459 INFO:teuthology.orchestra.run.smithi134.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-10T10:24:21.459 INFO:teuthology.orchestra.run.smithi134.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-10T10:24:21.639 INFO:teuthology.orchestra.run.smithi120.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-09-10T10:24:21.864 INFO:teuthology.orchestra.run.smithi120.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-10T10:24:21.951 INFO:teuthology.orchestra.run.smithi134.stdout:(43/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 14 MB/s | 49 MB 00:03 2024-09-10T10:24:21.976 INFO:teuthology.orchestra.run.smithi134.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 203 kB/s | 106 kB 00:00 2024-09-10T10:24:22.026 INFO:teuthology.orchestra.run.smithi134.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T10:24:22.056 INFO:teuthology.orchestra.run.smithi120.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-10T10:24:22.068 INFO:teuthology.orchestra.run.smithi134.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 1.9 MB/s | 1.2 MB 00:00 2024-09-10T10:24:22.093 INFO:teuthology.orchestra.run.smithi134.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 951 kB/s | 135 kB 00:00 2024-09-10T10:24:22.118 INFO:teuthology.orchestra.run.smithi134.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-10T10:24:22.474 INFO:teuthology.orchestra.run.smithi120.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 192 kB/s | 160 kB 00:00 2024-09-10T10:24:22.499 INFO:teuthology.orchestra.run.smithi120.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-09-10T10:24:22.827 INFO:teuthology.orchestra.run.smithi134.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 151 kB/s | 107 kB 00:00 2024-09-10T10:24:23.019 INFO:teuthology.orchestra.run.smithi134.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 725 kB/s | 689 kB 00:00 2024-09-10T10:24:23.041 INFO:teuthology.orchestra.run.smithi120.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 45 kB/s | 24 kB 00:00 2024-09-10T10:24:23.103 INFO:teuthology.orchestra.run.smithi134.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 934 kB/s | 943 kB 00:01 2024-09-10T10:24:23.353 INFO:teuthology.orchestra.run.smithi134.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 41 kB/s | 22 kB 00:00 2024-09-10T10:24:23.399 INFO:teuthology.orchestra.run.smithi120.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 494 kB/s | 662 kB 00:01 2024-09-10T10:24:23.541 INFO:teuthology.orchestra.run.smithi120.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 544 kB/s | 581 kB 00:01 2024-09-10T10:24:23.616 INFO:teuthology.orchestra.run.smithi120.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-10T10:24:23.687 INFO:teuthology.orchestra.run.smithi134.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 56 kB/s | 33 kB 00:00 2024-09-10T10:24:24.020 INFO:teuthology.orchestra.run.smithi134.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 313 kB/s | 313 kB 00:01 2024-09-10T10:24:24.183 INFO:teuthology.orchestra.run.smithi120.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 373 kB/s | 292 kB 00:00 2024-09-10T10:24:24.209 INFO:teuthology.orchestra.run.smithi120.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 66 kB/s | 38 kB 00:00 2024-09-10T10:24:24.220 INFO:teuthology.orchestra.run.smithi134.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-09-10T10:24:24.276 INFO:teuthology.orchestra.run.smithi120.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 257 kB/s | 188 kB 00:00 2024-09-10T10:24:24.479 INFO:teuthology.orchestra.run.smithi134.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-09-10T10:24:24.579 INFO:teuthology.orchestra.run.smithi134.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 20 kB/s | 11 kB 00:00 2024-09-10T10:24:25.013 INFO:teuthology.orchestra.run.smithi134.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 202 kB/s | 160 kB 00:00 2024-09-10T10:24:25.055 INFO:teuthology.orchestra.run.smithi134.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 80 kB/s | 45 kB 00:00 2024-09-10T10:24:25.463 INFO:teuthology.orchestra.run.smithi134.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 750 kB/s | 662 kB 00:00 2024-09-10T10:24:25.664 INFO:teuthology.orchestra.run.smithi134.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 40 kB/s | 24 kB 00:00 2024-09-10T10:24:25.759 INFO:teuthology.orchestra.run.smithi120.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.1 MB/s | 4.9 MB 00:01 2024-09-10T10:24:25.801 INFO:teuthology.orchestra.run.smithi120.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.0 MB/s | 1.6 MB 00:01 2024-09-10T10:24:25.868 INFO:teuthology.orchestra.run.smithi120.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-09-10T10:24:26.039 INFO:teuthology.orchestra.run.smithi134.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-10T10:24:26.222 INFO:teuthology.orchestra.run.smithi134.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 480 kB/s | 581 kB 00:01 2024-09-10T10:24:26.568 INFO:teuthology.orchestra.run.smithi120.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 179 kB/s | 137 kB 00:00 2024-09-10T10:24:26.572 INFO:teuthology.orchestra.run.smithi134.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 322 kB/s | 292 kB 00:00 2024-09-10T10:24:26.768 INFO:teuthology.orchestra.run.smithi120.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 902 kB/s | 910 kB 00:01 2024-09-10T10:24:26.781 INFO:teuthology.orchestra.run.smithi134.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 254 kB/s | 188 kB 00:00 2024-09-10T10:24:26.806 INFO:teuthology.orchestra.run.smithi134.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 66 kB/s | 38 kB 00:00 2024-09-10T10:24:27.252 INFO:teuthology.orchestra.run.smithi120.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.2 MB/s | 5.8 MB 00:01 2024-09-10T10:24:27.453 INFO:teuthology.orchestra.run.smithi120.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 128 kB/s | 87 kB 00:00 2024-09-10T10:24:28.078 INFO:teuthology.orchestra.run.smithi120.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 98 kB/s | 61 kB 00:00 2024-09-10T10:24:28.370 INFO:teuthology.orchestra.run.smithi120.stdout:(75/156): protobuf-3.14.0-13.el9.x86_64.rpm 574 kB/s | 1.0 MB 00:01 2024-09-10T10:24:28.456 INFO:teuthology.orchestra.run.smithi134.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 989 kB/s | 1.6 MB 00:01 2024-09-10T10:24:28.632 INFO:teuthology.orchestra.run.smithi134.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.4 MB/s | 4.4 MB 00:01 2024-09-10T10:24:28.695 INFO:teuthology.orchestra.run.smithi120.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-09-10T10:24:28.782 INFO:teuthology.orchestra.run.smithi134.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.2 MB/s | 4.9 MB 00:02 2024-09-10T10:24:28.829 INFO:teuthology.orchestra.run.smithi120.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 326 kB/s | 245 kB 00:00 2024-09-10T10:24:29.212 INFO:teuthology.orchestra.run.smithi120.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 295 kB/s | 248 kB 00:00 2024-09-10T10:24:29.262 INFO:teuthology.orchestra.run.smithi120.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 26 kB 00:00 2024-09-10T10:24:29.329 INFO:teuthology.orchestra.run.smithi120.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 19 kB 00:00 2024-09-10T10:24:29.340 INFO:teuthology.orchestra.run.smithi134.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 194 kB/s | 137 kB 00:00 2024-09-10T10:24:29.391 INFO:teuthology.orchestra.run.smithi134.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 975 kB/s | 910 kB 00:00 2024-09-10T10:24:29.846 INFO:teuthology.orchestra.run.smithi120.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 60 kB/s | 35 kB 00:00 2024-09-10T10:24:29.938 INFO:teuthology.orchestra.run.smithi120.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 237 kB/s | 172 kB 00:00 2024-09-10T10:24:30.040 INFO:teuthology.orchestra.run.smithi134.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 135 kB/s | 87 kB 00:00 2024-09-10T10:24:30.132 INFO:teuthology.orchestra.run.smithi134.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 4.3 MB/s | 5.8 MB 00:01 2024-09-10T10:24:30.332 INFO:teuthology.orchestra.run.smithi134.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-09-10T10:24:30.730 INFO:teuthology.orchestra.run.smithi120.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 284 kB/s | 250 kB 00:00 2024-09-10T10:24:30.757 INFO:teuthology.orchestra.run.smithi134.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 98 kB/s | 61 kB 00:00 2024-09-10T10:24:30.822 INFO:teuthology.orchestra.run.smithi120.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 251 kB/s | 222 kB 00:00 2024-09-10T10:24:31.141 INFO:teuthology.orchestra.run.smithi134.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 303 kB/s | 245 kB 00:00 2024-09-10T10:24:31.298 INFO:teuthology.orchestra.run.smithi120.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.6 MB/s | 5.1 MB 00:01 2024-09-10T10:24:31.398 INFO:teuthology.orchestra.run.smithi120.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 73 kB/s | 42 kB 00:00 2024-09-10T10:24:31.423 INFO:teuthology.orchestra.run.smithi120.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 111 kB/s | 77 kB 00:00 2024-09-10T10:24:31.583 INFO:teuthology.orchestra.run.smithi134.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-09-10T10:24:31.608 INFO:teuthology.orchestra.run.smithi134.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 292 kB/s | 248 kB 00:00 2024-09-10T10:24:31.875 INFO:teuthology.orchestra.run.smithi134.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 35 kB/s | 26 kB 00:00 2024-09-10T10:24:32.091 INFO:teuthology.orchestra.run.smithi134.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 19 kB 00:00 2024-09-10T10:24:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 337 kB/s | 269 kB 00:00 2024-09-10T10:24:32.124 INFO:teuthology.orchestra.run.smithi120.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 219 kB/s | 159 kB 00:00 2024-09-10T10:24:32.215 INFO:teuthology.orchestra.run.smithi120.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 352 kB/s | 279 kB 00:00 2024-09-10T10:24:32.359 INFO:teuthology.orchestra.run.smithi134.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 230 kB/s | 172 kB 00:00 2024-09-10T10:24:32.434 INFO:teuthology.orchestra.run.smithi134.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 62 kB/s | 35 kB 00:00 2024-09-10T10:24:32.741 INFO:teuthology.orchestra.run.smithi120.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 51 kB 00:00 2024-09-10T10:24:32.766 INFO:teuthology.orchestra.run.smithi120.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 83 kB/s | 54 kB 00:00 2024-09-10T10:24:33.234 INFO:teuthology.orchestra.run.smithi134.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 286 kB/s | 250 kB 00:00 2024-09-10T10:24:33.308 INFO:teuthology.orchestra.run.smithi120.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 74 kB/s | 42 kB 00:00 2024-09-10T10:24:33.333 INFO:teuthology.orchestra.run.smithi120.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 78 kB/s | 44 kB 00:00 2024-09-10T10:24:33.376 INFO:teuthology.orchestra.run.smithi134.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 236 kB/s | 222 kB 00:00 2024-09-10T10:24:33.883 INFO:teuthology.orchestra.run.smithi120.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 65 kB/s | 36 kB 00:00 2024-09-10T10:24:33.884 INFO:teuthology.orchestra.run.smithi134.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 119 kB/s | 77 kB 00:00 2024-09-10T10:24:33.951 INFO:teuthology.orchestra.run.smithi134.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 73 kB/s | 42 kB 00:00 2024-09-10T10:24:34.100 INFO:teuthology.orchestra.run.smithi120.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 279 kB/s | 221 kB 00:00 2024-09-10T10:24:34.310 INFO:teuthology.orchestra.run.smithi134.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.3 MB/s | 5.1 MB 00:02 2024-09-10T10:24:34.609 INFO:teuthology.orchestra.run.smithi120.stdout:(97/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.8 MB/s | 16 MB 00:02 2024-09-10T10:24:34.634 INFO:teuthology.orchestra.run.smithi120.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 63 kB/s | 47 kB 00:00 2024-09-10T10:24:34.685 INFO:teuthology.orchestra.run.smithi134.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 337 kB/s | 269 kB 00:00 2024-09-10T10:24:34.752 INFO:teuthology.orchestra.run.smithi134.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 198 kB/s | 159 kB 00:00 2024-09-10T10:24:35.193 INFO:teuthology.orchestra.run.smithi120.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 279 kB/s | 304 kB 00:01 2024-09-10T10:24:35.218 INFO:teuthology.orchestra.run.smithi120.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 95 kB/s | 58 kB 00:00 2024-09-10T10:24:35.261 INFO:teuthology.orchestra.run.smithi134.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 294 kB/s | 279 kB 00:00 2024-09-10T10:24:35.286 INFO:teuthology.orchestra.run.smithi134.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-09-10T10:24:35.326 INFO:teuthology.orchestra.run.smithi120.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 138 kB/s | 96 kB 00:00 2024-09-10T10:24:35.370 INFO:teuthology.orchestra.run.smithi134.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 87 kB/s | 54 kB 00:00 2024-09-10T10:24:35.377 INFO:teuthology.orchestra.run.smithi120.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 448 kB/s | 22 kB 00:00 2024-09-10T10:24:35.585 INFO:teuthology.orchestra.run.smithi120.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-09-10T10:24:35.644 INFO:teuthology.orchestra.run.smithi120.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-10T10:24:35.678 INFO:teuthology.orchestra.run.smithi120.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T10:24:35.703 INFO:teuthology.orchestra.run.smithi120.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-10T10:24:35.812 INFO:teuthology.orchestra.run.smithi120.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-10T10:24:35.837 INFO:teuthology.orchestra.run.smithi120.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T10:24:35.861 INFO:teuthology.orchestra.run.smithi134.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 72 kB/s | 42 kB 00:00 2024-09-10T10:24:35.862 INFO:teuthology.orchestra.run.smithi120.stdout:(109/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 95 kB/s | 64 kB 00:00 2024-09-10T10:24:35.887 INFO:teuthology.orchestra.run.smithi120.stdout:(110/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 67 kB/s | 45 kB 00:00 2024-09-10T10:24:35.996 INFO:teuthology.orchestra.run.smithi134.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 71 kB/s | 44 kB 00:00 2024-09-10T10:24:36.072 INFO:teuthology.orchestra.run.smithi120.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 208 kB/s | 49 kB 00:00 2024-09-10T10:24:36.197 INFO:teuthology.orchestra.run.smithi120.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 202 kB/s | 67 kB 00:00 2024-09-10T10:24:36.230 INFO:teuthology.orchestra.run.smithi120.stdout:(113/156): luarocks-3.9.2-5.el9.noarch.rpm 451 kB/s | 151 kB 00:00 2024-09-10T10:24:36.256 INFO:teuthology.orchestra.run.smithi120.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T10:24:36.281 INFO:teuthology.orchestra.run.smithi120.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T10:24:36.306 INFO:teuthology.orchestra.run.smithi120.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-10T10:24:36.340 INFO:teuthology.orchestra.run.smithi120.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-09-10T10:24:36.365 INFO:teuthology.orchestra.run.smithi120.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-09-10T10:24:36.390 INFO:teuthology.orchestra.run.smithi120.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-10T10:24:36.416 INFO:teuthology.orchestra.run.smithi120.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-10T10:24:36.441 INFO:teuthology.orchestra.run.smithi120.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T10:24:36.466 INFO:teuthology.orchestra.run.smithi120.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-09-10T10:24:36.492 INFO:teuthology.orchestra.run.smithi120.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-10T10:24:36.517 INFO:teuthology.orchestra.run.smithi120.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-10T10:24:36.542 INFO:teuthology.orchestra.run.smithi120.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-10T10:24:36.545 INFO:teuthology.orchestra.run.smithi134.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 65 kB/s | 36 kB 00:00 2024-09-10T10:24:36.567 INFO:teuthology.orchestra.run.smithi120.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-10T10:24:36.601 INFO:teuthology.orchestra.run.smithi120.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-09-10T10:24:36.626 INFO:teuthology.orchestra.run.smithi120.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-09-10T10:24:36.652 INFO:teuthology.orchestra.run.smithi120.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T10:24:36.670 INFO:teuthology.orchestra.run.smithi134.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 273 kB/s | 221 kB 00:00 2024-09-10T10:24:36.686 INFO:teuthology.orchestra.run.smithi120.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T10:24:36.711 INFO:teuthology.orchestra.run.smithi120.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T10:24:36.753 INFO:teuthology.orchestra.run.smithi120.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.7 MB/s | 1.0 MB 00:00 2024-09-10T10:24:36.778 INFO:teuthology.orchestra.run.smithi120.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-10T10:24:36.812 INFO:teuthology.orchestra.run.smithi120.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-10T10:24:36.837 INFO:teuthology.orchestra.run.smithi120.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T10:24:36.862 INFO:teuthology.orchestra.run.smithi120.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-10T10:24:36.888 INFO:teuthology.orchestra.run.smithi120.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-10T10:24:36.913 INFO:teuthology.orchestra.run.smithi120.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T10:24:36.938 INFO:teuthology.orchestra.run.smithi120.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-10T10:24:36.963 INFO:teuthology.orchestra.run.smithi120.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-10T10:24:36.989 INFO:teuthology.orchestra.run.smithi120.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T10:24:37.031 INFO:teuthology.orchestra.run.smithi120.stdout:(142/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-10T10:24:37.056 INFO:teuthology.orchestra.run.smithi120.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-10T10:24:37.082 INFO:teuthology.orchestra.run.smithi120.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-10T10:24:37.107 INFO:teuthology.orchestra.run.smithi120.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-10T10:24:37.132 INFO:teuthology.orchestra.run.smithi120.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T10:24:37.146 INFO:teuthology.orchestra.run.smithi134.stdout:(97/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.6 MB/s | 16 MB 00:01 2024-09-10T10:24:37.157 INFO:teuthology.orchestra.run.smithi120.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-10T10:24:37.171 INFO:teuthology.orchestra.run.smithi134.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-09-10T10:24:37.208 INFO:teuthology.orchestra.run.smithi120.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T10:24:37.433 INFO:teuthology.orchestra.run.smithi120.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-10T10:24:37.500 INFO:teuthology.orchestra.run.smithi120.stdout:(150/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 9.2 MB/s | 3.2 MB 00:00 2024-09-10T10:24:37.550 INFO:teuthology.orchestra.run.smithi120.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 817 kB/s | 41 kB 00:00 2024-09-10T10:24:37.563 INFO:teuthology.orchestra.run.smithi134.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 341 kB/s | 304 kB 00:00 2024-09-10T10:24:37.617 INFO:teuthology.orchestra.run.smithi120.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 5.7 MB/s | 2.3 MB 00:00 2024-09-10T10:24:37.668 INFO:teuthology.orchestra.run.smithi120.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.1 MB/s | 249 kB 00:00 2024-09-10T10:24:37.755 INFO:teuthology.orchestra.run.smithi134.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 95 kB/s | 58 kB 00:00 2024-09-10T10:24:37.855 INFO:teuthology.orchestra.run.smithi134.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 140 kB/s | 96 kB 00:00 2024-09-10T10:24:37.876 INFO:teuthology.orchestra.run.smithi120.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 5.2 MB/s | 1.4 MB 00:00 2024-09-10T10:24:37.939 INFO:teuthology.orchestra.run.smithi134.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-09-10T10:24:37.951 INFO:teuthology.orchestra.run.smithi120.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.4 MB/s | 2.4 MB 00:00 2024-09-10T10:24:38.101 INFO:teuthology.orchestra.run.smithi120.stdout:(156/156): NetworkManager-libnm-1.49.90-1.el9.x 2.7 MB/s | 1.8 MB 00:00 2024-09-10T10:24:38.107 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:24:38.107 INFO:teuthology.orchestra.run.smithi120.stdout:Total 9.1 MB/s | 226 MB 00:24 2024-09-10T10:24:38.189 INFO:teuthology.orchestra.run.smithi134.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-09-10T10:24:38.214 INFO:teuthology.orchestra.run.smithi134.stdout:(104/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 98 kB/s | 64 kB 00:00 2024-09-10T10:24:38.273 INFO:teuthology.orchestra.run.smithi134.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.5 MB/s | 548 kB 00:00 2024-09-10T10:24:38.297 INFO:teuthology.orchestra.run.smithi134.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-09-10T10:24:38.323 INFO:teuthology.orchestra.run.smithi134.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-09-10T10:24:38.348 INFO:teuthology.orchestra.run.smithi134.stdout:(108/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 75 kB/s | 45 kB 00:00 2024-09-10T10:24:38.373 INFO:teuthology.orchestra.run.smithi134.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 531 kB/s | 26 kB 00:00 2024-09-10T10:24:38.407 INFO:teuthology.orchestra.run.smithi134.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T10:24:38.432 INFO:teuthology.orchestra.run.smithi134.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-10T10:24:38.457 INFO:teuthology.orchestra.run.smithi134.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 445 kB/s | 49 kB 00:00 2024-09-10T10:24:38.516 INFO:teuthology.orchestra.run.smithi134.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-10T10:24:38.550 INFO:teuthology.orchestra.run.smithi134.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-10T10:24:38.575 INFO:teuthology.orchestra.run.smithi134.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-09-10T10:24:38.600 INFO:teuthology.orchestra.run.smithi134.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-09-10T10:24:38.626 INFO:teuthology.orchestra.run.smithi134.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-10T10:24:38.651 INFO:teuthology.orchestra.run.smithi134.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T10:24:38.685 INFO:teuthology.orchestra.run.smithi134.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T10:24:38.718 INFO:teuthology.orchestra.run.smithi134.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-10T10:24:38.752 INFO:teuthology.orchestra.run.smithi134.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-10T10:24:38.785 INFO:teuthology.orchestra.run.smithi134.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-10T10:24:38.811 INFO:teuthology.orchestra.run.smithi134.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-10T10:24:38.828 INFO:teuthology.orchestra.run.smithi134.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-10T10:24:38.845 INFO:teuthology.orchestra.run.smithi134.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-10T10:24:38.861 INFO:teuthology.orchestra.run.smithi134.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-10T10:24:38.887 INFO:teuthology.orchestra.run.smithi134.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-09-10T10:24:38.954 INFO:teuthology.orchestra.run.smithi134.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-09-10T10:24:38.979 INFO:teuthology.orchestra.run.smithi134.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T10:24:39.005 INFO:teuthology.orchestra.run.smithi134.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T10:24:39.030 INFO:teuthology.orchestra.run.smithi134.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T10:24:39.080 INFO:teuthology.orchestra.run.smithi134.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.6 MB/s | 2.0 MB 00:00 2024-09-10T10:24:39.105 INFO:teuthology.orchestra.run.smithi134.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-10T10:24:39.139 INFO:teuthology.orchestra.run.smithi134.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-10T10:24:39.164 INFO:teuthology.orchestra.run.smithi134.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-10T10:24:39.189 INFO:teuthology.orchestra.run.smithi134.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-10T10:24:39.214 INFO:teuthology.orchestra.run.smithi134.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-10T10:24:39.257 INFO:teuthology.orchestra.run.smithi134.stdout:(138/156): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-09-10T10:24:39.282 INFO:teuthology.orchestra.run.smithi134.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 887 kB/s | 59 kB 00:00 2024-09-10T10:24:39.291 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-10T10:24:39.307 INFO:teuthology.orchestra.run.smithi134.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-10T10:24:39.332 INFO:teuthology.orchestra.run.smithi134.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-10T10:24:39.357 INFO:teuthology.orchestra.run.smithi134.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-09-10T10:24:39.382 INFO:teuthology.orchestra.run.smithi134.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-10T10:24:39.408 INFO:teuthology.orchestra.run.smithi134.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T10:24:39.433 INFO:teuthology.orchestra.run.smithi134.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-10T10:24:39.443 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-10T10:24:39.444 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-10T10:24:39.458 INFO:teuthology.orchestra.run.smithi134.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T10:24:39.484 INFO:teuthology.orchestra.run.smithi134.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-10T10:24:39.558 INFO:teuthology.orchestra.run.smithi134.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-10T10:24:39.793 INFO:teuthology.orchestra.run.smithi134.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-10T10:24:39.918 INFO:teuthology.orchestra.run.smithi134.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 6.4 MB/s | 2.3 MB 00:00 2024-09-10T10:24:39.960 INFO:teuthology.orchestra.run.smithi134.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 984 kB/s | 41 kB 00:00 2024-09-10T10:24:40.019 INFO:teuthology.orchestra.run.smithi134.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 8.0 MB/s | 1.8 MB 00:00 2024-09-10T10:24:40.044 INFO:teuthology.orchestra.run.smithi134.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-10T10:24:40.111 INFO:teuthology.orchestra.run.smithi134.stdout:(154/156): librados2-19.1.1-305.g1cf819b7.el9.x 5.2 MB/s | 3.4 MB 00:00 2024-09-10T10:24:40.153 INFO:teuthology.orchestra.run.smithi134.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-10T10:24:40.211 INFO:teuthology.orchestra.run.smithi134.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-10T10:24:40.219 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:24:40.219 INFO:teuthology.orchestra.run.smithi134.stdout:Total 8.8 MB/s | 226 MB 00:25 2024-09-10T10:24:41.439 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-10T10:24:41.590 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-10T10:24:41.590 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-10T10:24:41.901 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-10T10:24:41.901 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-10T10:24:43.381 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-10T10:24:43.381 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-10T10:24:48.172 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-10T10:24:48.273 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T10:24:48.320 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T10:24:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T10:24:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T10:24:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T10:24:48.480 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T10:24:48.873 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T10:24:48.933 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:24:49.056 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:24:49.085 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:24:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:24:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:24:49.368 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T10:24:49.419 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T10:24:49.447 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:24:49.556 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:24:49.582 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:24:49.670 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:24:49.691 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:24:49.712 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:24:49.761 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-10T10:24:49.848 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T10:24:49.885 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:24:49.886 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T10:24:49.956 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T10:24:49.998 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T10:24:50.024 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T10:24:50.066 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T10:24:50.073 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:24:50.213 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T10:24:50.313 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:24:50.391 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T10:24:50.397 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:24:50.500 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:24:50.615 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:24:50.622 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:24:50.651 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:24:50.719 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T10:24:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:24:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:24:50.814 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T10:24:50.853 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T10:24:50.895 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T10:24:50.909 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T10:24:50.960 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T10:24:50.960 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T10:24:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:24:51.013 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T10:24:51.046 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T10:24:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T10:24:51.092 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:24:51.114 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:24:51.122 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T10:24:51.150 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T10:24:51.177 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:24:51.198 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:24:51.218 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:24:51.218 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T10:24:51.273 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T10:24:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:24:51.462 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T10:24:51.548 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:24:51.630 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T10:24:51.671 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T10:24:51.706 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T10:24:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T10:24:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:24:51.788 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T10:24:51.821 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T10:24:51.848 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:24:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T10:24:51.936 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T10:24:52.011 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T10:24:52.064 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:24:52.143 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T10:24:52.170 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T10:24:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T10:24:52.264 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T10:24:52.302 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T10:24:52.438 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T10:24:52.438 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T10:24:52.463 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T10:24:52.495 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T10:24:52.538 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T10:24:52.571 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T10:24:52.599 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T10:24:52.668 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T10:24:52.717 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T10:24:52.720 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T10:24:52.760 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T10:24:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T10:24:52.809 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T10:24:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T10:24:52.871 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T10:24:52.882 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T10:24:52.905 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T10:24:52.920 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T10:24:52.949 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T10:24:52.962 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T10:24:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T10:24:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T10:24:53.092 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T10:24:53.125 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T10:24:53.167 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T10:24:53.198 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T10:24:53.254 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T10:24:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T10:24:53.326 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T10:24:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T10:24:53.358 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T10:24:53.394 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T10:24:53.441 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T10:24:53.477 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T10:24:53.520 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T10:24:53.562 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T10:24:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T10:24:53.739 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T10:24:53.780 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T10:24:53.881 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T10:24:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T10:24:53.915 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T10:24:53.955 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T10:24:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T10:24:53.980 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T10:24:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T10:24:54.005 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T10:24:54.032 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T10:24:54.070 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T10:24:54.100 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T10:24:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T10:24:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T10:24:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T10:24:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T10:24:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T10:24:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T10:24:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T10:24:54.372 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T10:24:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T10:24:54.403 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T10:24:54.442 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T10:24:54.458 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T10:24:54.477 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T10:24:54.490 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T10:24:54.510 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T10:24:54.526 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T10:24:54.552 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T10:24:54.572 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T10:24:54.586 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T10:24:54.609 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T10:24:54.661 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T10:24:54.673 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T10:24:54.711 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T10:24:54.734 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T10:24:54.765 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T10:24:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T10:24:54.868 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T10:24:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T10:24:54.915 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T10:24:54.937 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T10:24:55.038 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T10:24:55.070 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T10:24:55.118 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T10:24:55.151 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T10:24:55.176 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T10:24:55.251 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T10:24:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T10:24:55.408 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T10:24:55.436 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T10:24:55.470 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T10:24:55.509 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T10:24:55.558 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T10:24:55.587 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T10:24:55.622 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T10:24:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T10:24:55.655 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T10:24:55.680 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T10:24:55.696 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T10:24:55.714 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T10:24:55.731 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T10:24:55.816 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T10:24:55.817 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T10:24:55.861 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T10:24:55.869 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T10:24:55.894 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T10:24:55.922 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T10:24:55.998 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T10:24:56.014 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T10:24:56.046 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T10:24:56.062 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T10:24:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T10:24:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T10:24:56.329 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T10:24:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T10:24:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T10:24:56.884 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T10:24:56.970 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T10:24:56.985 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T10:24:57.048 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T10:24:57.093 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T10:24:57.100 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T10:24:57.189 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T10:24:57.220 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T10:24:57.403 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T10:24:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T10:24:57.511 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T10:24:58.152 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T10:24:58.265 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T10:24:58.331 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T10:24:58.422 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T10:24:58.449 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T10:24:58.483 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T10:24:58.793 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T10:24:58.837 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:24:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:24:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T10:24:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T10:24:59.481 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T10:24:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T10:24:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:24:59.562 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T10:24:59.581 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:24:59.589 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T10:24:59.611 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T10:24:59.622 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T10:24:59.943 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T10:24:59.993 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:25:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:25:00.115 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T10:25:00.153 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T10:25:00.638 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T10:25:00.680 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:25:00.759 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:25:00.781 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T10:25:01.874 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:25:01.908 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:25:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:25:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:25:02.583 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:25:02.584 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T10:25:02.584 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T10:25:02.584 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:02.622 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:25:03.086 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:25:03.127 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:25:03.678 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:25:03.742 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:25:03.834 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:25:03.835 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T10:25:03.835 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T10:25:03.835 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:03.865 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:25:14.682 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:25:14.682 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-09-10T10:25:14.682 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-09-10T10:25:14.682 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-09-10T10:25:14.682 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-09-10T10:25:14.682 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-09-10T10:25:14.682 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-09-10T10:25:14.683 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-09-10T10:25:14.683 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:14.854 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:25:14.872 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:25:14.872 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:14.872 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T10:25:14.872 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:15.493 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:25:15.494 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-09-10T10:25:15.494 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-09-10T10:25:15.494 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-09-10T10:25:15.494 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-09-10T10:25:15.494 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-09-10T10:25:15.494 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-09-10T10:25:15.494 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-09-10T10:25:15.494 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:15.665 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:25:15.681 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:25:15.681 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:15.681 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T10:25:15.681 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:25:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:25:16.384 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:16.384 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T10:25:16.384 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:25:16.385 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:25:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:25:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:25:16.631 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:16.631 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T10:25:16.631 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:25:16.632 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:25:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:25:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:25:17.098 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:17.098 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T10:25:17.099 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:25:17.099 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:25:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:25:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:25:17.151 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:17.151 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T10:25:17.151 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:25:17.151 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:25:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T10:25:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T10:25:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T10:25:17.382 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:25:17.406 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:25:17.406 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:17.406 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T10:25:17.406 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:25:17.406 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:25:17.406 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T10:25:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T10:25:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T10:25:17.711 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T10:25:17.839 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T10:25:17.841 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:25:17.865 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:25:17.865 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:17.865 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T10:25:17.865 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:25:17.865 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:25:17.865 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:17.898 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T10:25:17.934 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T10:25:17.962 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T10:25:17.971 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T10:25:18.028 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T10:25:18.083 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T10:25:18.277 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T10:25:18.333 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T10:25:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T10:25:18.425 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:25:18.452 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:25:18.484 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T10:25:18.500 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T10:25:18.546 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T10:25:18.576 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:25:18.639 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T10:25:18.683 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T10:25:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T10:25:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T10:25:19.182 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:25:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:25:19.242 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T10:25:19.305 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T10:25:19.332 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:25:19.611 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:25:19.653 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:25:20.431 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:25:20.470 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:25:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:25:20.636 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:25:20.738 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:25:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T10:25:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:25:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:25:20.954 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:20.954 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T10:25:20.954 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:25:20.954 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:25:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:20.998 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:25:21.009 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:25:21.108 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T10:25:21.159 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T10:25:21.437 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:25:21.470 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:25:21.573 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:25:21.742 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T10:25:21.772 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:25:21.797 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:25:21.797 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:21.797 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T10:25:21.797 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:25:21.797 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:25:21.797 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:21.848 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:25:21.859 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:25:21.965 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T10:25:22.021 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T10:25:22.089 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T10:25:22.117 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:25:22.140 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:25:22.140 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:22.140 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T10:25:22.140 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:25:22.141 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:25:22.141 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:22.176 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:25:22.198 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:25:22.198 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:22.199 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T10:25:22.199 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:22.244 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:25:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:25:22.266 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:22.266 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T10:25:22.266 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:25:22.266 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:25:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:22.578 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:25:22.600 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:25:22.600 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:22.600 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T10:25:22.600 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:25:22.601 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:25:22.601 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:22.951 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T10:25:22.984 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:25:23.007 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:25:23.008 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:23.008 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T10:25:23.008 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:25:23.008 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:25:23.008 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:23.051 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:25:23.074 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:25:23.074 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:23.074 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T10:25:23.074 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:23.126 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:25:23.151 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:25:23.151 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:23.151 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T10:25:23.151 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:25:23.151 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:25:23.151 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:25:23.484 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:25:23.484 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:25:23.484 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T10:25:23.484 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:25:23.484 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:25:23.484 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:27.149 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T10:25:27.190 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T10:25:27.232 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T10:25:27.263 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T10:25:27.290 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T10:25:27.474 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T10:25:27.521 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T10:25:27.551 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T10:25:27.581 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T10:25:27.627 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T10:25:28.047 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T10:25:28.097 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T10:25:28.138 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T10:25:28.170 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T10:25:28.197 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T10:25:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:25:28.301 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:25:28.321 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:25:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T10:25:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T10:25:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T10:25:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T10:25:28.517 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T10:25:29.140 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:25:29.140 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T10:25:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:25:29.202 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:25:29.240 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T10:25:29.288 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T10:25:29.387 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T10:25:29.419 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T10:25:29.444 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T10:25:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T10:25:29.470 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:25:29.537 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:25:29.538 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:25:29.622 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:25:29.647 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T10:25:29.651 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:25:29.652 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:25:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:25:29.824 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:25:29.917 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:25:29.992 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:25:30.036 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T10:25:30.118 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T10:25:30.169 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T10:25:30.250 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T10:25:30.296 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T10:25:30.330 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T10:25:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T10:25:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:25:30.448 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:25:30.449 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:25:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:25:30.557 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T10:25:30.563 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:25:30.563 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:25:30.742 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:25:30.742 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:25:30.835 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:25:30.898 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:25:35.757 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:25:35.757 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T10:25:35.757 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T10:25:35.758 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T10:25:35.759 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T10:25:35.760 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T10:25:35.773 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T10:25:35.773 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T10:25:35.773 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T10:25:35.773 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T10:25:35.773 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T10:25:35.775 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T10:25:35.775 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T10:25:35.775 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T10:25:35.775 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T10:25:35.775 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T10:25:35.775 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T10:25:35.775 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T10:25:35.775 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T10:25:35.775 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T10:25:35.776 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T10:25:35.777 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T10:25:35.777 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T10:25:35.777 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T10:25:35.777 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T10:25:35.777 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T10:25:35.777 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T10:25:35.777 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T10:25:35.777 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T10:25:35.777 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T10:25:35.777 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T10:25:35.778 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T10:25:35.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T10:25:35.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T10:25:35.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T10:25:35.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T10:25:35.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T10:25:35.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T10:25:35.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T10:25:35.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T10:25:35.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T10:25:35.779 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T10:25:35.780 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T10:25:35.781 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T10:25:35.781 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T10:25:35.781 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T10:25:35.781 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T10:25:35.781 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T10:25:35.781 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T10:25:35.781 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T10:25:35.781 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T10:25:35.781 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T10:25:35.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T10:25:35.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T10:25:35.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T10:25:35.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T10:25:35.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T10:25:35.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T10:25:35.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T10:25:35.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T10:25:35.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T10:25:35.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T10:25:35.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T10:25:35.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T10:25:35.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T10:25:35.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T10:25:35.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T10:25:35.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T10:25:35.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T10:25:35.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T10:25:35.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T10:25:35.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T10:25:35.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T10:25:36.539 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:25:36.539 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T10:25:36.539 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T10:25:36.539 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T10:25:36.539 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T10:25:36.539 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T10:25:36.539 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T10:25:36.540 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T10:25:36.541 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T10:25:36.541 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T10:25:36.541 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T10:25:36.541 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T10:25:36.541 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T10:25:36.541 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T10:25:36.542 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T10:25:36.542 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T10:25:36.542 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T10:25:36.542 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T10:25:36.542 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T10:25:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T10:25:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T10:25:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T10:25:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T10:25:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T10:25:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T10:25:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T10:25:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T10:25:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T10:25:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T10:25:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T10:25:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T10:25:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T10:25:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T10:25:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T10:25:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T10:25:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T10:25:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T10:25:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T10:25:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T10:25:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T10:25:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T10:25:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T10:25:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T10:25:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T10:25:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T10:25:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T10:25:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T10:25:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T10:25:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T10:25:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T10:25:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T10:25:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T10:25:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T10:25:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T10:25:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T10:25:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T10:25:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T10:25:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T10:25:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T10:25:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T10:25:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T10:25:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T10:25:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T10:25:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T10:25:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T10:25:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T10:25:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T10:25:38.437 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:25:38.437 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:38.437 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-09-10T10:25:38.437 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T10:25:38.437 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T10:25:38.437 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T10:25:38.437 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T10:25:38.437 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.438 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.438 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T10:25:38.438 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T10:25:38.438 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-10T10:25:38.438 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T10:25:38.438 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T10:25:38.438 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T10:25:38.438 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T10:25:38.438 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.438 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.439 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.440 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:38.440 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.440 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.440 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.440 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:38.440 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:38.440 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T10:25:38.441 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T10:25:38.442 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.443 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T10:25:38.443 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T10:25:38.443 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T10:25:38.443 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T10:25:38.443 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T10:25:38.443 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T10:25:38.443 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T10:25:38.443 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T10:25:38.443 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T10:25:38.443 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T10:25:38.444 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T10:25:38.445 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T10:25:38.446 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T10:25:38.447 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T10:25:38.448 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T10:25:38.449 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T10:25:38.450 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T10:25:38.450 INFO:teuthology.orchestra.run.smithi120.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T10:25:38.450 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T10:25:38.450 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T10:25:38.450 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T10:25:38.450 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:38.450 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-10T10:25:38.864 DEBUG:teuthology.parallel:result is None 2024-09-10T10:25:39.071 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:25:39.071 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:39.071 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-09-10T10:25:39.071 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T10:25:39.072 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T10:25:39.072 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T10:25:39.072 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T10:25:39.072 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.072 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.073 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T10:25:39.073 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T10:25:39.073 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-10T10:25:39.073 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T10:25:39.073 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T10:25:39.074 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T10:25:39.074 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T10:25:39.074 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.074 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.074 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.075 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.075 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:39.075 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.075 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.075 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.076 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:39.076 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:39.076 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:39.076 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:39.076 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:39.076 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.077 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.077 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:39.077 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.077 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.077 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.078 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:39.078 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:39.078 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.078 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.078 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:25:39.078 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T10:25:39.079 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T10:25:39.079 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T10:25:39.079 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T10:25:39.079 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T10:25:39.079 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T10:25:39.079 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T10:25:39.080 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T10:25:39.080 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T10:25:39.080 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T10:25:39.080 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T10:25:39.080 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.080 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.081 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.081 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T10:25:39.081 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T10:25:39.081 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T10:25:39.081 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T10:25:39.081 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T10:25:39.082 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.082 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.082 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T10:25:39.082 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.082 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T10:25:39.082 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T10:25:39.082 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T10:25:39.083 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T10:25:39.083 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T10:25:39.083 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T10:25:39.083 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T10:25:39.084 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T10:25:39.084 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T10:25:39.084 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T10:25:39.084 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T10:25:39.084 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T10:25:39.084 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T10:25:39.085 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T10:25:39.085 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T10:25:39.085 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T10:25:39.085 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T10:25:39.085 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T10:25:39.085 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T10:25:39.086 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T10:25:39.086 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T10:25:39.086 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T10:25:39.086 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T10:25:39.086 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.086 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.086 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.087 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T10:25:39.087 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T10:25:39.087 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T10:25:39.087 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T10:25:39.087 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T10:25:39.087 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T10:25:39.088 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T10:25:39.088 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T10:25:39.088 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T10:25:39.088 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T10:25:39.088 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T10:25:39.088 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T10:25:39.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T10:25:39.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T10:25:39.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T10:25:39.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T10:25:39.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T10:25:39.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T10:25:39.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T10:25:39.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T10:25:39.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T10:25:39.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T10:25:39.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T10:25:39.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T10:25:39.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T10:25:39.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T10:25:39.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T10:25:39.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T10:25:39.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T10:25:39.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T10:25:39.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T10:25:39.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T10:25:39.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T10:25:39.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T10:25:39.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T10:25:39.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T10:25:39.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T10:25:39.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T10:25:39.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T10:25:39.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T10:25:39.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.094 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.094 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T10:25:39.094 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T10:25:39.095 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T10:25:39.095 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.095 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T10:25:39.095 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T10:25:39.095 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T10:25:39.095 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T10:25:39.096 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T10:25:39.096 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T10:25:39.096 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T10:25:39.096 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T10:25:39.096 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T10:25:39.097 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T10:25:39.097 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T10:25:39.097 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T10:25:39.097 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T10:25:39.097 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T10:25:39.097 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T10:25:39.098 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.098 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.098 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:25:39.098 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T10:25:39.098 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T10:25:39.098 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T10:25:39.098 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T10:25:39.099 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T10:25:39.099 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T10:25:39.099 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T10:25:39.100 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:39.100 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-10T10:25:39.548 DEBUG:teuthology.parallel:result is None 2024-09-10T10:25:39.549 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:25:39.695 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T10:25:39.719 INFO:teuthology.orchestra.run.smithi120.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T10:25:39.720 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T10:25:39.720 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T10:25:39.722 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:25:39.863 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T10:25:39.885 INFO:teuthology.orchestra.run.smithi134.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T10:25:39.885 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T10:25:39.885 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T10:25:39.887 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T10:25:39.887 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:39.887 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T10:25:39.917 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:25:39.917 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T10:25:39.952 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T10:25:39.952 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:39.952 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T10:25:39.983 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T10:25:40.050 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:25:40.050 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T10:25:40.075 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T10:25:40.140 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T10:25:40.141 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:40.141 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T10:25:40.166 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T10:25:40.233 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:25:40.233 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T10:25:40.257 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T10:25:40.322 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T10:25:40.322 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:40.323 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T10:25:40.348 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T10:25:40.414 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:25:40.414 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T10:25:40.438 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T10:25:40.502 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T10:25:40.583 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T10:25:40.584 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /var/log/ceph 2024-09-10T10:25:40.586 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /var/log/ceph 2024-09-10T10:25:40.610 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T10:25:40.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T10:25:40.652 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T10:25:40.676 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T10:25:40.676 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T10:25:40.716 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T10:25:40.748 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T10:25:40.748 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T10:25:40.748 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T10:25:40.749 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:40.749 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T10:25:40.795 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T10:25:40.901 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T10:25:40.926 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:25:40.926 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T10:25:40.942 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T10:25:41.050 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T10:25:41.080 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T10:25:41.081 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T10:25:41.081 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029', 'branch': 'wip-rishabh-testing-20240909.165402-squid', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}}', 'email': None, 'first_in_suite': False, 'job_id': '7899029', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240909.165402-squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240909.165402-squid', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 9893, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'sleep_before_teardown': 0, 'subset': '197/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240909.165402-squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi120.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOE4Q1fM/wqQOf9QV1j/xnQNLV/x1GN7k7t9K3Z12mdTp3KYRictUhGXKeLjLXnD4Ghegb/wveqP0HuZG4Zxnmw=', 'smithi134.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKoFGj0/6EhtvQ0kieazDWYYHO55D7S4rQ2Gfc2u8WY8c0ziZELLsmkvW4mMDpiNT9qqDhzNsQVqPXUp5Azdjks='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'thrashosds': None}, {'workunit': {'clients': {'all': ['fs/misc/trivial_sync.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_06:58:10', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T10:25:41.081 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T10:25:41.084 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T10:25:41.130 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T10:25:41.139 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T10:25:41.194 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:41.195 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T10:25:41.217 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T10:25:41.218 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-09-10T10:25:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T10:25:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:25:41.273 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-10T10:25:41.273 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:25:41.273 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:25:41.273 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 10:25:30.332664910 +0000 2024-09-10T10:25:41.273 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 10:23:13.525535539 +0000 2024-09-10T10:25:41.273 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 10:23:13.525535539 +0000 2024-09-10T10:25:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 10:23:13.525535539 +0000 2024-09-10T10:25:41.274 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T10:25:41.336 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T10:25:41.336 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T10:25:41.336 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000185468 s, 2.8 MB/s 2024-09-10T10:25:41.337 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T10:25:41.393 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-09-10T10:25:41.448 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T10:25:41.448 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:25:41.448 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-10T10:25:41.448 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:25:41.448 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:25:41.448 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 10:25:30.333664888 +0000 2024-09-10T10:25:41.449 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 10:23:13.856528154 +0000 2024-09-10T10:25:41.449 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 10:23:13.856528154 +0000 2024-09-10T10:25:41.449 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 10:23:13.856528154 +0000 2024-09-10T10:25:41.449 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T10:25:41.511 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T10:25:41.511 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T10:25:41.511 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.0002375 s, 2.2 MB/s 2024-09-10T10:25:41.512 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T10:25:41.569 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-09-10T10:25:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T10:25:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:25:41.624 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-10T10:25:41.624 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:25:41.624 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:25:41.624 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 10:25:30.333664888 +0000 2024-09-10T10:25:41.624 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 10:23:14.178520970 +0000 2024-09-10T10:25:41.624 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 10:23:14.178520970 +0000 2024-09-10T10:25:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 10:23:14.178520970 +0000 2024-09-10T10:25:41.625 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T10:25:41.687 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T10:25:41.687 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T10:25:41.687 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000251262 s, 2.0 MB/s 2024-09-10T10:25:41.688 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T10:25:41.744 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-09-10T10:25:41.799 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T10:25:41.799 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:25:41.799 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T10:25:41.799 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:25:41.799 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:25:41.799 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 10:25:30.334664866 +0000 2024-09-10T10:25:41.800 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 10:23:14.504513696 +0000 2024-09-10T10:25:41.800 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 10:23:14.504513696 +0000 2024-09-10T10:25:41.800 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 10:23:14.504513696 +0000 2024-09-10T10:25:41.800 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T10:25:41.862 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T10:25:41.862 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T10:25:41.862 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000161734 s, 3.2 MB/s 2024-09-10T10:25:41.863 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T10:25:41.919 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T10:25:41.919 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:25:41.919 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T10:25:41.937 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T10:25:41.937 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-09-10T10:25:41.993 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T10:25:41.994 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:25:41.994 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T10:25:41.994 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:25:41.994 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:25:41.994 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-10 10:25:31.238458863 +0000 2024-09-10T10:25:41.994 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-10 10:23:29.499489228 +0000 2024-09-10T10:25:41.994 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-10 10:23:29.499489228 +0000 2024-09-10T10:25:41.994 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-10 10:23:29.499489228 +0000 2024-09-10T10:25:41.994 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T10:25:42.059 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-10T10:25:42.059 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-10T10:25:42.059 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000271123 s, 1.9 MB/s 2024-09-10T10:25:42.060 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T10:25:42.118 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-09-10T10:25:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T10:25:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:25:42.175 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T10:25:42.175 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:25:42.175 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:25:42.175 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-10 10:25:31.238458863 +0000 2024-09-10T10:25:42.175 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-10 10:23:29.842480658 +0000 2024-09-10T10:25:42.175 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-10 10:23:29.842480658 +0000 2024-09-10T10:25:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-10 10:23:29.842480658 +0000 2024-09-10T10:25:42.176 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T10:25:42.239 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-10T10:25:42.240 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-10T10:25:42.240 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000190657 s, 2.7 MB/s 2024-09-10T10:25:42.241 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T10:25:42.297 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-09-10T10:25:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T10:25:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:25:42.353 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-10T10:25:42.353 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:25:42.354 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:25:42.354 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-10 10:25:31.238458863 +0000 2024-09-10T10:25:42.354 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-10 10:23:30.164472614 +0000 2024-09-10T10:25:42.354 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-10 10:23:30.164472614 +0000 2024-09-10T10:25:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-10 10:23:30.164472614 +0000 2024-09-10T10:25:42.354 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T10:25:42.417 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-10T10:25:42.417 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-10T10:25:42.417 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000187934 s, 2.7 MB/s 2024-09-10T10:25:42.418 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T10:25:42.474 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-09-10T10:25:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T10:25:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:25:42.529 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-10T10:25:42.529 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:25:42.529 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:25:42.529 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-10 10:25:31.239458838 +0000 2024-09-10T10:25:42.529 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-10 10:23:30.502464169 +0000 2024-09-10T10:25:42.529 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-10 10:23:30.502464169 +0000 2024-09-10T10:25:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-10 10:23:30.502464169 +0000 2024-09-10T10:25:42.530 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T10:25:42.592 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-10T10:25:42.592 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-10T10:25:42.592 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000250485 s, 2.0 MB/s 2024-09-10T10:25:42.594 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T10:25:42.650 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T10:25:42.650 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T10:25:42.650 INFO:tasks.ceph:Generating config... 2024-09-10T10:25:42.651 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-10T10:25:42.651 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-09-10T10:25:42.651 INFO:tasks.ceph:[client] debug client = 20 2024-09-10T10:25:42.652 INFO:tasks.ceph:[client] debug ms = 1 2024-09-10T10:25:42.652 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-10T10:25:42.652 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-10T10:25:42.652 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-10T10:25:42.652 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-10T10:25:42.652 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-10T10:25:42.652 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-10T10:25:42.652 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-09-10T10:25:42.652 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-09-10T10:25:42.652 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-09-10T10:25:42.652 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-09-10T10:25:42.652 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T10:25:42.653 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T10:25:42.654 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T10:25:42.654 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T10:25:42.654 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-10T10:25:42.654 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-10T10:25:42.654 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T10:25:42.654 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-10T10:25:42.654 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T10:25:42.654 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-10T10:25:42.654 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-10T10:25:42.654 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T10:25:42.654 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T10:25:42.654 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-10T10:25:42.654 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T10:25:42.654 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T10:25:42.655 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T10:25:42.655 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-10T10:25:42.655 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T10:25:42.655 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-10T10:25:42.655 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T10:25:42.655 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-10T10:25:42.655 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T10:25:42.655 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T10:25:42.712 INFO:teuthology.orchestra.run.smithi120.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T10:25:42.714 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T10:25:42.758 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T10:25:42.823 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.120'), ('mon.b', '172.21.15.134'), ('mon.c', '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]')] 2024-09-10T10:25:42.823 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.120,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-10T10:25:42.824 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:42.824 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T10:25:42.878 DEBUG:teuthology.orchestra.run.smithi120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.120 --add b 172.21.15.134 --addv c '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:monmaptool: generated fsid 7d0d0830-4370-41c3-9acc-b67b07024f68 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:setting min_mon_release = quincy 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:epoch 0 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:fsid 7d0d0830-4370-41c3-9acc-b67b07024f68 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:last_changed 2024-09-10T10:25:42.963671+0000 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:created 2024-09-10T10:25:42.963671+0000 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:min_mon_release 17 (quincy) 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:election_strategy: 1 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-10T10:25:42.964 INFO:teuthology.orchestra.run.smithi120.stdout:1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-10T10:25:42.965 INFO:teuthology.orchestra.run.smithi120.stdout:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.c 2024-09-10T10:25:42.965 INFO:teuthology.orchestra.run.smithi120.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T10:25:42.966 DEBUG:teuthology.orchestra.run.smithi120:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T10:25:42.980 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7d0d0830-4370-41c3-9acc-b67b07024f68... 2024-09-10T10:25:42.981 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T10:25:43.023 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T10:25:43.063 INFO:teuthology.orchestra.run.smithi120.stdout:[global] 2024-09-10T10:25:43.063 INFO:teuthology.orchestra.run.smithi120.stdout: chdir = "" 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: auth supported = cephx 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: filestore xattr use omap = true 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: mon clock drift allowed = 1.000 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: osd crush chooseleaf type = 0 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: auth debug = true 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on old message = true 2024-09-10T10:25:43.064 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on bug = true 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: mon pg warn max object skew = 0 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default size = 2 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow primary affinity = true 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow pg remap = true 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on legacy crush tunables = false 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on crush straw calc version zero = false 2024-09-10T10:25:43.065 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on no sortbitwise = false 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on osd down out interval zero = false 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on too few osds = false 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: mon_allow_pool_size_one = true 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: osd default data pool replay window = 5 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: mon allow pool delete = true 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: mon cluster log file level = debug 2024-09-10T10:25:43.066 INFO:teuthology.orchestra.run.smithi120.stdout: debug asserts on shutdown = true 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: mon health detail to clog = false 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: mon host = "172.21.15.120,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on skipped message = False 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: fsid = 7d0d0830-4370-41c3-9acc-b67b07024f68 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout:[osd] 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: osd journal size = 100 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub load threshold = 5.0 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub max interval = 600 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: osd mclock profile = high_recovery_ops 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: osd recover clone overlap = true 2024-09-10T10:25:43.067 INFO:teuthology.orchestra.run.smithi120.stdout: osd recovery max chunk = 1048576 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug shutdown = true 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug op order = true 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug verify stray on activate = true 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug trim objects = true 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: osd open classes on start = true 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug pg log writeout = true 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: osd deep scrub update digest min age = 30 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: osd map max advance = 10 2024-09-10T10:25:43.068 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: journal zero on create = true 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: filestore ondisk finisher threads = 3 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: filestore apply finisher threads = 3 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: bdev debug aio = true 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug misdirected ops = true 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore block size = 96636764160 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore compression mode = aggressive 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore fsck on mount = True 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluefs = 20 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluestore = 20 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-09-10T10:25:43.069 INFO:teuthology.orchestra.run.smithi120.stdout: debug osd = 20 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: debug rocksdb = 10 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd full ratio = 0.9 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd nearfull ratio = 0.8 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: osd blocked scrub grace period = 3600 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: osd failsafe full ratio = 0.95 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: osd max markdown count = 1000 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: osd objectstore = bluestore 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: osd op complaint time = 180 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr] 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: debug mgr = 20 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-09-10T10:25:43.070 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: mgr/telemetry/nag = false 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: debug client = 20 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout:[mon] 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: debug paxos = 20 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: mon data avail warn = 5 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: mon mgr mkfs grace = 240 2024-09-10T10:25:43.071 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd reporter subtree level = osd 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd prime pg temp = true 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: # 1m isn't quite enough 2024-09-10T10:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout: mon_down_mkfs_grace = 2m 2024-09-10T10:25:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_filestore_osds = false 2024-09-10T10:25:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: mon down mkfs grace = 300 2024-09-10T10:25:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: mon op complaint time = 120 2024-09-10T10:25:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on pool no app = False 2024-09-10T10:25:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:25:43.073 INFO:teuthology.orchestra.run.smithi120.stdout:[client] 2024-09-10T10:25:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: rgw cache enabled = true 2024-09-10T10:25:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable ops log = true 2024-09-10T10:25:43.073 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable usage log = true 2024-09-10T10:25:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T10:25:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T10:25:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: client mount timeout = 600 2024-09-10T10:25:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: client_shutdown_timeout = 120 2024-09-10T10:25:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: debug client = 20 2024-09-10T10:25:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-09-10T10:25:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: rados mon op timeout = 900 2024-09-10T10:25:43.074 INFO:teuthology.orchestra.run.smithi120.stdout: rados osd op timeout = 900 2024-09-10T10:25:43.074 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.a] 2024-09-10T10:25:43.075 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.b] 2024-09-10T10:25:43.075 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.c] 2024-09-10T10:25:43.075 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.a] 2024-09-10T10:25:43.075 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.c] 2024-09-10T10:25:43.075 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.e] 2024-09-10T10:25:43.075 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.b] 2024-09-10T10:25:43.075 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.d] 2024-09-10T10:25:43.075 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.f] 2024-09-10T10:25:43.075 INFO:teuthology.orchestra.run.smithi120.stdout:[mds] 2024-09-10T10:25:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: debug mds = 20 2024-09-10T10:25:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: debug mds balancer = 20 2024-09-10T10:25:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-09-10T10:25:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: mds bal fragment size max = 10000 2024-09-10T10:25:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: mds bal merge size = 5 2024-09-10T10:25:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: mds bal split bits = 3 2024-09-10T10:25:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: mds bal split size = 100 2024-09-10T10:25:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: mds debug frag = True 2024-09-10T10:25:43.076 INFO:teuthology.orchestra.run.smithi120.stdout: mds debug scatterstat = True 2024-09-10T10:25:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: mds oft prefetch dirfrags = True 2024-09-10T10:25:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: mds op complaint time = 180 2024-09-10T10:25:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: mds verify scatter = True 2024-09-10T10:25:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: mds_dir_prefetch = True 2024-09-10T10:25:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: mds_sleep_rank_change = 5000000.0 2024-09-10T10:25:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: osd op complaint time = 180 2024-09-10T10:25:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: rados mon op timeout = 900 2024-09-10T10:25:43.077 INFO:teuthology.orchestra.run.smithi120.stdout: rados osd op timeout = 900 2024-09-10T10:25:43.079 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-09-10T10:25:43.079 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-09-10T10:25:43.079 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T10:25:43.079 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-09-10T10:25:43.079 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.079 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-09-10T10:25:43.079 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.079 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T10:25:43.080 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.081 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.120,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on skipped message = False 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = 7d0d0830-4370-41c3-9acc-b67b07024f68 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-09-10T10:25:43.082 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug trim objects = true 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.083 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-09-10T10:25:43.084 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore compression mode = aggressive 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 20 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 20 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 10 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: osd blocked scrub grace period = 3600 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-09-10T10:25:43.085 INFO:teuthology.orchestra.run.smithi134.stdout: osd max markdown count = 1000 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.086 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T10:25:43.087 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: mon down mkfs grace = 300 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: mon op complaint time = 120 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on pool no app = False 2024-09-10T10:25:43.088 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: client mount timeout = 600 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: client_shutdown_timeout = 120 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-09-10T10:25:43.089 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.e] 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.f] 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds = 20 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds balancer = 20 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal fragment size max = 10000 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal merge size = 5 2024-09-10T10:25:43.090 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal split bits = 3 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal split size = 100 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug frag = True 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug scatterstat = True 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: mds oft prefetch dirfrags = True 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: mds op complaint time = 180 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = True 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: mds_dir_prefetch = True 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: mds_sleep_rank_change = 5000000.0 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2024-09-10T10:25:43.091 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2024-09-10T10:25:43.091 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T10:25:43.092 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T10:25:43.173 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T10:25:43.174 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:43.174 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T10:25:43.189 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:43.189 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T10:25:43.244 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:25:43.244 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:43.244 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T10:25:43.244 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T10:25:43.318 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:25:43.318 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:25:43.373 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:25:43.373 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:25:43.373 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T10:25:43.374 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T10:25:43.409 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:25:43.409 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:25:43.464 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T10:25:43.464 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T10:25:43.465 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T10:25:43.515 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T10:25:43.518 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T10:25:43.573 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T10:25:43.576 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T10:25:43.576 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T10:25:43.628 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T10:25:43.631 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T10:25:43.655 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T10:25:43.745 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T10:25:43.748 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T10:25:43.772 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-09-10T10:25:43.862 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-09-10T10:25:43.864 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-09-10T10:25:43.888 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T10:25:43.940 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T10:25:43.943 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T10:25:43.967 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T10:25:44.057 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T10:25:44.060 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T10:25:44.084 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-09-10T10:25:44.175 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-09-10T10:25:44.178 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-09-10T10:25:44.202 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T10:25:44.202 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T10:25:44.244 INFO:teuthology.orchestra.run.smithi120.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T10:25:44.257 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T10:25:44.257 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T10:25:44.258 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T10:25:44.321 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T10:25:44.321 INFO:tasks.ceph:role: osd.0 2024-09-10T10:25:44.321 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:25:44.322 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T10:25:44.394 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:25:44.394 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:25:44.394 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:25:44.394 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:25:44.394 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:25:44.395 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:25:44.395 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:25:44.395 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:25:44.395 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:25:44.395 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:25:44.883 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-09-10T10:25:44.885 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-09-10T10:25:44.885 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T10:25:44.989 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T10:25:45.022 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T10:25:45.087 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T10:25:45.087 INFO:tasks.ceph:role: osd.1 2024-09-10T10:25:45.087 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:25:45.087 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T10:25:45.159 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:25:45.159 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:25:45.159 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:25:45.159 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:25:45.159 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:25:45.159 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:25:45.160 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:25:45.160 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:25:45.160 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:25:45.160 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:25:45.621 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-09-10T10:25:45.623 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-09-10T10:25:45.623 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T10:25:45.700 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T10:25:45.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T10:25:45.834 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T10:25:45.834 INFO:tasks.ceph:role: osd.2 2024-09-10T10:25:45.834 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:25:45.834 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T10:25:45.906 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:25:45.906 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:25:45.906 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:25:45.906 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:25:45.906 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:25:45.906 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:25:45.906 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:25:45.906 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:25:45.906 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:25:45.906 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:25:46.392 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-09-10T10:25:46.394 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-09-10T10:25:46.394 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T10:25:46.484 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T10:25:46.513 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T10:25:46.578 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T10:25:46.579 INFO:tasks.ceph:role: osd.3 2024-09-10T10:25:46.579 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:25:46.579 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T10:25:46.651 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:25:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:25:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:25:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:25:46.652 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:25:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:25:46.652 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:25:46.652 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:25:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:25:46.652 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:25:47.172 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-09-10T10:25:47.174 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-09-10T10:25:47.174 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T10:25:47.325 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T10:25:47.359 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:25:47.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:47.456+0000 7fd33bcdc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T10:25:47.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:47.457+0000 7fd33bcdc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T10:25:47.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:47.457+0000 7fd33bcdc640 -1 bdev(0x5633e2fbd800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T10:25:47.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:47.457+0000 7fd33bcdc640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T10:25:50.614 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T10:25:50.643 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:25:50.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:50.732+0000 7f0c037b0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T10:25:50.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:50.732+0000 7f0c037b0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T10:25:50.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:50.732+0000 7f0c037b0640 -1 bdev(0x559a8e26b800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T10:25:50.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:50.732+0000 7f0c037b0640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T10:25:53.636 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T10:25:53.665 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:25:53.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:53.754+0000 7fcfcf3bb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T10:25:53.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:53.754+0000 7fcfcf3bb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T10:25:53.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:53.754+0000 7fcfcf3bb640 -1 bdev(0x555f292f7800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T10:25:53.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:53.755+0000 7fcfcf3bb640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T10:25:56.658 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T10:25:56.686 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:25:56.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:56.773+0000 7f193340e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T10:25:56.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:56.773+0000 7f193340e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T10:25:56.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:56.773+0000 7f193340e640 -1 bdev(0x5555b29d1800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T10:25:56.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:25:56.773+0000 7f193340e640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T10:25:59.688 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T10:25:59.717 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T10:25:59.748 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T10:25:59.748 INFO:tasks.ceph:role: osd.4 2024-09-10T10:25:59.748 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:25:59.748 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T10:25:59.834 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:25:59.834 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:25:59.834 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:25:59.834 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:25:59.834 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:25:59.834 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:25:59.834 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:25:59.834 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:25:59.835 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:25:59.835 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:26:00.350 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-10T10:26:00.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-10T10:26:00.353 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T10:26:00.426 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T10:26:00.458 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T10:26:00.523 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T10:26:00.523 INFO:tasks.ceph:role: osd.5 2024-09-10T10:26:00.523 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:26:00.523 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T10:26:00.596 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:26:00.596 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:26:00.596 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:26:00.596 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:26:00.596 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:26:00.596 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:26:00.597 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:26:00.597 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:26:00.597 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:26:00.597 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:26:01.049 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-10T10:26:01.052 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-10T10:26:01.052 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T10:26:01.124 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T10:26:01.193 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T10:26:01.259 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T10:26:01.259 INFO:tasks.ceph:role: osd.6 2024-09-10T10:26:01.259 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:26:01.259 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T10:26:01.330 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:26:01.331 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:26:01.331 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:26:01.331 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:26:01.331 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:26:01.331 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:26:01.331 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:26:01.331 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:26:01.331 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:26:01.331 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:26:01.814 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-10T10:26:01.816 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-10T10:26:01.816 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T10:26:01.919 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T10:26:01.948 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T10:26:02.014 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T10:26:02.014 INFO:tasks.ceph:role: osd.7 2024-09-10T10:26:02.014 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:26:02.014 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T10:26:02.086 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:26:02.086 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:26:02.087 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:26:02.087 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:26:02.087 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:26:02.087 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:26:02.087 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:26:02.087 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:26:02.087 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:26:02.087 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:26:02.580 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-10T10:26:02.583 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-10T10:26:02.583 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T10:26:02.737 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T10:26:02.770 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:26:02.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:02.869+0000 7f2038556640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T10:26:02.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:02.874+0000 7f2038556640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T10:26:02.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:02.874+0000 7f2038556640 -1 bdev(0x559ed46bf800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T10:26:02.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:02.874+0000 7f2038556640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T10:26:05.788 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T10:26:05.815 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:26:05.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:05.902+0000 7f6eeada8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T10:26:05.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:05.902+0000 7f6eeada8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T10:26:05.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:05.902+0000 7f6eeada8640 -1 bdev(0x55b295161800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T10:26:05.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:05.902+0000 7f6eeada8640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T10:26:08.823 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T10:26:08.849 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:26:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:08.936+0000 7ff288bdd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T10:26:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:08.937+0000 7ff288bdd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T10:26:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:08.937+0000 7ff288bdd640 -1 bdev(0x556bc009d800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T10:26:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:08.937+0000 7ff288bdd640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T10:26:11.082 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:26:11.085 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:26:11.844 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T10:26:11.870 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:26:11.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:11.954+0000 7f421d817640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T10:26:11.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:11.954+0000 7f421d817640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T10:26:11.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:11.954+0000 7f421d817640 -1 bdev(0x55d4d4669800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T10:26:11.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:11.954+0000 7f421d817640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T10:26:15.112 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T10:26:15.142 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T10:26:15.142 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:15.142 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T10:26:15.172 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:15.172 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T10:26:15.240 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:15.240 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T10:26:15.307 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:15.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-09-10T10:26:15.373 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:15.374 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T10:26:15.440 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:15.440 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T10:26:15.506 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:15.506 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T10:26:15.572 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:15.572 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T10:26:15.637 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:15.638 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T10:26:15.668 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:15.668 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T10:26:15.737 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:15.737 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T10:26:15.804 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:15.804 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-09-10T10:26:15.871 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:15.871 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T10:26:15.937 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:15.937 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T10:26:16.003 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:16.003 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T10:26:16.068 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:16.068 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T10:26:16.133 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:16.133 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T10:26:16.150 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T10:26:16.150 DEBUG:teuthology.orchestra.run.smithi120:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T10:26:16.192 DEBUG:teuthology.orchestra.run.smithi134:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T10:26:16.216 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.y] 2024-09-10T10:26:16.216 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCnHuBmYwW4HhAASoj7TKEy/+bXV6+2i/s1YQ== 2024-09-10T10:26:16.216 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-09-10T10:26:16.216 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCnHuBmXlF2JRAAPDQkZTvYNlTQw9k4ZmmodA== 2024-09-10T10:26:16.216 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-09-10T10:26:16.216 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCnHuBmcotpLBAAFO2GTLudTluEvZzkA+Ex7g== 2024-09-10T10:26:16.216 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.e] 2024-09-10T10:26:16.217 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCnHuBmvdFiMxAAGmhbMTVtb4kbi29M/mdPEA== 2024-09-10T10:26:16.217 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.0] 2024-09-10T10:26:16.217 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCrHuBmSdJEGxAAJnVAOkSvc8fx6NszKG9DtQ== 2024-09-10T10:26:16.217 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.1] 2024-09-10T10:26:16.217 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCuHuBmNgO1KxAACjCuaz5nbiElfcZlKaT5ug== 2024-09-10T10:26:16.217 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.2] 2024-09-10T10:26:16.218 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCxHuBmskoHLRAAeFbDjgdkXOvJ77lubuck0w== 2024-09-10T10:26:16.218 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.3] 2024-09-10T10:26:16.218 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQC0HuBmq5MhLhAAdhmuc5MW5QCMIAm8fURaXw== 2024-09-10T10:26:16.218 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.x] 2024-09-10T10:26:16.218 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCnHuBmGccxIhAANP7RXYTIrp2QQLP9PmmXiQ== 2024-09-10T10:26:16.218 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-09-10T10:26:16.218 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCnHuBmDhYNOBAAw0L3waGg9+pWsYUahPrF2Q== 2024-09-10T10:26:16.218 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-09-10T10:26:16.219 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCoHuBmUdhwAxAAcLLLmnOpdDUdnKrQ+ZQA0w== 2024-09-10T10:26:16.219 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.f] 2024-09-10T10:26:16.219 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCoHuBm4A52ChAAElAkhCZkDVpD9uNiSdD3LA== 2024-09-10T10:26:16.219 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.4] 2024-09-10T10:26:16.219 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQC6HuBmkknfMxAAZ92v4no1ipZJywg9QVTcAw== 2024-09-10T10:26:16.219 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.5] 2024-09-10T10:26:16.219 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQC9HuBm69fTNRAAmVUhGYWN17VMGvEILqNXTg== 2024-09-10T10:26:16.220 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.6] 2024-09-10T10:26:16.220 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDAHuBmxhDjNxAAte8nAt9GQl9Zzrc8NU+hmg== 2024-09-10T10:26:16.220 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.7] 2024-09-10T10:26:16.220 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDDHuBmaCDsOBAAZ+vWeP5PU8tOaYnIzuP7NA== 2024-09-10T10:26:16.220 INFO:teuthology.orchestra.run.smithi134.stdout:[client.0] 2024-09-10T10:26:16.220 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCoHuBmJLmUDhAAOHQnZZF0AlCTNmUlLMfApg== 2024-09-10T10:26:16.222 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.y] 2024-09-10T10:26:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCnHuBmYwW4HhAASoj7TKEy/+bXV6+2i/s1YQ== 2024-09-10T10:26:16.222 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.a] 2024-09-10T10:26:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCnHuBmXlF2JRAAPDQkZTvYNlTQw9k4ZmmodA== 2024-09-10T10:26:16.222 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.c] 2024-09-10T10:26:16.222 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCnHuBmcotpLBAAFO2GTLudTluEvZzkA+Ex7g== 2024-09-10T10:26:16.222 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.e] 2024-09-10T10:26:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCnHuBmvdFiMxAAGmhbMTVtb4kbi29M/mdPEA== 2024-09-10T10:26:16.223 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.0] 2024-09-10T10:26:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCrHuBmSdJEGxAAJnVAOkSvc8fx6NszKG9DtQ== 2024-09-10T10:26:16.223 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.1] 2024-09-10T10:26:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCuHuBmNgO1KxAACjCuaz5nbiElfcZlKaT5ug== 2024-09-10T10:26:16.223 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.2] 2024-09-10T10:26:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCxHuBmskoHLRAAeFbDjgdkXOvJ77lubuck0w== 2024-09-10T10:26:16.223 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.3] 2024-09-10T10:26:16.223 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQC0HuBmq5MhLhAAdhmuc5MW5QCMIAm8fURaXw== 2024-09-10T10:26:16.224 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.x] 2024-09-10T10:26:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCnHuBmGccxIhAANP7RXYTIrp2QQLP9PmmXiQ== 2024-09-10T10:26:16.224 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.b] 2024-09-10T10:26:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCnHuBmDhYNOBAAw0L3waGg9+pWsYUahPrF2Q== 2024-09-10T10:26:16.224 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.d] 2024-09-10T10:26:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCoHuBmUdhwAxAAcLLLmnOpdDUdnKrQ+ZQA0w== 2024-09-10T10:26:16.224 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.f] 2024-09-10T10:26:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCoHuBm4A52ChAAElAkhCZkDVpD9uNiSdD3LA== 2024-09-10T10:26:16.224 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.4] 2024-09-10T10:26:16.224 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQC6HuBmkknfMxAAZ92v4no1ipZJywg9QVTcAw== 2024-09-10T10:26:16.225 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.5] 2024-09-10T10:26:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQC9HuBm69fTNRAAmVUhGYWN17VMGvEILqNXTg== 2024-09-10T10:26:16.225 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.6] 2024-09-10T10:26:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQDAHuBmxhDjNxAAte8nAt9GQl9Zzrc8NU+hmg== 2024-09-10T10:26:16.225 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.7] 2024-09-10T10:26:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQDDHuBmaCDsOBAAZ+vWeP5PU8tOaYnIzuP7NA== 2024-09-10T10:26:16.225 INFO:teuthology.orchestra.run.smithi120.stdout:[client.0] 2024-09-10T10:26:16.225 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQCoHuBmJLmUDhAAOHQnZZF0AlCTNmUlLMfApg== 2024-09-10T10:26:16.226 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T10:26:16.265 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T10:26:16.317 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.319 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.373 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.376 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.429 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.432 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.486 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.488 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.538 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.541 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.592 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.594 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.646 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.648 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.699 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T10:26:16.701 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T10:26:16.752 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.754 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.807 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.810 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.864 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.867 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:26:16.921 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.923 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.974 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:16.976 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:17.028 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:17.030 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:17.080 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:17.082 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:26:17.133 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T10:26:17.135 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T10:26:17.188 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T10:26:17.189 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T10:26:17.213 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T10:26:17.311 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T10:26:17.336 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T10:26:17.361 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T10:26:17.457 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T10:26:17.482 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T10:26:17.547 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T10:26:17.642 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T10:26:17.666 DEBUG:teuthology.orchestra.run.smithi120:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:26:17.669 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:26:17.722 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T10:26:17.722 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T10:26:17.723 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T10:26:17.725 INFO:tasks.ceph.mon.a:Started 2024-09-10T10:26:17.725 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T10:26:17.725 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T10:26:17.765 INFO:tasks.ceph.mon.b:Started 2024-09-10T10:26:17.765 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T10:26:17.765 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T10:26:17.767 INFO:tasks.ceph.mon.c:Started 2024-09-10T10:26:17.767 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T10:26:17.768 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T10:26:17.768 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T10:26:17.770 INFO:tasks.ceph.mgr.y:Started 2024-09-10T10:26:17.770 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T10:26:17.770 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T10:26:17.772 INFO:tasks.ceph.mgr.x:Started 2024-09-10T10:26:17.772 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T10:26:17.772 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T10:26:17.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.870+0000 7f04c265e640 1 Processor -- start 2024-09-10T10:26:17.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.870+0000 7f04c265e640 1 -- start start 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04c265e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f04bc154de0 0x7f04bc0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04c265e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04bc0d0dd0 con 0x7f04bc15e0d0 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04c265e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04bc0cb5f0 con 0x7f04bc0d02d0 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04c265e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f04bc15bb40 con 0x7f04bc154de0 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04bbfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f04bc154de0 0x7f04bc0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04bbfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f04bc154de0 0x7f04bc0cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52552/0 (socket says 172.21.15.120:52552) 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04b3fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f04bc15e0d0 0x7f04bc15ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:42246/0 (socket says 172.21.15.120:42246) 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04bbfff640 1 -- 172.21.15.120:0/630757 learned_addr learned my addr 172.21.15.120:0/630757 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04bbfff640 1 -- 172.21.15.120:0/630757 >> v2:172.21.15.120:3300/0 conn(0x7f04bc154de0 msgr2=0x7f04bc0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04bbfff640 1 -- 172.21.15.120:0/630757 >> v2:172.21.15.120:3300/0 conn(0x7f04bc154de0 msgr2=0x7f04bc0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04bbfff640 1 --2- 172.21.15.120:0/630757 >> v2:172.21.15.120:3300/0 conn(0x7f04bc154de0 0x7f04bc0cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:26:17.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:17.871+0000 7f04bbfff640 1 --2- 172.21.15.120:0/630757 >> v2:172.21.15.120:3300/0 conn(0x7f04bc154de0 0x7f04bc0cc090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:26:18.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3895887134 0 0) 0x7f04a4001110 con 0x7f04bc15e0d0 2024-09-10T10:26:18.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04a8003520 con 0x7f04bc15e0d0 2024-09-10T10:26:18.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 847530411 0 0) 0x7f04bc0cb5f0 con 0x7f04bc0d02d0 2024-09-10T10:26:18.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04a4001110 con 0x7f04bc0d02d0 2024-09-10T10:26:18.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1545020449 0 0) 0x7f04a8003520 con 0x7f04bc15e0d0 2024-09-10T10:26:18.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 >> v2:172.21.15.120:3300/0 conn(0x7f04bc154de0 msgr2=0x7f04bc0cc090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:26:18.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 --2- 172.21.15.120:0/630757 >> v2:172.21.15.120:3300/0 conn(0x7f04bc154de0 0x7f04bc0cc090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 >> v1:172.21.15.134:6789/0 conn(0x7f04bc0d02d0 legacy=0x7f04bc165400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc15b580 con 0x7f04bc15e0d0 2024-09-10T10:26:18.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f04a4001840 con 0x7f04bc15e0d0 2024-09-10T10:26:18.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 >> v1:172.21.15.120:6789/0 conn(0x7f04bc15e0d0 legacy=0x7f04bc15ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.027+0000 7f04bb7fe640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a8003b90 0x7f04a8003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04bb7fe640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8004800 0x7f04a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04bb7fe640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04a8005210 0x7f04a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc15bb40 con 0x7f04a8005210 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04a800c140 con 0x7f04a8003b90 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f04a800c340 con 0x7f04a8004800 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04c0bfb640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04a8005210 0x7f04a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04b3fff640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8004800 0x7f04a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04bbfff640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a8003b90 0x7f04a8003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04c0bfb640 1 -- 172.21.15.120:0/630757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8004800 msgr2=0x7f04a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04c0bfb640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8004800 0x7f04a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04c0bfb640 1 -- 172.21.15.120:0/630757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a8003b90 msgr2=0x7f04a8003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04c0bfb640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a8003b90 0x7f04a8003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.028+0000 7f04c0bfb640 1 -- 172.21.15.120:0/630757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a8003520 con 0x7f04a8005210 2024-09-10T10:26:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.029+0000 7f04c0bfb640 1 -- 172.21.15.120:0/630757 reap_dead start 2024-09-10T10:26:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.029+0000 7f04c0bfb640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04a8005210 0x7f04a800b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f04b4001fa0 tx=0x7f04a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36c1bf66f92a21c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.030+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4001bf0 con 0x7f04a8005210 2024-09-10T10:26:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.030+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b4003b70 con 0x7f04a8005210 2024-09-10T10:26:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.030+0000 7f04bb7fe640 1 -- 172.21.15.120:0/630757 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b40050d0 con 0x7f04a8005210 2024-09-10T10:26:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.031+0000 7f04c265e640 1 -- 172.21.15.120:0/630757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04a8005210 msgr2=0x7f04a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.031+0000 7f04c265e640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04a8005210 0x7f04a800b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f04b4001fa0 tx=0x7f04a8005830 comp rx=0 tx=0).stop 2024-09-10T10:26:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.031+0000 7f04c265e640 1 -- 172.21.15.120:0/630757 shutdown_connections 2024-09-10T10:26:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.031+0000 7f04c265e640 1 --2- 172.21.15.120:0/630757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04a8005210 0x7f04a800b9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.031+0000 7f04c265e640 1 -- 172.21.15.120:0/630757 >> 172.21.15.120:0/630757 conn(0x7f04bc06cab0 msgr2=0x7f04bc06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.031+0000 7f04c265e640 1 -- 172.21.15.120:0/630757 shutdown_connections 2024-09-10T10:26:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.031+0000 7f04c265e640 1 -- 172.21.15.120:0/630757 wait complete. 2024-09-10T10:26:18.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.032+0000 7f04c265e640 1 Processor -- start 2024-09-10T10:26:18.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.032+0000 7f04c265e640 1 -- start start 2024-09-10T10:26:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04c265e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8005210 0x7f04bc14a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04c265e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04bc151040 0x7f04bc14aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04c265e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc14e760 0x7f04bc0cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04c265e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0cbb70 con 0x7f04bc151040 2024-09-10T10:26:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04c265e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0cb890 con 0x7f04bc14e760 2024-09-10T10:26:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04c265e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc1551c0 con 0x7f04a8005210 2024-09-10T10:26:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04b3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04bc151040 0x7f04bc14aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8005210 0x7f04bc14a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04b3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04bc151040 0x7f04bc14aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52572/0 (socket says 172.21.15.120:52572) 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8005210 0x7f04bc14a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:48142/0 (socket says 172.21.15.120:48142) 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04b3fff640 1 -- 172.21.15.120:0/3367652002 learned_addr learned my addr 172.21.15.120:0/3367652002 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04c0bfb640 1 --2- 172.21.15.120:0/3367652002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc14e760 0x7f04bc0cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04b3fff640 1 -- 172.21.15.120:0/3367652002 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8005210 msgr2=0x7f04bc14a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04b3fff640 1 --2- 172.21.15.120:0/3367652002 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8005210 0x7f04bc14a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04b3fff640 1 -- 172.21.15.120:0/3367652002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc14e760 msgr2=0x7f04bc0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04b3fff640 1 --2- 172.21.15.120:0/3367652002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc14e760 0x7f04bc0cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.033+0000 7f04b3fff640 1 -- 172.21.15.120:0/3367652002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc14b570 con 0x7f04bc151040 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.034+0000 7f04c0bfb640 1 --2- 172.21.15.120:0/3367652002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc14e760 0x7f04bc0cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.034+0000 7f04bbfff640 1 --2- 172.21.15.120:0/3367652002 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8005210 0x7f04bc14a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.034+0000 7f04b3fff640 1 --2- 172.21.15.120:0/3367652002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04bc151040 0x7f04bc14aee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04a4007340 tx=0x7f04a4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.034+0000 7f04b97fa640 1 -- 172.21.15.120:0/3367652002 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a400f940 con 0x7f04bc151040 2024-09-10T10:26:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.034+0000 7f04b97fa640 1 -- 172.21.15.120:0/3367652002 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a4018070 con 0x7f04bc151040 2024-09-10T10:26:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.034+0000 7f04b97fa640 1 -- 172.21.15.120:0/3367652002 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a400fd60 con 0x7f04bc151040 2024-09-10T10:26:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.034+0000 7f04c265e640 1 -- 172.21.15.120:0/3367652002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc15c770 con 0x7f04bc151040 2024-09-10T10:26:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.034+0000 7f04c265e640 1 -- 172.21.15.120:0/3367652002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc0ce250 con 0x7f04bc151040 2024-09-10T10:26:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.034+0000 7f04b97fa640 1 -- 172.21.15.120:0/3367652002 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f04a401d460 con 0x7f04bc151040 2024-09-10T10:26:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.034+0000 7f04b97fa640 1 -- 172.21.15.120:0/3367652002 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f04a4013760 con 0x7f04bc151040 2024-09-10T10:26:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.035+0000 7f04c265e640 1 -- 172.21.15.120:0/3367652002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc0c7760 con 0x7f04bc151040 2024-09-10T10:26:18.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.039+0000 7f04b97fa640 1 -- 172.21.15.120:0/3367652002 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f04bc0d0dd0 con 0x7f04bc151040 2024-09-10T10:26:18.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.102+0000 7f04c265e640 1 -- 172.21.15.120:0/3367652002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f04bc0cc8a0 con 0x7f04bc151040 2024-09-10T10:26:18.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.103+0000 7f04b97fa640 1 -- 172.21.15.120:0/3367652002 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f04a4013960 con 0x7f04bc151040 2024-09-10T10:26:18.105 INFO:teuthology.orchestra.run.smithi120.stderr:adjusted tunables profile to default 2024-09-10T10:26:18.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.104+0000 7f04c265e640 1 -- 172.21.15.120:0/3367652002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04bc151040 msgr2=0x7f04bc14aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.104+0000 7f04c265e640 1 --2- 172.21.15.120:0/3367652002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04bc151040 0x7f04bc14aee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04a4007340 tx=0x7f04a4007800 comp rx=0 tx=0).stop 2024-09-10T10:26:18.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.104+0000 7f04c265e640 1 -- 172.21.15.120:0/3367652002 shutdown_connections 2024-09-10T10:26:18.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.104+0000 7f04c265e640 1 --2- 172.21.15.120:0/3367652002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc14e760 0x7f04bc0cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.104+0000 7f04c265e640 1 --2- 172.21.15.120:0/3367652002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04bc151040 0x7f04bc14aee0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.104+0000 7f04c265e640 1 --2- 172.21.15.120:0/3367652002 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a8005210 0x7f04bc14a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.104+0000 7f04c265e640 1 -- 172.21.15.120:0/3367652002 >> 172.21.15.120:0/3367652002 conn(0x7f04bc06cab0 msgr2=0x7f04bc15e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:18.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.104+0000 7f04c265e640 1 -- 172.21.15.120:0/3367652002 shutdown_connections 2024-09-10T10:26:18.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:18.105+0000 7f04c265e640 1 -- 172.21.15.120:0/3367652002 wait complete. 2024-09-10T10:26:18.116 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T10:26:18.116 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T10:26:18.116 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:18.116 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T10:26:18.135 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:18.134+0000 7fc697d21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:26:18.135 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:18.134+0000 7fce8218a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:26:18.141 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:18.141 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T10:26:18.176 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:18.175+0000 7fce8218a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:26:18.177 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:18.176+0000 7fc697d21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:26:18.201 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:18.202 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T10:26:18.267 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:18.267 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T10:26:18.321 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:18.320+0000 7fce8218a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:26:18.325 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:18.324+0000 7fc697d21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:26:18.331 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:18.331 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T10:26:18.356 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:18.356 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T10:26:18.373 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:18.372+0000 7fce8218a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:26:18.376 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:18.376+0000 7fc697d21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:26:18.397 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:18.397 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T10:26:18.463 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-10T10:26:18.463 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T10:26:18.528 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new e3b9c3c3-cf7d-4b33-913e-60e928439d8e 0 2024-09-10T10:26:18.533 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:18.532+0000 7fce8218a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:26:18.539 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:18.539+0000 7fc697d21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:26:18.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.629+0000 7f9d8951a640 1 Processor -- start 2024-09-10T10:26:18.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.629+0000 7f9d8951a640 1 -- start start 2024-09-10T10:26:18.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d8951a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9d84154be0 0x7f9d840c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d8951a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154fc0 0x7f9d840cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d8951a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d8415e190 con 0x7f9d840d0270 2024-09-10T10:26:18.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d8951a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9d8415e470 con 0x7f9d84154be0 2024-09-10T10:26:18.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d8951a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8415e750 con 0x7f9d84154fc0 2024-09-10T10:26:18.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d827fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154fc0 0x7f9d840cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d82ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9d84154be0 0x7f9d840c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d827fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154fc0 0x7f9d840cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51846/0 (socket says 172.21.15.134:51846) 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d827fc640 1 -- 172.21.15.134:0/1570061900 learned_addr learned my addr 172.21.15.134:0/1570061900 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d837fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f9d840d0270 0x7f9d8415aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56218/0 (socket says 172.21.15.134:56218) 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d827fc640 1 -- 172.21.15.134:0/1570061900 >> v2:172.21.15.134:3300/0 conn(0x7f9d84154be0 msgr2=0x7f9d840c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d827fc640 1 --2- 172.21.15.134:0/1570061900 >> v2:172.21.15.134:3300/0 conn(0x7f9d84154be0 0x7f9d840c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d827fc640 1 -- 172.21.15.134:0/1570061900 >> v1:172.21.15.134:6789/0 conn(0x7f9d840d0270 legacy=0x7f9d8415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d827fc640 1 -- 172.21.15.134:0/1570061900 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8415ea30 con 0x7f9d84154fc0 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d81ffb640 1 -- 172.21.15.134:0/1570061900 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2419417908 0 0) 0x7f9d8415e190 con 0x7f9d840d0270 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.630+0000 7f9d82ffd640 1 --2- 172.21.15.134:0/1570061900 >> v2:172.21.15.134:3300/0 conn(0x7f9d84154be0 0x7f9d840c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.631+0000 7f9d827fc640 1 --2- 172.21.15.134:0/1570061900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154fc0 0x7f9d840cc060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009870 tx=0x7f9d7402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c524567f94c79a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.631+0000 7f9d81ffb640 1 -- 172.21.15.134:0/1570061900 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74037070 con 0x7f9d84154fc0 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.631+0000 7f9d81ffb640 1 -- 172.21.15.134:0/1570061900 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7402ae50 con 0x7f9d84154fc0 2024-09-10T10:26:18.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.631+0000 7f9d81ffb640 1 -- 172.21.15.134:0/1570061900 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74032860 con 0x7f9d84154fc0 2024-09-10T10:26:18.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.632+0000 7f9d8951a640 1 -- 172.21.15.134:0/1570061900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154fc0 msgr2=0x7f9d840cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.632+0000 7f9d8951a640 1 --2- 172.21.15.134:0/1570061900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154fc0 0x7f9d840cc060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009870 tx=0x7f9d7402a300 comp rx=0 tx=0).stop 2024-09-10T10:26:18.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.632+0000 7f9d8951a640 1 -- 172.21.15.134:0/1570061900 shutdown_connections 2024-09-10T10:26:18.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.632+0000 7f9d8951a640 1 --2- 172.21.15.134:0/1570061900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154fc0 0x7f9d840cc060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.632+0000 7f9d8951a640 1 --2- 172.21.15.134:0/1570061900 >> v2:172.21.15.134:3300/0 conn(0x7f9d84154be0 0x7f9d840c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.632+0000 7f9d8951a640 1 -- 172.21.15.134:0/1570061900 >> 172.21.15.134:0/1570061900 conn(0x7f9d8406cb00 msgr2=0x7f9d8406e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:18.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.632+0000 7f9d8951a640 1 -- 172.21.15.134:0/1570061900 shutdown_connections 2024-09-10T10:26:18.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.632+0000 7f9d8951a640 1 -- 172.21.15.134:0/1570061900 wait complete. 2024-09-10T10:26:18.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.634+0000 7f9d8951a640 1 Processor -- start 2024-09-10T10:26:18.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.634+0000 7f9d8951a640 1 -- start start 2024-09-10T10:26:18.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d8951a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d840d0270 0x7f9d840c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d8951a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154be0 0x7f9d840c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d8951a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d84154fc0 0x7f9d84150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d8951a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9d840ccae0 con 0x7f9d840d0270 2024-09-10T10:26:18.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d8951a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d840cc960 con 0x7f9d84154fc0 2024-09-10T10:26:18.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d8951a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8415b3b0 con 0x7f9d84154be0 2024-09-10T10:26:18.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d827fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154be0 0x7f9d840c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d837fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d84154fc0 0x7f9d84150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d827fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154be0 0x7f9d840c1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51852/0 (socket says 172.21.15.134:51852) 2024-09-10T10:26:18.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d827fc640 1 -- 172.21.15.134:0/643667057 learned_addr learned my addr 172.21.15.134:0/643667057 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d82ffd640 1 --2- 172.21.15.134:0/643667057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d840d0270 0x7f9d840c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d827fc640 1 -- 172.21.15.134:0/643667057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d84154fc0 msgr2=0x7f9d84150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d827fc640 1 --2- 172.21.15.134:0/643667057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d84154fc0 0x7f9d84150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d827fc640 1 -- 172.21.15.134:0/643667057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d840d0270 msgr2=0x7f9d840c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d827fc640 1 --2- 172.21.15.134:0/643667057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d840d0270 0x7f9d840c07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.635+0000 7f9d827fc640 1 -- 172.21.15.134:0/643667057 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d840c1310 con 0x7f9d84154be0 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.636+0000 7f9d82ffd640 1 --2- 172.21.15.134:0/643667057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d840d0270 0x7f9d840c07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.636+0000 7f9d827fc640 1 --2- 172.21.15.134:0/643667057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154be0 0x7f9d840c1d10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d740090b0 tx=0x7f9d740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.636+0000 7f9d7b7fe640 1 -- 172.21.15.134:0/643667057 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74041020 con 0x7f9d84154be0 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.636+0000 7f9d7b7fe640 1 -- 172.21.15.134:0/643667057 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7403c890 con 0x7f9d84154be0 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.636+0000 7f9d8951a640 1 -- 172.21.15.134:0/643667057 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d840c15e0 con 0x7f9d84154be0 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.636+0000 7f9d7b7fe640 1 -- 172.21.15.134:0/643667057 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74037040 con 0x7f9d84154be0 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.636+0000 7f9d8951a640 1 -- 172.21.15.134:0/643667057 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d841f3980 con 0x7f9d84154be0 2024-09-10T10:26:18.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.636+0000 7f9d7b7fe640 1 -- 172.21.15.134:0/643667057 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9d74044020 con 0x7f9d84154be0 2024-09-10T10:26:18.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.636+0000 7f9d7b7fe640 1 -- 172.21.15.134:0/643667057 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f9d74006320 con 0x7f9d84154be0 2024-09-10T10:26:18.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.637+0000 7f9d8951a640 1 -- 172.21.15.134:0/643667057 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d840c7990 con 0x7f9d84154be0 2024-09-10T10:26:18.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.639+0000 7f9d7b7fe640 1 -- 172.21.15.134:0/643667057 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9d7404b070 con 0x7f9d84154be0 2024-09-10T10:26:18.648 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:18.647+0000 7fc697d21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:26:18.651 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:18.650+0000 7fce8218a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:26:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.708+0000 7f9d8951a640 1 -- 172.21.15.134:0/643667057 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e3b9c3c3-cf7d-4b33-913e-60e928439d8e", "id": 0} v 0) -- 0x7f9d840cc1d0 con 0x7f9d84154be0 2024-09-10T10:26:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.711+0000 7f9d7b7fe640 1 -- 172.21.15.134:0/643667057 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e3b9c3c3-cf7d-4b33-913e-60e928439d8e", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f9d7403cd50 con 0x7f9d84154be0 2024-09-10T10:26:18.713 INFO:teuthology.orchestra.run.smithi134.stdout:0 2024-09-10T10:26:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.712+0000 7f9d8951a640 1 -- 172.21.15.134:0/643667057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154be0 msgr2=0x7f9d840c1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.712+0000 7f9d8951a640 1 --2- 172.21.15.134:0/643667057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154be0 0x7f9d840c1d10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d740090b0 tx=0x7f9d740046c0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.712+0000 7f9d8951a640 1 -- 172.21.15.134:0/643667057 shutdown_connections 2024-09-10T10:26:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.712+0000 7f9d8951a640 1 --2- 172.21.15.134:0/643667057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d84154fc0 0x7f9d84150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.712+0000 7f9d8951a640 1 --2- 172.21.15.134:0/643667057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d84154be0 0x7f9d840c1d10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.712+0000 7f9d8951a640 1 --2- 172.21.15.134:0/643667057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d840d0270 0x7f9d840c07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.712+0000 7f9d8951a640 1 -- 172.21.15.134:0/643667057 >> 172.21.15.134:0/643667057 conn(0x7f9d8406cb00 msgr2=0x7f9d840cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.712+0000 7f9d8951a640 1 -- 172.21.15.134:0/643667057 shutdown_connections 2024-09-10T10:26:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.712+0000 7f9d8951a640 1 -- 172.21.15.134:0/643667057 wait complete. 2024-09-10T10:26:18.724 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 62c20a9d-e6ab-446d-998c-9ee46e46aa90 1 2024-09-10T10:26:18.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.821+0000 7fbbd1a8b640 1 Processor -- start 2024-09-10T10:26:18.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.821+0000 7fbbd1a8b640 1 -- start start 2024-09-10T10:26:18.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbd1a8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc067d50 0x7fbbcc0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbd1a8b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbcc15b970 con 0x7fbbcc06bc10 2024-09-10T10:26:18.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbd1a8b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbcc0c8570 con 0x7fbbcc0c7a80 2024-09-10T10:26:18.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbd1a8b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc15b430 con 0x7fbbcc067d50 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbcaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc067d50 0x7fbbcc0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbcb7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fbbcc0c7a80 0x7fbbcc0c7e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56224/0 (socket says 172.21.15.134:56224) 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbcb7fe640 1 -- 172.21.15.134:0/1639617440 learned_addr learned my addr 172.21.15.134:0/1639617440 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbca7fc640 1 -- 172.21.15.134:0/1639617440 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132958720 0 0) 0x7fbbcc0c8570 con 0x7fbbcc0c7a80 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbcaffd640 1 -- 172.21.15.134:0/1639617440 >> v1:172.21.15.134:6789/0 conn(0x7fbbcc0c7a80 legacy=0x7fbbcc0c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbcaffd640 1 -- 172.21.15.134:0/1639617440 >> v1:172.21.15.120:6789/0 conn(0x7fbbcc06bc10 legacy=0x7fbbcc1547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.822+0000 7fbbcaffd640 1 -- 172.21.15.134:0/1639617440 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc15b690 con 0x7fbbcc067d50 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.823+0000 7fbbcaffd640 1 --2- 172.21.15.134:0/1639617440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc067d50 0x7fbbcc0caa50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00097d0 tx=0x7fbbc002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751f6e09f066ad46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.823+0000 7fbbca7fc640 1 -- 172.21.15.134:0/1639617440 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0037070 con 0x7fbbcc067d50 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.823+0000 7fbbca7fc640 1 -- 172.21.15.134:0/1639617440 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc0032440 con 0x7fbbcc067d50 2024-09-10T10:26:18.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.823+0000 7fbbca7fc640 1 -- 172.21.15.134:0/1639617440 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc00327f0 con 0x7fbbcc067d50 2024-09-10T10:26:18.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.824+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/1639617440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc067d50 msgr2=0x7fbbcc0caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.824+0000 7fbbd1a8b640 1 --2- 172.21.15.134:0/1639617440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc067d50 0x7fbbcc0caa50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00097d0 tx=0x7fbbc002a290 comp rx=0 tx=0).stop 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.824+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/1639617440 shutdown_connections 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.824+0000 7fbbd1a8b640 1 --2- 172.21.15.134:0/1639617440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc067d50 0x7fbbcc0caa50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.824+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/1639617440 >> 172.21.15.134:0/1639617440 conn(0x7fbbcc0c9220 msgr2=0x7fbbcc0c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.824+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/1639617440 shutdown_connections 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.824+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/1639617440 wait complete. 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.825+0000 7fbbd1a8b640 1 Processor -- start 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.825+0000 7fbbd1a8b640 1 -- start start 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbd1a8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbcc067d50 0x7fbbcc155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbd1a8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc06bc10 0x7fbbcc14d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbd1a8b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0c7a80 0x7fbbcc0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbd1a8b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc0cb8a0 con 0x7fbbcc0c7a80 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbd1a8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc0cb5c0 con 0x7fbbcc067d50 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbd1a8b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc154ed0 con 0x7fbbcc06bc10 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbcc067d50 0x7fbbcc155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc06bc10 0x7fbbcc14d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbcc067d50 0x7fbbcc155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:52654/0 (socket says 172.21.15.134:52654) 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc06bc10 0x7fbbcc14d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51868/0 (socket says 172.21.15.134:51868) 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcaffd640 1 -- 172.21.15.134:0/3781674896 learned_addr learned my addr 172.21.15.134:0/3781674896 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcbfff640 1 --2- 172.21.15.134:0/3781674896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0c7a80 0x7fbbcc0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcb7fe640 1 -- 172.21.15.134:0/3781674896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc06bc10 msgr2=0x7fbbcc14d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcb7fe640 1 --2- 172.21.15.134:0/3781674896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc06bc10 0x7fbbcc14d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcb7fe640 1 -- 172.21.15.134:0/3781674896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0c7a80 msgr2=0x7fbbcc0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcb7fe640 1 --2- 172.21.15.134:0/3781674896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0c7a80 0x7fbbcc0c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcb7fe640 1 -- 172.21.15.134:0/3781674896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc1f0870 con 0x7fbbcc067d50 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcaffd640 1 --2- 172.21.15.134:0/3781674896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc06bc10 0x7fbbcc14d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.826+0000 7fbbcb7fe640 1 --2- 172.21.15.134:0/3781674896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbcc067d50 0x7fbbcc155fe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00c410 tx=0x7fbbbc00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.827+0000 7fbbb3fff640 1 -- 172.21.15.134:0/3781674896 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc0074e0 con 0x7fbbcc067d50 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.827+0000 7fbbb3fff640 1 -- 172.21.15.134:0/3781674896 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbbc0047e0 con 0x7fbbcc067d50 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.827+0000 7fbbb3fff640 1 -- 172.21.15.134:0/3781674896 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc004bb0 con 0x7fbbcc067d50 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.827+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/3781674896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbcc1f0b30 con 0x7fbbcc067d50 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.827+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/3781674896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbcc1f10f0 con 0x7fbbcc067d50 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.827+0000 7fbbb3fff640 1 -- 172.21.15.134:0/3781674896 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbbbc011070 con 0x7fbbcc067d50 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.827+0000 7fbbb3fff640 1 -- 172.21.15.134:0/3781674896 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fbbbc013770 con 0x7fbbcc067d50 2024-09-10T10:26:18.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.828+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/3781674896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbcc151a90 con 0x7fbbcc067d50 2024-09-10T10:26:18.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.830+0000 7fbbb3fff640 1 -- 172.21.15.134:0/3781674896 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbbbc0139c0 con 0x7fbbcc067d50 2024-09-10T10:26:18.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.923+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/3781674896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "62c20a9d-e6ab-446d-998c-9ee46e46aa90", "id": 1} v 0) -- 0x7fbbcc1e69a0 con 0x7fbbcc067d50 2024-09-10T10:26:18.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.927+0000 7fbbb3fff640 1 -- 172.21.15.134:0/3781674896 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62c20a9d-e6ab-446d-998c-9ee46e46aa90", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fbbbc013ba0 con 0x7fbbcc067d50 2024-09-10T10:26:18.928 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-09-10T10:26:18.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.928+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/3781674896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbcc067d50 msgr2=0x7fbbcc155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:18.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.928+0000 7fbbd1a8b640 1 --2- 172.21.15.134:0/3781674896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbcc067d50 0x7fbbcc155fe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00c410 tx=0x7fbbbc00c8d0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.928+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/3781674896 shutdown_connections 2024-09-10T10:26:18.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.928+0000 7fbbd1a8b640 1 --2- 172.21.15.134:0/3781674896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0c7a80 0x7fbbcc0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.928+0000 7fbbd1a8b640 1 --2- 172.21.15.134:0/3781674896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbcc06bc10 0x7fbbcc14d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.928+0000 7fbbd1a8b640 1 --2- 172.21.15.134:0/3781674896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbcc067d50 0x7fbbcc155fe0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:18.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.928+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/3781674896 >> 172.21.15.134:0/3781674896 conn(0x7fbbcc0c9220 msgr2=0x7fbbcc154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:18.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.928+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/3781674896 shutdown_connections 2024-09-10T10:26:18.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:18.928+0000 7fbbd1a8b640 1 -- 172.21.15.134:0/3781674896 wait complete. 2024-09-10T10:26:18.939 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 1528f8e0-5e13-4ec7-9d98-0d94b7a505c3 2 2024-09-10T10:26:18.977 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:18.977+0000 7fc697d21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:26:18.988 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:18.987+0000 7fce8218a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:26:19.028 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:19.027+0000 7fc697d21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:26:19.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.038+0000 7f6fdeae4640 1 Processor -- start 2024-09-10T10:26:19.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.038+0000 7f6fdeae4640 1 -- start start 2024-09-10T10:26:19.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fdeae4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd8152980 0x7f6fd80cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fdeae4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd80c8c20 con 0x7f6fd80c64c0 2024-09-10T10:26:19.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fdeae4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd81591c0 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fdeae4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd80c7480 con 0x7f6fd8152980 2024-09-10T10:26:19.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fdc880640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd8152980 0x7f6fd80cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fd7fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f6fd80ca5a0 0x7f6fd80c5db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56228/0 (socket says 172.21.15.134:56228) 2024-09-10T10:26:19.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fd7fff640 1 -- 172.21.15.134:0/834249830 learned_addr learned my addr 172.21.15.134:0/834249830 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-10T10:26:19.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fdc880640 1 -- 172.21.15.134:0/834249830 >> v1:172.21.15.134:6789/0 conn(0x7f6fd80ca5a0 legacy=0x7f6fd80c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fdc880640 1 -- 172.21.15.134:0/834249830 >> v1:172.21.15.120:6789/0 conn(0x7f6fd80c64c0 legacy=0x7f6fd8163020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fd77fe640 1 -- 172.21.15.134:0/834249830 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383324591 0 0) 0x7f6fd81591c0 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.039+0000 7f6fdc880640 1 -- 172.21.15.134:0/834249830 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd81594a0 con 0x7f6fd8152980 2024-09-10T10:26:19.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.040+0000 7f6fdc880640 1 --2- 172.21.15.134:0/834249830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd8152980 0x7f6fd80cb660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0097d0 tx=0x7f6fcc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09a5fd58bacb68e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:19.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.040+0000 7f6fd77fe640 1 -- 172.21.15.134:0/834249830 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc037070 con 0x7f6fd8152980 2024-09-10T10:26:19.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.040+0000 7f6fd77fe640 1 -- 172.21.15.134:0/834249830 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fcc02ad90 con 0x7f6fd8152980 2024-09-10T10:26:19.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.040+0000 7f6fd77fe640 1 -- 172.21.15.134:0/834249830 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc032660 con 0x7f6fd8152980 2024-09-10T10:26:19.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.040+0000 7f6fdeae4640 1 -- 172.21.15.134:0/834249830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd8152980 msgr2=0x7f6fd80cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.040+0000 7f6fdeae4640 1 --2- 172.21.15.134:0/834249830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd8152980 0x7f6fd80cb660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0097d0 tx=0x7f6fcc02a100 comp rx=0 tx=0).stop 2024-09-10T10:26:19.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.040+0000 7f6fdeae4640 1 -- 172.21.15.134:0/834249830 shutdown_connections 2024-09-10T10:26:19.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.040+0000 7f6fdeae4640 1 --2- 172.21.15.134:0/834249830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd8152980 0x7f6fd80cb660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.040+0000 7f6fdeae4640 1 -- 172.21.15.134:0/834249830 >> 172.21.15.134:0/834249830 conn(0x7f6fd80c9220 msgr2=0x7f6fd80c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:19.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.040+0000 7f6fdeae4640 1 -- 172.21.15.134:0/834249830 shutdown_connections 2024-09-10T10:26:19.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.041+0000 7f6fdeae4640 1 -- 172.21.15.134:0/834249830 wait complete. 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.042+0000 7f6fdeae4640 1 Processor -- start 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.042+0000 7f6fdeae4640 1 -- start start 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.042+0000 7f6fdeae4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fd80c64c0 0x7f6fd8148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.042+0000 7f6fdeae4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd80ca5a0 0x7f6fd8159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fdeae4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd8152980 0x7f6fd814eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fdeae4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd8152d60 con 0x7f6fd80c64c0 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fdeae4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd80c8f00 con 0x7f6fd8152980 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fdeae4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd80c8680 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd80ca5a0 0x7f6fd8159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fdd081640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd8152980 0x7f6fd814eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd80ca5a0 0x7f6fd8159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51880/0 (socket says 172.21.15.134:51880) 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd7fff640 1 -- 172.21.15.134:0/2168587371 learned_addr learned my addr 172.21.15.134:0/2168587371 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fdc880640 1 --2- 172.21.15.134:0/2168587371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fd80c64c0 0x7f6fd8148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd7fff640 1 -- 172.21.15.134:0/2168587371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd8152980 msgr2=0x7f6fd814eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd7fff640 1 --2- 172.21.15.134:0/2168587371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd8152980 0x7f6fd814eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd7fff640 1 -- 172.21.15.134:0/2168587371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fd80c64c0 msgr2=0x7f6fd8148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd7fff640 1 --2- 172.21.15.134:0/2168587371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fd80c64c0 0x7f6fd8148f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd7fff640 1 -- 172.21.15.134:0/2168587371 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd8149540 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd7fff640 1 --2- 172.21.15.134:0/2168587371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd80ca5a0 0x7f6fd8159940 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400c430 tx=0x7f6fc400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd57fa640 1 -- 172.21.15.134:0/2168587371 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40074e0 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd57fa640 1 -- 172.21.15.134:0/2168587371 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc40047e0 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fdeae4640 1 -- 172.21.15.134:0/2168587371 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd8147080 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fd57fa640 1 -- 172.21.15.134:0/2168587371 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4004c10 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.043+0000 7f6fdeae4640 1 -- 172.21.15.134:0/2168587371 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd8147640 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.044+0000 7f6fd57fa640 1 -- 172.21.15.134:0/2168587371 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6fc4011070 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.044+0000 7f6fd57fa640 1 -- 172.21.15.134:0/2168587371 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f6fc40137e0 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.049 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:19.044+0000 7fce8218a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:26:19.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.044+0000 7f6fdeae4640 1 -- 172.21.15.134:0/2168587371 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd8067950 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.046+0000 7f6fd57fa640 1 -- 172.21.15.134:0/2168587371 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6fc4004db0 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.116+0000 7f6fdeae4640 1 -- 172.21.15.134:0/2168587371 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1528f8e0-5e13-4ec7-9d98-0d94b7a505c3", "id": 2} v 0) -- 0x7f6fd814ff40 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.122+0000 7f6fd57fa640 1 -- 172.21.15.134:0/2168587371 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1528f8e0-5e13-4ec7-9d98-0d94b7a505c3", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f6fc40133c0 con 0x7f6fd80ca5a0 2024-09-10T10:26:19.123 INFO:teuthology.orchestra.run.smithi134.stdout:2 2024-09-10T10:26:19.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.123+0000 7f6fdeae4640 1 -- 172.21.15.134:0/2168587371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd80ca5a0 msgr2=0x7f6fd8159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.123+0000 7f6fdeae4640 1 --2- 172.21.15.134:0/2168587371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd80ca5a0 0x7f6fd8159940 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400c430 tx=0x7f6fc400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.123+0000 7f6fdeae4640 1 -- 172.21.15.134:0/2168587371 shutdown_connections 2024-09-10T10:26:19.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.123+0000 7f6fdeae4640 1 --2- 172.21.15.134:0/2168587371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd8152980 0x7f6fd814eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.123+0000 7f6fdeae4640 1 --2- 172.21.15.134:0/2168587371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fd80ca5a0 0x7f6fd8159940 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.123+0000 7f6fdeae4640 1 --2- 172.21.15.134:0/2168587371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fd80c64c0 0x7f6fd8148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.123+0000 7f6fdeae4640 1 -- 172.21.15.134:0/2168587371 >> 172.21.15.134:0/2168587371 conn(0x7f6fd80c9220 msgr2=0x7f6fd806fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:19.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.123+0000 7f6fdeae4640 1 -- 172.21.15.134:0/2168587371 shutdown_connections 2024-09-10T10:26:19.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.123+0000 7f6fdeae4640 1 -- 172.21.15.134:0/2168587371 wait complete. 2024-09-10T10:26:19.134 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 74942f78-c1fc-4886-abf0-950722c5edc6 3 2024-09-10T10:26:19.178 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:19.177+0000 7fc697d21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:26:19.202 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:19.201+0000 7fce8218a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:26:19.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.231+0000 7ff909786640 1 Processor -- start 2024-09-10T10:26:19.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.231+0000 7ff909786640 1 -- start start 2024-09-10T10:26:19.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.232+0000 7ff909786640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff9040c9890 0x7ff904070450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.232+0000 7ff909786640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9040c5ef0 con 0x7ff9040c9c70 2024-09-10T10:26:19.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.232+0000 7ff909786640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff90406d8c0 con 0x7ff9040c6d90 2024-09-10T10:26:19.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.232+0000 7ff909786640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff90406db80 con 0x7ff9040c9890 2024-09-10T10:26:19.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.232+0000 7ff902ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff9040c9890 0x7ff904070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.232+0000 7ff9037fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff9040c6d90 0x7ff9041655e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56238/0 (socket says 172.21.15.134:56238) 2024-09-10T10:26:19.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.232+0000 7ff9037fe640 1 -- 172.21.15.134:0/2749329624 learned_addr learned my addr 172.21.15.134:0/2749329624 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-10T10:26:19.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.233+0000 7ff902ffd640 1 -- 172.21.15.134:0/2749329624 >> v1:172.21.15.134:6789/0 conn(0x7ff9040c6d90 legacy=0x7ff9041655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.233+0000 7ff902ffd640 1 -- 172.21.15.134:0/2749329624 >> v1:172.21.15.120:6789/0 conn(0x7ff9040c9c70 legacy=0x7ff9040c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.233+0000 7ff902ffd640 1 -- 172.21.15.134:0/2749329624 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90406de60 con 0x7ff9040c9890 2024-09-10T10:26:19.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.233+0000 7ff902ffd640 1 --2- 172.21.15.134:0/2749329624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9040c9890 0x7ff904070450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40097a0 tx=0x7ff8f402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66967a2428016be1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:19.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.233+0000 7ff901ffb640 1 -- 172.21.15.134:0/2749329624 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4037070 con 0x7ff9040c9890 2024-09-10T10:26:19.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.233+0000 7ff901ffb640 1 -- 172.21.15.134:0/2749329624 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f4032640 con 0x7ff9040c9890 2024-09-10T10:26:19.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.233+0000 7ff901ffb640 1 -- 172.21.15.134:0/2749329624 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4032a10 con 0x7ff9040c9890 2024-09-10T10:26:19.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.234+0000 7ff909786640 1 -- 172.21.15.134:0/2749329624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9040c9890 msgr2=0x7ff904070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.234+0000 7ff909786640 1 --2- 172.21.15.134:0/2749329624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9040c9890 0x7ff904070450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40097a0 tx=0x7ff8f402a2c0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.234+0000 7ff909786640 1 -- 172.21.15.134:0/2749329624 shutdown_connections 2024-09-10T10:26:19.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.234+0000 7ff909786640 1 --2- 172.21.15.134:0/2749329624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9040c9890 0x7ff904070450 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.234+0000 7ff909786640 1 -- 172.21.15.134:0/2749329624 >> 172.21.15.134:0/2749329624 conn(0x7ff9040cac70 msgr2=0x7ff9040cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:19.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.234+0000 7ff909786640 1 -- 172.21.15.134:0/2749329624 shutdown_connections 2024-09-10T10:26:19.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.234+0000 7ff909786640 1 -- 172.21.15.134:0/2749329624 wait complete. 2024-09-10T10:26:19.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.235+0000 7ff909786640 1 Processor -- start 2024-09-10T10:26:19.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff909786640 1 -- start start 2024-09-10T10:26:19.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff909786640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9040c6d90 0x7ff9041f1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff909786640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9040c9890 0x7ff9041f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff909786640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9040c9c70 0x7ff904070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff909786640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040cbab0 con 0x7ff9040c6d90 2024-09-10T10:26:19.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff909786640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040c6160 con 0x7ff9040c9890 2024-09-10T10:26:19.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff909786640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9040c9090 con 0x7ff9040c9c70 2024-09-10T10:26:19.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff9037fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9040c9c70 0x7ff904070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff9027fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9040c9890 0x7ff9041f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff9037fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9040c9c70 0x7ff904070490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51896/0 (socket says 172.21.15.134:51896) 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff9027fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9040c9890 0x7ff9041f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:52684/0 (socket says 172.21.15.134:52684) 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff9037fe640 1 -- 172.21.15.134:0/3831959184 learned_addr learned my addr 172.21.15.134:0/3831959184 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.236+0000 7ff902ffd640 1 --2- 172.21.15.134:0/3831959184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9040c6d90 0x7ff9041f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff9037fe640 1 -- 172.21.15.134:0/3831959184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9040c9890 msgr2=0x7ff9041f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff9037fe640 1 --2- 172.21.15.134:0/3831959184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9040c9890 0x7ff9041f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff9037fe640 1 -- 172.21.15.134:0/3831959184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9040c6d90 msgr2=0x7ff9041f1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff9037fe640 1 --2- 172.21.15.134:0/3831959184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9040c6d90 0x7ff9041f1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff9037fe640 1 -- 172.21.15.134:0/3831959184 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9040c3640 con 0x7ff9040c9c70 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff9037fe640 1 --2- 172.21.15.134:0/3831959184 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9040c9c70 0x7ff904070490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00cbf0 tx=0x7ff8fc00a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff8fb7fe640 1 -- 172.21.15.134:0/3831959184 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc017070 con 0x7ff9040c9c70 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff8fb7fe640 1 -- 172.21.15.134:0/3831959184 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8fc00ad40 con 0x7ff9040c9c70 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff8fb7fe640 1 -- 172.21.15.134:0/3831959184 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc012600 con 0x7ff9040c9c70 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff909786640 1 -- 172.21.15.134:0/3831959184 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9040c46b0 con 0x7ff9040c9c70 2024-09-10T10:26:19.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff909786640 1 -- 172.21.15.134:0/3831959184 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9040c4c70 con 0x7ff9040c9c70 2024-09-10T10:26:19.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff8fb7fe640 1 -- 172.21.15.134:0/3831959184 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff8fc01a070 con 0x7ff9040c9c70 2024-09-10T10:26:19.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.237+0000 7ff8fb7fe640 1 -- 172.21.15.134:0/3831959184 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7ff8fc01cce0 con 0x7ff9040c9c70 2024-09-10T10:26:19.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.238+0000 7ff909786640 1 -- 172.21.15.134:0/3831959184 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9040c8a70 con 0x7ff9040c9c70 2024-09-10T10:26:19.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.242+0000 7ff8fb7fe640 1 -- 172.21.15.134:0/3831959184 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff8fc01c440 con 0x7ff9040c9c70 2024-09-10T10:26:19.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.312+0000 7ff909786640 1 -- 172.21.15.134:0/3831959184 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "74942f78-c1fc-4886-abf0-950722c5edc6", "id": 3} v 0) -- 0x7ff9040c1a10 con 0x7ff9040c9c70 2024-09-10T10:26:19.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.317+0000 7ff8fb7fe640 1 -- 172.21.15.134:0/3831959184 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "74942f78-c1fc-4886-abf0-950722c5edc6", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7ff8fc01ca00 con 0x7ff9040c9c70 2024-09-10T10:26:19.319 INFO:teuthology.orchestra.run.smithi134.stdout:3 2024-09-10T10:26:19.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.318+0000 7ff909786640 1 -- 172.21.15.134:0/3831959184 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9040c9c70 msgr2=0x7ff904070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.318+0000 7ff909786640 1 --2- 172.21.15.134:0/3831959184 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9040c9c70 0x7ff904070490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00cbf0 tx=0x7ff8fc00a430 comp rx=0 tx=0).stop 2024-09-10T10:26:19.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.318+0000 7ff909786640 1 -- 172.21.15.134:0/3831959184 shutdown_connections 2024-09-10T10:26:19.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.318+0000 7ff909786640 1 --2- 172.21.15.134:0/3831959184 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9040c9c70 0x7ff904070490 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.318+0000 7ff909786640 1 --2- 172.21.15.134:0/3831959184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9040c9890 0x7ff9041f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.318+0000 7ff909786640 1 --2- 172.21.15.134:0/3831959184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9040c6d90 0x7ff9041f1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.318+0000 7ff909786640 1 -- 172.21.15.134:0/3831959184 >> 172.21.15.134:0/3831959184 conn(0x7ff9040cac70 msgr2=0x7ff904157590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:19.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.319+0000 7ff909786640 1 -- 172.21.15.134:0/3831959184 shutdown_connections 2024-09-10T10:26:19.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.319+0000 7ff909786640 1 -- 172.21.15.134:0/3831959184 wait complete. 2024-09-10T10:26:19.330 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new e71319d4-5318-4441-84a8-7665ae037596 4 2024-09-10T10:26:19.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.427+0000 7f09faf03640 1 Processor -- start 2024-09-10T10:26:19.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.428+0000 7f09faf03640 1 -- start start 2024-09-10T10:26:19.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.428+0000 7f09faf03640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f415b440 0x7f09f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.428+0000 7f09faf03640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09f406c5a0 con 0x7f09f40c6d00 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.428+0000 7f09faf03640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09f40c7a80 con 0x7f09f4158140 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.428+0000 7f09faf03640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09f40c8040 con 0x7f09f415b440 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.428+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f415b440 0x7f09f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.428+0000 7f09f8c9f640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f09f4158140 0x7f09f406cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56254/0 (socket says 172.21.15.134:56254) 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.428+0000 7f09f8c9f640 1 -- 172.21.15.134:0/1373393104 learned_addr learned my addr 172.21.15.134:0/1373393104 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.429+0000 7f09f37fe640 1 -- 172.21.15.134:0/1373393104 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4023453606 0 0) 0x7f09f40c7a80 con 0x7f09f4158140 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.429+0000 7f09f37fe640 1 -- 172.21.15.134:0/1373393104 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09e4003520 con 0x7f09f4158140 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.429+0000 7f09f3fff640 1 -- 172.21.15.134:0/1373393104 >> v1:172.21.15.134:6789/0 conn(0x7f09f4158140 legacy=0x7f09f406cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.429+0000 7f09f3fff640 1 -- 172.21.15.134:0/1373393104 >> v1:172.21.15.120:6789/0 conn(0x7f09f40c6d00 legacy=0x7f09f41652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.429+0000 7f09f3fff640 1 -- 172.21.15.134:0/1373393104 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f40c7d60 con 0x7f09f415b440 2024-09-10T10:26:19.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.429+0000 7f09f3fff640 1 --2- 172.21.15.134:0/1373393104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f415b440 0x7f09f406e770 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f09e00097d0 tx=0x7f09e002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ef81cc7ca877ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:19.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.430+0000 7f09f37fe640 1 -- 172.21.15.134:0/1373393104 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0037070 con 0x7f09f415b440 2024-09-10T10:26:19.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.430+0000 7f09f37fe640 1 -- 172.21.15.134:0/1373393104 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e0032440 con 0x7f09f415b440 2024-09-10T10:26:19.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.430+0000 7f09f37fe640 1 -- 172.21.15.134:0/1373393104 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e00327f0 con 0x7f09f415b440 2024-09-10T10:26:19.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.430+0000 7f09faf03640 1 -- 172.21.15.134:0/1373393104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f415b440 msgr2=0x7f09f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.430+0000 7f09faf03640 1 --2- 172.21.15.134:0/1373393104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f415b440 0x7f09f406e770 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f09e00097d0 tx=0x7f09e002a290 comp rx=0 tx=0).stop 2024-09-10T10:26:19.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.430+0000 7f09faf03640 1 -- 172.21.15.134:0/1373393104 shutdown_connections 2024-09-10T10:26:19.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.430+0000 7f09faf03640 1 --2- 172.21.15.134:0/1373393104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f415b440 0x7f09f406e770 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.430+0000 7f09faf03640 1 -- 172.21.15.134:0/1373393104 >> 172.21.15.134:0/1373393104 conn(0x7f09f40728b0 msgr2=0x7f09f4072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:19.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.430+0000 7f09faf03640 1 -- 172.21.15.134:0/1373393104 shutdown_connections 2024-09-10T10:26:19.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.430+0000 7f09faf03640 1 -- 172.21.15.134:0/1373393104 wait complete. 2024-09-10T10:26:19.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.431+0000 7f09faf03640 1 Processor -- start 2024-09-10T10:26:19.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.431+0000 7f09faf03640 1 -- start start 2024-09-10T10:26:19.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09faf03640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09f40c6d00 0x7f09f41569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09faf03640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f4158140 0x7f09f4158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09faf03640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f415b440 0x7f09f4153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09faf03640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09f40735e0 con 0x7f09f40c6d00 2024-09-10T10:26:19.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09faf03640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09f40c0b20 con 0x7f09f415b440 2024-09-10T10:26:19.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09faf03640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09f40c0ca0 con 0x7f09f4158140 2024-09-10T10:26:19.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f4158140 0x7f09f4158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09f94a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f415b440 0x7f09f4153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f4158140 0x7f09f4158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51924/0 (socket says 172.21.15.134:51924) 2024-09-10T10:26:19.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09f94a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f415b440 0x7f09f4153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:52688/0 (socket says 172.21.15.134:52688) 2024-09-10T10:26:19.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09f3fff640 1 -- 172.21.15.134:0/2871499468 learned_addr learned my addr 172.21.15.134:0/2871499468 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:19.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.432+0000 7f09f8c9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09f40c6d00 0x7f09f41569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f94a0640 1 -- 172.21.15.134:0/2871499468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f4158140 msgr2=0x7f09f4158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f94a0640 1 --2- 172.21.15.134:0/2871499468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f4158140 0x7f09f4158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f94a0640 1 -- 172.21.15.134:0/2871499468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09f40c6d00 msgr2=0x7f09f41569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f94a0640 1 --2- 172.21.15.134:0/2871499468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09f40c6d00 0x7f09f41569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f94a0640 1 -- 172.21.15.134:0/2871499468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f4150d60 con 0x7f09f415b440 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f3fff640 1 --2- 172.21.15.134:0/2871499468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f4158140 0x7f09f4158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f94a0640 1 --2- 172.21.15.134:0/2871499468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f415b440 0x7f09f4153680 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00c690 tx=0x7f09ec00cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f17fa640 1 -- 172.21.15.134:0/2871499468 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec003b70 con 0x7f09f415b440 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f17fa640 1 -- 172.21.15.134:0/2871499468 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09ec003d10 con 0x7f09f415b440 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09faf03640 1 -- 172.21.15.134:0/2871499468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f41502a0 con 0x7f09f415b440 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f17fa640 1 -- 172.21.15.134:0/2871499468 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec005310 con 0x7f09f415b440 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09faf03640 1 -- 172.21.15.134:0/2871499468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f415a2b0 con 0x7f09f415b440 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f17fa640 1 -- 172.21.15.134:0/2871499468 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f09ec0054b0 con 0x7f09f415b440 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.433+0000 7f09f17fa640 1 -- 172.21.15.134:0/2871499468 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f09ec012980 con 0x7f09f415b440 2024-09-10T10:26:19.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.434+0000 7f09faf03640 1 -- 172.21.15.134:0/2871499468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f41f1340 con 0x7f09f415b440 2024-09-10T10:26:19.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.436+0000 7f09f17fa640 1 -- 172.21.15.134:0/2871499468 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f09ec012c60 con 0x7f09f415b440 2024-09-10T10:26:19.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.505+0000 7f09faf03640 1 -- 172.21.15.134:0/2871499468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e71319d4-5318-4441-84a8-7665ae037596", "id": 4} v 0) -- 0x7f09f40c83f0 con 0x7f09f415b440 2024-09-10T10:26:19.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.511+0000 7f09f17fa640 1 -- 172.21.15.134:0/2871499468 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e71319d4-5318-4441-84a8-7665ae037596", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f09ec012470 con 0x7f09f415b440 2024-09-10T10:26:19.512 INFO:teuthology.orchestra.run.smithi134.stdout:4 2024-09-10T10:26:19.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.512+0000 7f09faf03640 1 -- 172.21.15.134:0/2871499468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f415b440 msgr2=0x7f09f4153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.512+0000 7f09faf03640 1 --2- 172.21.15.134:0/2871499468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f415b440 0x7f09f4153680 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00c690 tx=0x7f09ec00cb50 comp rx=0 tx=0).stop 2024-09-10T10:26:19.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.512+0000 7f09faf03640 1 -- 172.21.15.134:0/2871499468 shutdown_connections 2024-09-10T10:26:19.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.512+0000 7f09faf03640 1 --2- 172.21.15.134:0/2871499468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f415b440 0x7f09f4153680 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.512+0000 7f09faf03640 1 --2- 172.21.15.134:0/2871499468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f4158140 0x7f09f4158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.512+0000 7f09faf03640 1 --2- 172.21.15.134:0/2871499468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09f40c6d00 0x7f09f41569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.512+0000 7f09faf03640 1 -- 172.21.15.134:0/2871499468 >> 172.21.15.134:0/2871499468 conn(0x7f09f40728b0 msgr2=0x7f09f4150f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:19.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.512+0000 7f09faf03640 1 -- 172.21.15.134:0/2871499468 shutdown_connections 2024-09-10T10:26:19.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.512+0000 7f09faf03640 1 -- 172.21.15.134:0/2871499468 wait complete. 2024-09-10T10:26:19.526 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 29e193cd-37af-43e5-bd0c-2f85ce0631e8 5 2024-09-10T10:26:19.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.623+0000 7fce43cd0640 1 Processor -- start 2024-09-10T10:26:19.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.624+0000 7fce43cd0640 1 -- start start 2024-09-10T10:26:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.624+0000 7fce43cd0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fce3c158140 0x7fce3c06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.624+0000 7fce43cd0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c15b440 0x7fce3c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.624+0000 7fce43cd0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce3c06c5a0 con 0x7fce3c0c6d00 2024-09-10T10:26:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.624+0000 7fce43cd0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fce3c0c8040 con 0x7fce3c158140 2024-09-10T10:26:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.624+0000 7fce43cd0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fce3c0c4b60 con 0x7fce3c15b440 2024-09-10T10:26:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.624+0000 7fce4126b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c15b440 0x7fce3c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.625+0000 7fce4126b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c15b440 0x7fce3c06e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51934/0 (socket says 172.21.15.134:51934) 2024-09-10T10:26:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.625+0000 7fce4126b640 1 -- 172.21.15.134:0/4087607519 learned_addr learned my addr 172.21.15.134:0/4087607519 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.625+0000 7fce41a6c640 1 --2- 172.21.15.134:0/4087607519 >> v2:172.21.15.120:3300/0 conn(0x7fce3c158140 0x7fce3c06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.625+0000 7fce4126b640 1 -- 172.21.15.134:0/4087607519 >> v2:172.21.15.120:3300/0 conn(0x7fce3c158140 msgr2=0x7fce3c06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.625+0000 7fce4126b640 1 --2- 172.21.15.134:0/4087607519 >> v2:172.21.15.120:3300/0 conn(0x7fce3c158140 0x7fce3c06cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.625+0000 7fce4126b640 1 -- 172.21.15.134:0/4087607519 >> v1:172.21.15.120:6789/0 conn(0x7fce3c0c6d00 legacy=0x7fce3c15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.625+0000 7fce4126b640 1 -- 172.21.15.134:0/4087607519 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c0c7a80 con 0x7fce3c15b440 2024-09-10T10:26:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce4126b640 1 --2- 172.21.15.134:0/4087607519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c15b440 0x7fce3c06e770 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce38009870 tx=0x7fce3802a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e4a5ab9cdedd5ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce40a6a640 1 -- 172.21.15.134:0/4087607519 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38037070 con 0x7fce3c15b440 2024-09-10T10:26:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce40a6a640 1 -- 172.21.15.134:0/4087607519 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3802ae50 con 0x7fce3c15b440 2024-09-10T10:26:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce40a6a640 1 -- 172.21.15.134:0/4087607519 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38032860 con 0x7fce3c15b440 2024-09-10T10:26:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce43cd0640 1 -- 172.21.15.134:0/4087607519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c15b440 msgr2=0x7fce3c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce43cd0640 1 --2- 172.21.15.134:0/4087607519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c15b440 0x7fce3c06e770 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce38009870 tx=0x7fce3802a300 comp rx=0 tx=0).stop 2024-09-10T10:26:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce43cd0640 1 -- 172.21.15.134:0/4087607519 shutdown_connections 2024-09-10T10:26:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce43cd0640 1 --2- 172.21.15.134:0/4087607519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c15b440 0x7fce3c06e770 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce43cd0640 1 --2- 172.21.15.134:0/4087607519 >> v2:172.21.15.120:3300/0 conn(0x7fce3c158140 0x7fce3c06cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce43cd0640 1 -- 172.21.15.134:0/4087607519 >> 172.21.15.134:0/4087607519 conn(0x7fce3c0728b0 msgr2=0x7fce3c072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce43cd0640 1 -- 172.21.15.134:0/4087607519 shutdown_connections 2024-09-10T10:26:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.626+0000 7fce43cd0640 1 -- 172.21.15.134:0/4087607519 wait complete. 2024-09-10T10:26:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce43cd0640 1 Processor -- start 2024-09-10T10:26:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce43cd0640 1 -- start start 2024-09-10T10:26:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce43cd0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c0c6d00 0x7fce3c070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce43cd0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3c158140 0x7fce3c070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce43cd0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce3c15b440 0x7fce3c1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce43cd0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c0735e0 con 0x7fce3c158140 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce43cd0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c06c840 con 0x7fce3c15b440 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce43cd0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fce3c06d2a0 con 0x7fce3c0c6d00 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce41a6c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c0c6d00 0x7fce3c070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce41a6c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c0c6d00 0x7fce3c070100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51950/0 (socket says 172.21.15.134:51950) 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce41a6c640 1 -- 172.21.15.134:0/3569979828 learned_addr learned my addr 172.21.15.134:0/3569979828 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce4226d640 1 --2- 172.21.15.134:0/3569979828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce3c15b440 0x7fce3c1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.628+0000 7fce4126b640 1 --2- 172.21.15.134:0/3569979828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3c158140 0x7fce3c070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce41a6c640 1 -- 172.21.15.134:0/3569979828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce3c15b440 msgr2=0x7fce3c1534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce41a6c640 1 --2- 172.21.15.134:0/3569979828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce3c15b440 0x7fce3c1534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce41a6c640 1 -- 172.21.15.134:0/3569979828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3c158140 msgr2=0x7fce3c070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce41a6c640 1 --2- 172.21.15.134:0/3569979828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3c158140 0x7fce3c070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce41a6c640 1 -- 172.21.15.134:0/3569979828 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c1f4a40 con 0x7fce3c0c6d00 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce4126b640 1 --2- 172.21.15.134:0/3569979828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3c158140 0x7fce3c070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce41a6c640 1 --2- 172.21.15.134:0/3569979828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c0c6d00 0x7fce3c070100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fce3000e9e0 tx=0x7fce3000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce2e7fc640 1 -- 172.21.15.134:0/3569979828 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30017070 con 0x7fce3c0c6d00 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce2e7fc640 1 -- 172.21.15.134:0/3569979828 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3000c9d0 con 0x7fce3c0c6d00 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce43cd0640 1 -- 172.21.15.134:0/3569979828 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3c159570 con 0x7fce3c0c6d00 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce2e7fc640 1 -- 172.21.15.134:0/3569979828 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3000cdc0 con 0x7fce3c0c6d00 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce43cd0640 1 -- 172.21.15.134:0/3569979828 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3c159b30 con 0x7fce3c0c6d00 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce2e7fc640 1 -- 172.21.15.134:0/3569979828 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fce3001a070 con 0x7fce3c0c6d00 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.629+0000 7fce2e7fc640 1 -- 172.21.15.134:0/3569979828 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fce3001cb10 con 0x7fce3c0c6d00 2024-09-10T10:26:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.630+0000 7fce43cd0640 1 -- 172.21.15.134:0/3569979828 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3c0c0b20 con 0x7fce3c0c6d00 2024-09-10T10:26:19.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.634+0000 7fce2e7fc640 1 -- 172.21.15.134:0/3569979828 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fce3001c650 con 0x7fce3c0c6d00 2024-09-10T10:26:19.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.703+0000 7fce43cd0640 1 -- 172.21.15.134:0/3569979828 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "29e193cd-37af-43e5-bd0c-2f85ce0631e8", "id": 5} v 0) -- 0x7fce3c1f0680 con 0x7fce3c0c6d00 2024-09-10T10:26:19.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.707+0000 7fce2e7fc640 1 -- 172.21.15.134:0/3569979828 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "29e193cd-37af-43e5-bd0c-2f85ce0631e8", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fce30005990 con 0x7fce3c0c6d00 2024-09-10T10:26:19.708 INFO:teuthology.orchestra.run.smithi134.stdout:5 2024-09-10T10:26:19.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.708+0000 7fce43cd0640 1 -- 172.21.15.134:0/3569979828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c0c6d00 msgr2=0x7fce3c070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.708+0000 7fce43cd0640 1 --2- 172.21.15.134:0/3569979828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c0c6d00 0x7fce3c070100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fce3000e9e0 tx=0x7fce3000eea0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.708+0000 7fce43cd0640 1 -- 172.21.15.134:0/3569979828 shutdown_connections 2024-09-10T10:26:19.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.708+0000 7fce43cd0640 1 --2- 172.21.15.134:0/3569979828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce3c15b440 0x7fce3c1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.708+0000 7fce43cd0640 1 --2- 172.21.15.134:0/3569979828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3c158140 0x7fce3c070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.708+0000 7fce43cd0640 1 --2- 172.21.15.134:0/3569979828 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce3c0c6d00 0x7fce3c070100 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.708+0000 7fce43cd0640 1 -- 172.21.15.134:0/3569979828 >> 172.21.15.134:0/3569979828 conn(0x7fce3c0728b0 msgr2=0x7fce3c156eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:19.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.708+0000 7fce43cd0640 1 -- 172.21.15.134:0/3569979828 shutdown_connections 2024-09-10T10:26:19.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.708+0000 7fce43cd0640 1 -- 172.21.15.134:0/3569979828 wait complete. 2024-09-10T10:26:19.719 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 55523ec2-cf12-44b3-aeea-f1cedabc996a 6 2024-09-10T10:26:19.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.817+0000 7fc592e2e640 1 Processor -- start 2024-09-10T10:26:19.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.817+0000 7fc592e2e640 1 -- start start 2024-09-10T10:26:19.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc592e2e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc58c067d50 0x7fc58c0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc592e2e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc58c06bc10 0x7fc58c0cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc592e2e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc58c15b970 con 0x7fc58c0c7a80 2024-09-10T10:26:19.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc592e2e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc58c15b430 con 0x7fc58c06bc10 2024-09-10T10:26:19.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc592e2e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc58c0cc5e0 con 0x7fc58c067d50 2024-09-10T10:26:19.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc58bfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc58c067d50 0x7fc58c0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc590bca640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc58c0c7a80 0x7fc58c0c7e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56270/0 (socket says 172.21.15.134:56270) 2024-09-10T10:26:19.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc590bca640 1 -- 172.21.15.134:0/273957331 learned_addr learned my addr 172.21.15.134:0/273957331 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-10T10:26:19.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc5913cb640 1 --2- 172.21.15.134:0/273957331 >> v2:172.21.15.120:3300/0 conn(0x7fc58c06bc10 0x7fc58c0cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc58bfff640 1 -- 172.21.15.134:0/273957331 >> v2:172.21.15.120:3300/0 conn(0x7fc58c06bc10 msgr2=0x7fc58c0cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:19.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc58bfff640 1 --2- 172.21.15.134:0/273957331 >> v2:172.21.15.120:3300/0 conn(0x7fc58c06bc10 0x7fc58c0cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc58bfff640 1 -- 172.21.15.134:0/273957331 >> v1:172.21.15.134:6789/0 conn(0x7fc58c0c7a80 legacy=0x7fc58c0c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc58bfff640 1 -- 172.21.15.134:0/273957331 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c0c8570 con 0x7fc58c067d50 2024-09-10T10:26:19.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.818+0000 7fc58b7fe640 1 -- 172.21.15.134:0/273957331 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3805825075 0 0) 0x7fc58c15b970 con 0x7fc58c0c7a80 2024-09-10T10:26:19.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.819+0000 7fc58bfff640 1 --2- 172.21.15.134:0/273957331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 0x7fc58c0caa50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc58c06f2b0 tx=0x7fc57802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ca1fce1ab5f053 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.819+0000 7fc58b7fe640 1 -- 172.21.15.134:0/273957331 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578037070 con 0x7fc58c067d50 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.819+0000 7fc58b7fe640 1 -- 172.21.15.134:0/273957331 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc578032440 con 0x7fc58c067d50 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.819+0000 7fc58b7fe640 1 -- 172.21.15.134:0/273957331 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578032810 con 0x7fc58c067d50 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.820+0000 7fc592e2e640 1 -- 172.21.15.134:0/273957331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 msgr2=0x7fc58c0caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.820+0000 7fc592e2e640 1 --2- 172.21.15.134:0/273957331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 0x7fc58c0caa50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc58c06f2b0 tx=0x7fc57802a230 comp rx=0 tx=0).stop 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.820+0000 7fc592e2e640 1 -- 172.21.15.134:0/273957331 shutdown_connections 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.820+0000 7fc592e2e640 1 --2- 172.21.15.134:0/273957331 >> v2:172.21.15.120:3300/0 conn(0x7fc58c06bc10 0x7fc58c0cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.820+0000 7fc592e2e640 1 --2- 172.21.15.134:0/273957331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 0x7fc58c0caa50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.820+0000 7fc592e2e640 1 -- 172.21.15.134:0/273957331 >> 172.21.15.134:0/273957331 conn(0x7fc58c0c9220 msgr2=0x7fc58c0c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.820+0000 7fc592e2e640 1 -- 172.21.15.134:0/273957331 shutdown_connections 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.820+0000 7fc592e2e640 1 -- 172.21.15.134:0/273957331 wait complete. 2024-09-10T10:26:19.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.821+0000 7fc592e2e640 1 Processor -- start 2024-09-10T10:26:19.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.821+0000 7fc592e2e640 1 -- start start 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc592e2e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 0x7fc58c1f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc592e2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc58c06bc10 0x7fc58c157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc592e2e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58c0c7a80 0x7fc58c0c0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc592e2e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c0cb5c0 con 0x7fc58c06bc10 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc592e2e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c0cb2e0 con 0x7fc58c067d50 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc592e2e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc58c0cb8a0 con 0x7fc58c0c7a80 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc590bca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 0x7fc58c1f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc590bca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 0x7fc58c1f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:52708/0 (socket says 172.21.15.134:52708) 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc5913cb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58c0c7a80 0x7fc58c0c0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc590bca640 1 -- 172.21.15.134:0/1293517552 learned_addr learned my addr 172.21.15.134:0/1293517552 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:19.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc58bfff640 1 --2- 172.21.15.134:0/1293517552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc58c06bc10 0x7fc58c157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc590bca640 1 -- 172.21.15.134:0/1293517552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58c0c7a80 msgr2=0x7fc58c0c0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc590bca640 1 --2- 172.21.15.134:0/1293517552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58c0c7a80 0x7fc58c0c0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc590bca640 1 -- 172.21.15.134:0/1293517552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc58c06bc10 msgr2=0x7fc58c157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc590bca640 1 --2- 172.21.15.134:0/1293517552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc58c06bc10 0x7fc58c157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc590bca640 1 -- 172.21.15.134:0/1293517552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c1f4400 con 0x7fc58c067d50 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc5913cb640 1 --2- 172.21.15.134:0/1293517552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58c0c7a80 0x7fc58c0c0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc58bfff640 1 --2- 172.21.15.134:0/1293517552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc58c06bc10 0x7fc58c157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc590bca640 1 --2- 172.21.15.134:0/1293517552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 0x7fc58c1f3b40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc58000c4d0 tx=0x7fc58000c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc5897fa640 1 -- 172.21.15.134:0/1293517552 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc580018070 con 0x7fc58c067d50 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc5897fa640 1 -- 172.21.15.134:0/1293517552 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc580007500 con 0x7fc58c067d50 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc592e2e640 1 -- 172.21.15.134:0/1293517552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58c0bd480 con 0x7fc58c067d50 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc5897fa640 1 -- 172.21.15.134:0/1293517552 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc580004a20 con 0x7fc58c067d50 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.822+0000 7fc592e2e640 1 -- 172.21.15.134:0/1293517552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58c156e20 con 0x7fc58c067d50 2024-09-10T10:26:19.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.823+0000 7fc5897fa640 1 -- 172.21.15.134:0/1293517552 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc58001b070 con 0x7fc58c067d50 2024-09-10T10:26:19.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.823+0000 7fc5897fa640 1 -- 172.21.15.134:0/1293517552 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fc5800129f0 con 0x7fc58c067d50 2024-09-10T10:26:19.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.823+0000 7fc592e2e640 1 -- 172.21.15.134:0/1293517552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc58c154010 con 0x7fc58c067d50 2024-09-10T10:26:19.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.825+0000 7fc5897fa640 1 -- 172.21.15.134:0/1293517552 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc58001d440 con 0x7fc58c067d50 2024-09-10T10:26:19.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.894+0000 7fc592e2e640 1 -- 172.21.15.134:0/1293517552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "55523ec2-cf12-44b3-aeea-f1cedabc996a", "id": 6} v 0) -- 0x7fc58c0ca9d0 con 0x7fc58c067d50 2024-09-10T10:26:19.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.900+0000 7fc5897fa640 1 -- 172.21.15.134:0/1293517552 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "55523ec2-cf12-44b3-aeea-f1cedabc996a", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fc58001d7b0 con 0x7fc58c067d50 2024-09-10T10:26:19.901 INFO:teuthology.orchestra.run.smithi134.stdout:6 2024-09-10T10:26:19.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.901+0000 7fc592e2e640 1 -- 172.21.15.134:0/1293517552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 msgr2=0x7fc58c1f3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:19.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.901+0000 7fc592e2e640 1 --2- 172.21.15.134:0/1293517552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 0x7fc58c1f3b40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc58000c4d0 tx=0x7fc58000c990 comp rx=0 tx=0).stop 2024-09-10T10:26:19.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.901+0000 7fc592e2e640 1 -- 172.21.15.134:0/1293517552 shutdown_connections 2024-09-10T10:26:19.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.901+0000 7fc592e2e640 1 --2- 172.21.15.134:0/1293517552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc58c0c7a80 0x7fc58c0c0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.901+0000 7fc592e2e640 1 --2- 172.21.15.134:0/1293517552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc58c06bc10 0x7fc58c157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.901+0000 7fc592e2e640 1 --2- 172.21.15.134:0/1293517552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c067d50 0x7fc58c1f3b40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:19.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.901+0000 7fc592e2e640 1 -- 172.21.15.134:0/1293517552 >> 172.21.15.134:0/1293517552 conn(0x7fc58c0c9220 msgr2=0x7fc58c06cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:19.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.901+0000 7fc592e2e640 1 -- 172.21.15.134:0/1293517552 shutdown_connections 2024-09-10T10:26:19.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:19.901+0000 7fc592e2e640 1 -- 172.21.15.134:0/1293517552 wait complete. 2024-09-10T10:26:19.913 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 097b1149-3e3c-4e5a-890b-d4cc4361d51b 7 2024-09-10T10:26:20.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.010+0000 7f01de79f640 1 Processor -- start 2024-09-10T10:26:20.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.010+0000 7f01de79f640 1 -- start start 2024-09-10T10:26:20.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.011+0000 7f01de79f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f01d815b440 0x7f01d806e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:20.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.011+0000 7f01de79f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80c6d00 0x7f01d806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.011+0000 7f01de79f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d806c5a0 con 0x7f01d8158140 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.011+0000 7f01de79f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f01d80c8040 con 0x7f01d815b440 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.011+0000 7f01de79f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01d80c4b60 con 0x7f01d80c6d00 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.011+0000 7f01d77fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f01d815b440 0x7f01d806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.011+0000 7f01dcd3c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80c6d00 0x7f01d806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.011+0000 7f01d77fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f01d815b440 0x7f01d806e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:52718/0 (socket says 172.21.15.134:52718) 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.011+0000 7f01d77fe640 1 -- 172.21.15.134:0/4018976533 learned_addr learned my addr 172.21.15.134:0/4018976533 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.012+0000 7f01d77fe640 1 -- 172.21.15.134:0/4018976533 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80c6d00 msgr2=0x7f01d806ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.012+0000 7f01d77fe640 1 --2- 172.21.15.134:0/4018976533 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80c6d00 0x7f01d806ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.012+0000 7f01d77fe640 1 -- 172.21.15.134:0/4018976533 >> v1:172.21.15.120:6789/0 conn(0x7f01d8158140 legacy=0x7f01d806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.012+0000 7f01d77fe640 1 -- 172.21.15.134:0/4018976533 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d80c7a80 con 0x7f01d815b440 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.012+0000 7f01dcd3c640 1 --2- 172.21.15.134:0/4018976533 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80c6d00 0x7f01d806ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:20.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.012+0000 7f01d77fe640 1 --2- 172.21.15.134:0/4018976533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d815b440 0x7f01d806e770 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009870 tx=0x7f01cc02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8525a26ebf3d520 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:20.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.012+0000 7f01d6ffd640 1 -- 172.21.15.134:0/4018976533 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc037070 con 0x7f01d815b440 2024-09-10T10:26:20.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.012+0000 7f01d6ffd640 1 -- 172.21.15.134:0/4018976533 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc02ae50 con 0x7f01d815b440 2024-09-10T10:26:20.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.012+0000 7f01d6ffd640 1 -- 172.21.15.134:0/4018976533 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc032860 con 0x7f01d815b440 2024-09-10T10:26:20.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.013+0000 7f01de79f640 1 -- 172.21.15.134:0/4018976533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d815b440 msgr2=0x7f01d806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:20.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.013+0000 7f01de79f640 1 --2- 172.21.15.134:0/4018976533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d815b440 0x7f01d806e770 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009870 tx=0x7f01cc02a300 comp rx=0 tx=0).stop 2024-09-10T10:26:20.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.013+0000 7f01de79f640 1 -- 172.21.15.134:0/4018976533 shutdown_connections 2024-09-10T10:26:20.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.013+0000 7f01de79f640 1 --2- 172.21.15.134:0/4018976533 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80c6d00 0x7f01d806ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.013+0000 7f01de79f640 1 --2- 172.21.15.134:0/4018976533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d815b440 0x7f01d806e770 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.013+0000 7f01de79f640 1 -- 172.21.15.134:0/4018976533 >> 172.21.15.134:0/4018976533 conn(0x7f01d80728b0 msgr2=0x7f01d8072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:20.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.013+0000 7f01de79f640 1 -- 172.21.15.134:0/4018976533 shutdown_connections 2024-09-10T10:26:20.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.013+0000 7f01de79f640 1 -- 172.21.15.134:0/4018976533 wait complete. 2024-09-10T10:26:20.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.014+0000 7f01de79f640 1 Processor -- start 2024-09-10T10:26:20.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01de79f640 1 -- start start 2024-09-10T10:26:20.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01de79f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d80c6d00 0x7f01d81f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01de79f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d8158140 0x7f01d80cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01de79f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01d815b440 0x7f01d8147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01de79f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01d80735e0 con 0x7f01d815b440 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01de79f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01d806c840 con 0x7f01d80c6d00 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01de79f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01d80c0b20 con 0x7f01d8158140 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01d77fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d8158140 0x7f01d80cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01d7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d80c6d00 0x7f01d81f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01d77fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d8158140 0x7f01d80cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51976/0 (socket says 172.21.15.134:51976) 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.015+0000 7f01d77fe640 1 -- 172.21.15.134:0/1814466112 learned_addr learned my addr 172.21.15.134:0/1814466112 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01dcd3c640 1 --2- 172.21.15.134:0/1814466112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01d815b440 0x7f01d8147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d77fe640 1 -- 172.21.15.134:0/1814466112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d80c6d00 msgr2=0x7f01d81f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d77fe640 1 --2- 172.21.15.134:0/1814466112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d80c6d00 0x7f01d81f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d77fe640 1 -- 172.21.15.134:0/1814466112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01d815b440 msgr2=0x7f01d8147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d77fe640 1 --2- 172.21.15.134:0/1814466112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01d815b440 0x7f01d8147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d77fe640 1 -- 172.21.15.134:0/1814466112 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d8159b70 con 0x7f01d8158140 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d7fff640 1 --2- 172.21.15.134:0/1814466112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d80c6d00 0x7f01d81f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d77fe640 1 --2- 172.21.15.134:0/1814466112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d8158140 0x7f01d80cc230 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f01cc02a340 tx=0x7f01cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d4ff9640 1 -- 172.21.15.134:0/1814466112 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc041020 con 0x7f01d8158140 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d4ff9640 1 -- 172.21.15.134:0/1814466112 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc03c890 con 0x7f01d8158140 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01de79f640 1 -- 172.21.15.134:0/1814466112 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d8151090 con 0x7f01d8158140 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d4ff9640 1 -- 172.21.15.134:0/1814466112 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc037040 con 0x7f01d8158140 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01de79f640 1 -- 172.21.15.134:0/1814466112 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d8159de0 con 0x7f01d8158140 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d4ff9640 1 -- 172.21.15.134:0/1814466112 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f01cc044020 con 0x7f01d8158140 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.016+0000 7f01d4ff9640 1 -- 172.21.15.134:0/1814466112 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f01cc03db90 con 0x7f01d8158140 2024-09-10T10:26:20.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.017+0000 7f01de79f640 1 -- 172.21.15.134:0/1814466112 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d815a020 con 0x7f01d8158140 2024-09-10T10:26:20.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.019+0000 7f01d4ff9640 1 -- 172.21.15.134:0/1814466112 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f01d815a020 con 0x7f01d8158140 2024-09-10T10:26:20.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.089+0000 7f01de79f640 1 -- 172.21.15.134:0/1814466112 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "097b1149-3e3c-4e5a-890b-d4cc4361d51b", "id": 7} v 0) -- 0x7f01d80c7250 con 0x7f01d8158140 2024-09-10T10:26:20.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.097+0000 7f01d4ff9640 1 -- 172.21.15.134:0/1814466112 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "097b1149-3e3c-4e5a-890b-d4cc4361d51b", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f01d80c7250 con 0x7f01d8158140 2024-09-10T10:26:20.098 INFO:teuthology.orchestra.run.smithi134.stdout:7 2024-09-10T10:26:20.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.098+0000 7f01de79f640 1 -- 172.21.15.134:0/1814466112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d8158140 msgr2=0x7f01d80cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:20.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.098+0000 7f01de79f640 1 --2- 172.21.15.134:0/1814466112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d8158140 0x7f01d80cc230 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f01cc02a340 tx=0x7f01cc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.098+0000 7f01de79f640 1 -- 172.21.15.134:0/1814466112 shutdown_connections 2024-09-10T10:26:20.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.098+0000 7f01de79f640 1 --2- 172.21.15.134:0/1814466112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01d815b440 0x7f01d8147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.098+0000 7f01de79f640 1 --2- 172.21.15.134:0/1814466112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d8158140 0x7f01d80cc230 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.098+0000 7f01de79f640 1 --2- 172.21.15.134:0/1814466112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d80c6d00 0x7f01d81f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.098+0000 7f01de79f640 1 -- 172.21.15.134:0/1814466112 >> 172.21.15.134:0/1814466112 conn(0x7f01d80728b0 msgr2=0x7f01d8156580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:20.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.098+0000 7f01de79f640 1 -- 172.21.15.134:0/1814466112 shutdown_connections 2024-09-10T10:26:20.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-10T10:26:20.098+0000 7f01de79f640 1 -- 172.21.15.134:0/1814466112 wait complete. 2024-09-10T10:26:20.109 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T10:26:20.110 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T10:26:20.112 INFO:tasks.ceph.osd.0:Started 2024-09-10T10:26:20.112 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T10:26:20.112 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T10:26:20.114 INFO:tasks.ceph.osd.1:Started 2024-09-10T10:26:20.114 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T10:26:20.115 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T10:26:20.116 INFO:tasks.ceph.osd.2:Started 2024-09-10T10:26:20.117 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T10:26:20.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T10:26:20.119 INFO:tasks.ceph.osd.3:Started 2024-09-10T10:26:20.119 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T10:26:20.119 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T10:26:20.121 INFO:tasks.ceph.osd.4:Started 2024-09-10T10:26:20.121 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T10:26:20.121 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T10:26:20.123 INFO:tasks.ceph.osd.5:Started 2024-09-10T10:26:20.124 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T10:26:20.124 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T10:26:20.126 INFO:tasks.ceph.osd.6:Started 2024-09-10T10:26:20.126 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T10:26:20.126 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T10:26:20.128 INFO:tasks.ceph.osd.7:Started 2024-09-10T10:26:20.129 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:26:20.139 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:20.139+0000 7fc697d21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:26:20.206 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:20.205+0000 7fce8218a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:26:20.212 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:20.212+0000 7fc697d21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:26:20.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.255+0000 7fca39ff1640 1 Processor -- start 2024-09-10T10:26:20.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.255+0000 7fca39ff1640 1 -- start start 2024-09-10T10:26:20.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca39ff1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fca340cc0c0 0x7fca3415aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:20.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca39ff1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fca340cb2f0 0x7fca340cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:20.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca39ff1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3415b1d0 con 0x7fca3415e9d0 2024-09-10T10:26:20.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca39ff1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fca340cbd00 con 0x7fca340cb2f0 2024-09-10T10:26:20.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca39ff1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fca3415ba70 con 0x7fca340cc0c0 2024-09-10T10:26:20.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca32ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fca340cb2f0 0x7fca340cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:20.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca32ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fca340cb2f0 0x7fca340cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52638/0 (socket says 172.21.15.120:52638) 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca32ffd640 1 -- 172.21.15.120:0/1816047486 learned_addr learned my addr 172.21.15.120:0/1816047486 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca337fe640 1 --2- 172.21.15.120:0/1816047486 >> v2:172.21.15.134:3300/0 conn(0x7fca340cc0c0 0x7fca3415aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca32ffd640 1 -- 172.21.15.120:0/1816047486 >> v2:172.21.15.134:3300/0 conn(0x7fca340cc0c0 msgr2=0x7fca3415aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca32ffd640 1 --2- 172.21.15.120:0/1816047486 >> v2:172.21.15.134:3300/0 conn(0x7fca340cc0c0 0x7fca3415aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca32ffd640 1 -- 172.21.15.120:0/1816047486 >> v1:172.21.15.134:6789/0 conn(0x7fca3415e9d0 legacy=0x7fca340cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca32ffd640 1 -- 172.21.15.120:0/1816047486 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3415b4b0 con 0x7fca340cb2f0 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.256+0000 7fca337fe640 1 --2- 172.21.15.120:0/1816047486 >> v2:172.21.15.134:3300/0 conn(0x7fca340cc0c0 0x7fca3415aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.257+0000 7fca32ffd640 1 --2- 172.21.15.120:0/1816047486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 0x7fca340cc4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fca340ce150 tx=0x7fca2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18b94783f5557a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.257+0000 7fca327fc640 1 -- 172.21.15.120:0/1816047486 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28037070 con 0x7fca340cb2f0 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.257+0000 7fca327fc640 1 -- 172.21.15.120:0/1816047486 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca28032440 con 0x7fca340cb2f0 2024-09-10T10:26:20.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.257+0000 7fca327fc640 1 -- 172.21.15.120:0/1816047486 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28032810 con 0x7fca340cb2f0 2024-09-10T10:26:20.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.257+0000 7fca39ff1640 1 -- 172.21.15.120:0/1816047486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 msgr2=0x7fca340cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:20.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.257+0000 7fca39ff1640 1 --2- 172.21.15.120:0/1816047486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 0x7fca340cc4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fca340ce150 tx=0x7fca2802a270 comp rx=0 tx=0).stop 2024-09-10T10:26:20.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.257+0000 7fca39ff1640 1 -- 172.21.15.120:0/1816047486 shutdown_connections 2024-09-10T10:26:20.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.257+0000 7fca39ff1640 1 --2- 172.21.15.120:0/1816047486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 0x7fca340cc4a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.257+0000 7fca39ff1640 1 --2- 172.21.15.120:0/1816047486 >> v2:172.21.15.134:3300/0 conn(0x7fca340cc0c0 0x7fca3415aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.257+0000 7fca39ff1640 1 -- 172.21.15.120:0/1816047486 >> 172.21.15.120:0/1816047486 conn(0x7fca3406d560 msgr2=0x7fca3406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.258+0000 7fca39ff1640 1 -- 172.21.15.120:0/1816047486 shutdown_connections 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.258+0000 7fca39ff1640 1 -- 172.21.15.120:0/1816047486 wait complete. 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.259+0000 7fca39ff1640 1 Processor -- start 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.259+0000 7fca39ff1640 1 -- start start 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.259+0000 7fca39ff1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 0x7fca3415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca39ff1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca340cc0c0 0x7fca341f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca39ff1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca3415e9d0 0x7fca341f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca39ff1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fca3415e1d0 con 0x7fca340cb2f0 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca39ff1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca3415e540 con 0x7fca3415e9d0 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca39ff1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fca340c5560 con 0x7fca340cc0c0 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca337fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 0x7fca3415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca337fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 0x7fca3415d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52650/0 (socket says 172.21.15.120:52650) 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca337fe640 1 -- 172.21.15.120:0/1055000390 learned_addr learned my addr 172.21.15.120:0/1055000390 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:20.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca33fff640 1 --2- 172.21.15.120:0/1055000390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca3415e9d0 0x7fca341f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca32ffd640 1 --2- 172.21.15.120:0/1055000390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca340cc0c0 0x7fca341f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca337fe640 1 -- 172.21.15.120:0/1055000390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca340cc0c0 msgr2=0x7fca341f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca337fe640 1 --2- 172.21.15.120:0/1055000390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca340cc0c0 0x7fca341f16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca337fe640 1 -- 172.21.15.120:0/1055000390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca3415e9d0 msgr2=0x7fca341f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca337fe640 1 --2- 172.21.15.120:0/1055000390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca3415e9d0 0x7fca341f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca337fe640 1 -- 172.21.15.120:0/1055000390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca341520e0 con 0x7fca340cb2f0 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca337fe640 1 --2- 172.21.15.120:0/1055000390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 0x7fca3415d4d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fca2400d7d0 tx=0x7fca2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca1bfff640 1 -- 172.21.15.120:0/1055000390 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24004030 con 0x7fca340cb2f0 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca1bfff640 1 -- 172.21.15.120:0/1055000390 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca240041d0 con 0x7fca340cb2f0 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca39ff1640 1 -- 172.21.15.120:0/1055000390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca340c7bd0 con 0x7fca340cb2f0 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca1bfff640 1 -- 172.21.15.120:0/1055000390 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240096a0 con 0x7fca340cb2f0 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca39ff1640 1 -- 172.21.15.120:0/1055000390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca340c9410 con 0x7fca340cb2f0 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.260+0000 7fca1bfff640 1 -- 172.21.15.120:0/1055000390 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fca24009840 con 0x7fca340cb2f0 2024-09-10T10:26:20.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.261+0000 7fca1bfff640 1 -- 172.21.15.120:0/1055000390 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fca24014b00 con 0x7fca340cb2f0 2024-09-10T10:26:20.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.261+0000 7fca39ff1640 1 -- 172.21.15.120:0/1055000390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca340caab0 con 0x7fca340cb2f0 2024-09-10T10:26:20.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.263+0000 7fca1bfff640 1 -- 172.21.15.120:0/1055000390 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fca24014d00 con 0x7fca340cb2f0 2024-09-10T10:26:20.267 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:20.266+0000 7fce8218a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:26:20.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.322+0000 7fca39ff1640 1 -- 172.21.15.120:0/1055000390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca340cbea0 con 0x7fca340cb2f0 2024-09-10T10:26:20.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.323+0000 7fca1bfff640 1 -- 172.21.15.120:0/1055000390 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fca24009c70 con 0x7fca340cb2f0 2024-09-10T10:26:20.324 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:26:20.324 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":10,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:20.091201+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:26:20.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.324+0000 7fca39ff1640 1 -- 172.21.15.120:0/1055000390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 msgr2=0x7fca3415d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:20.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.324+0000 7fca39ff1640 1 --2- 172.21.15.120:0/1055000390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 0x7fca3415d4d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fca2400d7d0 tx=0x7fca2400dc90 comp rx=0 tx=0).stop 2024-09-10T10:26:20.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.324+0000 7fca39ff1640 1 -- 172.21.15.120:0/1055000390 shutdown_connections 2024-09-10T10:26:20.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.324+0000 7fca39ff1640 1 --2- 172.21.15.120:0/1055000390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca3415e9d0 0x7fca341f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.324+0000 7fca39ff1640 1 --2- 172.21.15.120:0/1055000390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca340cc0c0 0x7fca341f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.324+0000 7fca39ff1640 1 --2- 172.21.15.120:0/1055000390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca340cb2f0 0x7fca3415d4d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:20.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.324+0000 7fca39ff1640 1 -- 172.21.15.120:0/1055000390 >> 172.21.15.120:0/1055000390 conn(0x7fca3406d560 msgr2=0x7fca340cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:20.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.324+0000 7fca39ff1640 1 -- 172.21.15.120:0/1055000390 shutdown_connections 2024-09-10T10:26:20.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:20.324+0000 7fca39ff1640 1 -- 172.21.15.120:0/1055000390 wait complete. 2024-09-10T10:26:20.335 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T10:26:20.336 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T10:26:20.591 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:20.590+0000 7fc697d21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:26:20.635 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:20.634+0000 7fce8218a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:26:20.675 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:20.674+0000 7fc697d21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:26:20.718 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:20.716+0000 7fce8218a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:26:20.782 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:20.781+0000 7fc697d21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:26:20.827 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:20.826+0000 7fce8218a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:26:20.965 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:20.873+0000 7fce8218a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:26:20.965 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:20.829+0000 7fc697d21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:26:20.965 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:20.941+0000 7fc697d21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:26:20.987 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:20.985+0000 7fce8218a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:26:20.994 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:20.993+0000 7fc697d21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:26:21.040 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:21.038+0000 7fc697d21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:26:21.040 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:21.039+0000 7fce8218a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:26:21.085 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:26:21.083+0000 7fe70b431640 -1 Falling back to public interface 2024-09-10T10:26:21.086 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:21.085+0000 7fce8218a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:26:21.154 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:21.152+0000 7fc697d21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:26:21.198 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:21.197+0000 7fce8218a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:26:21.202 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:21.201+0000 7fc697d21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:26:21.247 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:21.246+0000 7fce8218a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:26:21.253 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:21.252+0000 7fc697d21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:26:21.266 DEBUG:teuthology.orchestra.run.smithi120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T10:26:21.297 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:21.296+0000 7fce8218a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:26:21.323 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:21.322+0000 7fc697d21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:26:21.329 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:26:21.328+0000 7fd136aa5640 -1 Falling back to public interface 2024-09-10T10:26:21.337 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:26:21.336+0000 7fa40fc0d640 -1 Falling back to public interface 2024-09-10T10:26:21.338 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:26:21.337+0000 7f8da3024640 -1 Falling back to public interface 2024-09-10T10:26:21.345 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:26:21.344+0000 7f8efaae4640 -1 Falling back to public interface 2024-09-10T10:26:21.360 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.359+0000 7fe996998640 1 Processor -- start 2024-09-10T10:26:21.360 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.359+0000 7fe996998640 1 -- start start 2024-09-10T10:26:21.363 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe996998640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900ca510 0x7fe9900c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:21.363 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe996998640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe9900cac80 0x7fe9900c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:21.363 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe996998640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9900cb230 con 0x7fe990152d60 2024-09-10T10:26:21.364 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe996998640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe9900c82d0 con 0x7fe9900cac80 2024-09-10T10:26:21.364 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe996998640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9900cb7f0 con 0x7fe9900ca510 2024-09-10T10:26:21.364 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe98f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900ca510 0x7fe9900c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:21.364 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe994f35640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe9900cac80 0x7fe9900c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:21.364 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe98f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900ca510 0x7fe9900c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:48174/0 (socket says 172.21.15.120:48174) 2024-09-10T10:26:21.364 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe98ffff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe990152d60 0x7fe9900c9610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:59548/0 (socket says 172.21.15.120:59548) 2024-09-10T10:26:21.364 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe98f7fe640 1 -- 172.21.15.120:0/3539286877 learned_addr learned my addr 172.21.15.120:0/3539286877 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:21.364 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe994f35640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe9900cac80 0x7fe9900c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:47734/0 (socket says 172.21.15.120:47734) 2024-09-10T10:26:21.365 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe98f7fe640 1 -- 172.21.15.120:0/3539286877 >> v2:172.21.15.134:3300/0 conn(0x7fe9900cac80 msgr2=0x7fe9900c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:21.365 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe98f7fe640 1 --2- 172.21.15.120:0/3539286877 >> v2:172.21.15.134:3300/0 conn(0x7fe9900cac80 0x7fe9900c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:21.365 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe98f7fe640 1 -- 172.21.15.120:0/3539286877 >> v1:172.21.15.134:6789/0 conn(0x7fe990152d60 legacy=0x7fe9900c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:21.365 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe98effd640 1 -- 172.21.15.120:0/3539286877 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2154718787 0 0) 0x7fe9900cb230 con 0x7fe990152d60 2024-09-10T10:26:21.365 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe98f7fe640 1 -- 172.21.15.120:0/3539286877 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9900cb510 con 0x7fe9900ca510 2024-09-10T10:26:21.365 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.360+0000 7fe994f35640 1 --2- 172.21.15.120:0/3539286877 >> v2:172.21.15.134:3300/0 conn(0x7fe9900cac80 0x7fe9900c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:21.365 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.361+0000 7fe98f7fe640 1 --2- 172.21.15.120:0/3539286877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900ca510 0x7fe9900c8af0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe978009870 tx=0x7fe97802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=521d4272b577276c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:21.365 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.361+0000 7fe98effd640 1 -- 172.21.15.120:0/3539286877 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978037070 con 0x7fe9900ca510 2024-09-10T10:26:21.365 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.361+0000 7fe98effd640 1 -- 172.21.15.120:0/3539286877 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe978032440 con 0x7fe9900ca510 2024-09-10T10:26:21.366 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.361+0000 7fe98effd640 1 -- 172.21.15.120:0/3539286877 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978032810 con 0x7fe9900ca510 2024-09-10T10:26:21.366 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.362+0000 7fe996998640 1 -- 172.21.15.120:0/3539286877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900ca510 msgr2=0x7fe9900c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:21.366 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.362+0000 7fe996998640 1 --2- 172.21.15.120:0/3539286877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900ca510 0x7fe9900c8af0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe978009870 tx=0x7fe97802a270 comp rx=0 tx=0).stop 2024-09-10T10:26:21.366 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.362+0000 7fe996998640 1 -- 172.21.15.120:0/3539286877 shutdown_connections 2024-09-10T10:26:21.366 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.362+0000 7fe996998640 1 --2- 172.21.15.120:0/3539286877 >> v2:172.21.15.134:3300/0 conn(0x7fe9900cac80 0x7fe9900c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:21.366 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.362+0000 7fe996998640 1 --2- 172.21.15.120:0/3539286877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900ca510 0x7fe9900c8af0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:21.366 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.362+0000 7fe996998640 1 -- 172.21.15.120:0/3539286877 >> 172.21.15.120:0/3539286877 conn(0x7fe990071630 msgr2=0x7fe990071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:21.366 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.362+0000 7fe996998640 1 -- 172.21.15.120:0/3539286877 shutdown_connections 2024-09-10T10:26:21.366 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.362+0000 7fe996998640 1 -- 172.21.15.120:0/3539286877 wait complete. 2024-09-10T10:26:21.368 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.363+0000 7fe996998640 1 Processor -- start 2024-09-10T10:26:21.368 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.363+0000 7fe996998640 1 -- start start 2024-09-10T10:26:21.368 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe996998640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9900ca510 0x7fe9901e0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe996998640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900cac80 0x7fe9900c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe996998640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe990152d60 0x7fe990150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe996998640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900c7db0 con 0x7fe9900ca510 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe996998640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900c7ad0 con 0x7fe990152d60 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe996998640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9900c1e10 con 0x7fe9900cac80 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe98ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9900ca510 0x7fe9901e0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe98ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9900ca510 0x7fe9901e0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52660/0 (socket says 172.21.15.120:52660) 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe98ffff640 1 -- 172.21.15.120:0/3751655396 learned_addr learned my addr 172.21.15.120:0/3751655396 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe994f35640 1 --2- 172.21.15.120:0/3751655396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe990152d60 0x7fe990150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe98f7fe640 1 --2- 172.21.15.120:0/3751655396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900cac80 0x7fe9900c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe98ffff640 1 -- 172.21.15.120:0/3751655396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900cac80 msgr2=0x7fe9900c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:21.369 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe98ffff640 1 --2- 172.21.15.120:0/3751655396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900cac80 0x7fe9900c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe98ffff640 1 -- 172.21.15.120:0/3751655396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe990152d60 msgr2=0x7fe990150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe98ffff640 1 --2- 172.21.15.120:0/3751655396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe990152d60 0x7fe990150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe98ffff640 1 -- 172.21.15.120:0/3751655396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9901e4690 con 0x7fe9900ca510 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe994f35640 1 --2- 172.21.15.120:0/3751655396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe990152d60 0x7fe990150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.364+0000 7fe98ffff640 1 --2- 172.21.15.120:0/3751655396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9900ca510 0x7fe9901e0720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c4d0 tx=0x7fe98000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.365+0000 7fe98cff9640 1 -- 172.21.15.120:0/3751655396 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9800074e0 con 0x7fe9900ca510 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.365+0000 7fe98cff9640 1 -- 172.21.15.120:0/3751655396 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9800047e0 con 0x7fe9900ca510 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.365+0000 7fe996998640 1 -- 172.21.15.120:0/3751655396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9901ee810 con 0x7fe9900ca510 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.365+0000 7fe98cff9640 1 -- 172.21.15.120:0/3751655396 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980004bd0 con 0x7fe9900ca510 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.365+0000 7fe996998640 1 -- 172.21.15.120:0/3751655396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9901eeab0 con 0x7fe9900ca510 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.365+0000 7fe98cff9640 1 -- 172.21.15.120:0/3751655396 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe980011070 con 0x7fe9900ca510 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.365+0000 7fe98cff9640 1 -- 172.21.15.120:0/3751655396 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fe980013a90 con 0x7fe9900ca510 2024-09-10T10:26:21.370 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.366+0000 7fe996998640 1 -- 172.21.15.120:0/3751655396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe990070960 con 0x7fe9900ca510 2024-09-10T10:26:21.371 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:21.368+0000 7fce8218a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:26:21.371 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.370+0000 7fe98cff9640 1 -- 172.21.15.120:0/3751655396 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe980004d70 con 0x7fe9900ca510 2024-09-10T10:26:21.373 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:21.372+0000 7fc697d21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:26:21.414 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:21.413+0000 7fce8218a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:26:21.418 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:21.416+0000 7fc697d21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:26:21.428 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.427+0000 7fe996998640 1 -- 172.21.15.120:0/3751655396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe990070d00 con 0x7fe9900ca510 2024-09-10T10:26:21.429 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.428+0000 7fe98cff9640 1 -- 172.21.15.120:0/3751655396 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fe980013ce0 con 0x7fe9900ca510 2024-09-10T10:26:21.430 INFO:teuthology.misc.health.smithi120.stdout: 2024-09-10T10:26:21.430 INFO:teuthology.misc.health.smithi120.stdout:{"epoch":10,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:20.091201+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:26:21.431 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.429+0000 7fe996998640 1 -- 172.21.15.120:0/3751655396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9900ca510 msgr2=0x7fe9901e0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:21.431 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.429+0000 7fe996998640 1 --2- 172.21.15.120:0/3751655396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9900ca510 0x7fe9901e0720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c4d0 tx=0x7fe98000c8d0 comp rx=0 tx=0).stop 2024-09-10T10:26:21.431 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.429+0000 7fe996998640 1 -- 172.21.15.120:0/3751655396 shutdown_connections 2024-09-10T10:26:21.431 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.429+0000 7fe996998640 1 --2- 172.21.15.120:0/3751655396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe990152d60 0x7fe990150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:21.431 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.429+0000 7fe996998640 1 --2- 172.21.15.120:0/3751655396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9900cac80 0x7fe9900c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:21.431 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.429+0000 7fe996998640 1 --2- 172.21.15.120:0/3751655396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9900ca510 0x7fe9901e0720 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:21.431 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.429+0000 7fe996998640 1 -- 172.21.15.120:0/3751655396 >> 172.21.15.120:0/3751655396 conn(0x7fe990071630 msgr2=0x7fe990158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:21.432 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.429+0000 7fe996998640 1 -- 172.21.15.120:0/3751655396 shutdown_connections 2024-09-10T10:26:21.432 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:21.429+0000 7fe996998640 1 -- 172.21.15.120:0/3751655396 wait complete. 2024-09-10T10:26:21.440 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T10:26:21.457 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:21.456+0000 7fce8218a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:26:21.582 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:26:21.581+0000 7fa162e15640 -1 Falling back to public interface 2024-09-10T10:26:21.589 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:26:21.588+0000 7f4ff0f58640 -1 Falling back to public interface 2024-09-10T10:26:21.826 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:26:21.825+0000 7fc37ac40640 -1 Falling back to public interface 2024-09-10T10:26:22.078 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:22.077+0000 7fc697d21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:26:22.120 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-10T10:26:22.119+0000 7fce8218a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:26:23.093 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:23.092+0000 7fc637d23640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T10:26:23.093 INFO:tasks.ceph.mgr.y.smithi120.stderr:2024-09-10T10:26:23.092+0000 7fc637d23640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T10:26:25.152 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:26:25.151+0000 7fe70b431640 -1 osd.7 0 log_to_monitors true 2024-09-10T10:26:25.164 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:26:25.163+0000 7f8da3024640 -1 osd.0 0 log_to_monitors true 2024-09-10T10:26:25.171 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:26:25.170+0000 7f8efaae4640 -1 osd.4 0 log_to_monitors true 2024-09-10T10:26:25.381 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:26:25.380+0000 7fd136aa5640 -1 osd.1 0 log_to_monitors true 2024-09-10T10:26:25.429 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:26:25.428+0000 7fa40fc0d640 -1 osd.5 0 log_to_monitors true 2024-09-10T10:26:25.575 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:26:25.572+0000 7fc37ac40640 -1 osd.3 0 log_to_monitors true 2024-09-10T10:26:25.603 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:26:25.601+0000 7fa162e15640 -1 osd.6 0 log_to_monitors true 2024-09-10T10:26:25.655 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:26:25.654+0000 7f4ff0f58640 -1 osd.2 0 log_to_monitors true 2024-09-10T10:26:28.058 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:26:28.057+0000 7fc376bcf640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T10:26:28.059 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:26:28.058+0000 7fa15e726640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T10:26:28.061 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:26:28.060+0000 7fa40b51e640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T10:26:28.069 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:26:28.068+0000 7f4fecee7640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T10:26:28.072 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:26:28.071+0000 7fa40333e640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:26:28.073 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:26:28.072+0000 7fa156546640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:26:28.084 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:26:28.083+0000 7fc36e9ef640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:26:28.088 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:26:28.085+0000 7fe7073c0640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T10:26:28.089 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:26:28.087+0000 7f8ef6a73640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T10:26:28.095 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:26:28.094+0000 7f4fe4d07640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:26:28.100 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:26:28.099+0000 7f8eee893640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:26:28.100 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:26:28.099+0000 7fe6ff1e0640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:26:28.246 DEBUG:teuthology.orchestra.run.smithi120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T10:26:28.277 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:26:28.275+0000 7f8d9efb3640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T10:26:28.289 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:26:28.288+0000 7f8d96dd3640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:26:28.371 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.370+0000 7f8d84cd0640 1 Processor -- start 2024-09-10T10:26:28.371 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.370+0000 7f8d84cd0640 1 -- start start 2024-09-10T10:26:28.373 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d84cd0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8d800c6c60 0x7f8d80158720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:28.373 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d84cd0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800ca860 0x7f8d800c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:28.373 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d84cd0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8d800c7040 0x7f8d801525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:28.373 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d84cd0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f8d800c8790 con 0x7f8d800c7040 2024-09-10T10:26:28.373 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d84cd0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f8d800c9d40 con 0x7f8d800c6c60 2024-09-10T10:26:28.373 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d84cd0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8d800ca0a0 con 0x7f8d800ca860 2024-09-10T10:26:28.373 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d7ed9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8d800c7040 0x7f8d801525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:28.373 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d7ed9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8d800c7040 0x7f8d801525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51654/0 (socket says 172.21.15.120:51654) 2024-09-10T10:26:28.373 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d7ed9d640 1 -- 172.21.15.120:0/4196173395 learned_addr learned my addr 172.21.15.120:0/4196173395 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:28.374 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d7dd9b640 1 --2- 172.21.15.120:0/4196173395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800ca860 0x7f8d800c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:28.374 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d7e59c640 1 --2- 172.21.15.120:0/4196173395 >> v2:172.21.15.134:3300/0 conn(0x7f8d800c6c60 0x7f8d80158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:28.374 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d7ed9d640 1 -- 172.21.15.120:0/4196173395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800ca860 msgr2=0x7f8d800c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:28.374 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d7ed9d640 1 --2- 172.21.15.120:0/4196173395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800ca860 0x7f8d800c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.374 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d7ed9d640 1 -- 172.21.15.120:0/4196173395 >> v2:172.21.15.134:3300/0 conn(0x7f8d800c6c60 msgr2=0x7f8d80158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:28.374 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d7ed9d640 1 --2- 172.21.15.120:0/4196173395 >> v2:172.21.15.134:3300/0 conn(0x7f8d800c6c60 0x7f8d80158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.374 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.371+0000 7f8d7ed9d640 1 -- 172.21.15.120:0/4196173395 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d800c94a0 con 0x7f8d800c7040 2024-09-10T10:26:28.375 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.372+0000 7f8d7ed9d640 1 --2- 172.21.15.120:0/4196173395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800c7040 0x7f8d801525e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d8006f1d0 tx=0x7f8d7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a123ba79dcc2c619 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:28.375 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.373+0000 7f8d7d59a640 1 -- 172.21.15.120:0/4196173395 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78036070 con 0x7f8d800c7040 2024-09-10T10:26:28.375 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.373+0000 7f8d7d59a640 1 -- 172.21.15.120:0/4196173395 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d78031600 con 0x7f8d800c7040 2024-09-10T10:26:28.375 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.373+0000 7f8d7d59a640 1 -- 172.21.15.120:0/4196173395 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d780319d0 con 0x7f8d800c7040 2024-09-10T10:26:28.376 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.373+0000 7f8d84cd0640 1 -- 172.21.15.120:0/4196173395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800c7040 msgr2=0x7f8d801525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:28.376 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.373+0000 7f8d84cd0640 1 --2- 172.21.15.120:0/4196173395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800c7040 0x7f8d801525e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d8006f1d0 tx=0x7f8d7802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.376 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.374+0000 7f8d84cd0640 1 -- 172.21.15.120:0/4196173395 shutdown_connections 2024-09-10T10:26:28.376 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.374+0000 7f8d84cd0640 1 --2- 172.21.15.120:0/4196173395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800c7040 0x7f8d801525e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.376 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.374+0000 7f8d84cd0640 1 --2- 172.21.15.120:0/4196173395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800ca860 0x7f8d800c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.376 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.374+0000 7f8d84cd0640 1 --2- 172.21.15.120:0/4196173395 >> v2:172.21.15.134:3300/0 conn(0x7f8d800c6c60 0x7f8d80158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.376 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.374+0000 7f8d84cd0640 1 -- 172.21.15.120:0/4196173395 >> 172.21.15.120:0/4196173395 conn(0x7f8d80072990 msgr2=0x7f8d80072d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:28.376 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.374+0000 7f8d84cd0640 1 -- 172.21.15.120:0/4196173395 shutdown_connections 2024-09-10T10:26:28.376 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.374+0000 7f8d84cd0640 1 -- 172.21.15.120:0/4196173395 wait complete. 2024-09-10T10:26:28.377 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.375+0000 7f8d84cd0640 1 Processor -- start 2024-09-10T10:26:28.377 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.376+0000 7f8d84cd0640 1 -- start start 2024-09-10T10:26:28.379 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d84cd0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d800c6c60 0x7f8d801dfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:28.379 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d84cd0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800c7040 0x7f8d801e1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:28.379 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d84cd0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800ca860 0x7f8d8014b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:28.379 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d84cd0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d800ca340 con 0x7f8d800ca860 2024-09-10T10:26:28.379 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d84cd0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d800c9a60 con 0x7f8d800c6c60 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d84cd0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8d80158c60 con 0x7f8d800c7040 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800ca860 0x7f8d8014b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7dd9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800c7040 0x7f8d801e1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d800c6c60 0x7f8d801dfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800ca860 0x7f8d8014b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51660/0 (socket says 172.21.15.120:51660) 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7ed9d640 1 -- 172.21.15.120:0/1773744565 learned_addr learned my addr 172.21.15.120:0/1773744565 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7ed9d640 1 -- 172.21.15.120:0/1773744565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800c7040 msgr2=0x7f8d801e1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7ed9d640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800c7040 0x7f8d801e1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7ed9d640 1 -- 172.21.15.120:0/1773744565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d800c6c60 msgr2=0x7f8d801dfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7ed9d640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d800c6c60 0x7f8d801dfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7ed9d640 1 -- 172.21.15.120:0/1773744565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d800cc8b0 con 0x7f8d800ca860 2024-09-10T10:26:28.380 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7dd9b640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800c7040 0x7f8d801e1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:28.381 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d7ed9d640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800ca860 0x7f8d8014b8e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d78034040 tx=0x7f8d78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:28.381 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d6effd640 1 -- 172.21.15.120:0/1773744565 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78036050 con 0x7f8d800ca860 2024-09-10T10:26:28.381 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d6effd640 1 -- 172.21.15.120:0/1773744565 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d78042070 con 0x7f8d800ca860 2024-09-10T10:26:28.381 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.377+0000 7f8d84cd0640 1 -- 172.21.15.120:0/1773744565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d801dff70 con 0x7f8d800ca860 2024-09-10T10:26:28.381 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.378+0000 7f8d84cd0640 1 -- 172.21.15.120:0/1773744565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d801f2640 con 0x7f8d800ca860 2024-09-10T10:26:28.381 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.378+0000 7f8d6effd640 1 -- 172.21.15.120:0/1773744565 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78009510 con 0x7f8d800ca860 2024-09-10T10:26:28.382 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.379+0000 7f8d84cd0640 1 -- 172.21.15.120:0/1773744565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d801598a0 con 0x7f8d800ca860 2024-09-10T10:26:28.382 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.379+0000 7f8d6effd640 1 -- 172.21.15.120:0/1773744565 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f8d78007bf0 con 0x7f8d800ca860 2024-09-10T10:26:28.382 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.380+0000 7f8d6effd640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8d6006bda0 0x7f8d6006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:28.382 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.380+0000 7f8d6effd640 1 -- 172.21.15.120:0/1773744565 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f8d780aab90 con 0x7f8d800ca860 2024-09-10T10:26:28.382 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.380+0000 7f8d7e59c640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8d6006bda0 0x7f8d6006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:28.382 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.380+0000 7f8d7e59c640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8d6006bda0 0x7f8d6006e260 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8d80149060 tx=0x7f8d70009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:28.389 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.388+0000 7f8d6effd640 1 -- 172.21.15.120:0/1773744565 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8d801598a0 con 0x7f8d800ca860 2024-09-10T10:26:28.393 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:26:28.392+0000 7fd133248640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T10:26:28.403 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:26:28.402+0000 7fd12a854640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:26:28.558 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.556+0000 7f8d84cd0640 1 -- 172.21.15.120:0/1773744565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d8006db20 con 0x7f8d800ca860 2024-09-10T10:26:28.558 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.557+0000 7f8d6effd640 1 -- 172.21.15.120:0/1773744565 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f8d78079d60 con 0x7f8d800ca860 2024-09-10T10:26:28.559 INFO:teuthology.misc.health.smithi120.stdout: 2024-09-10T10:26:28.559 INFO:teuthology.misc.health.smithi120.stdout:{"epoch":12,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:27.097770+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:26:28.561 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.559+0000 7f8d84cd0640 1 -- 172.21.15.120:0/1773744565 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8d6006bda0 msgr2=0x7f8d6006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:28.561 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.559+0000 7f8d84cd0640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8d6006bda0 0x7f8d6006e260 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8d80149060 tx=0x7f8d70009190 comp rx=0 tx=0).stop 2024-09-10T10:26:28.561 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.559+0000 7f8d84cd0640 1 -- 172.21.15.120:0/1773744565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800ca860 msgr2=0x7f8d8014b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:28.561 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.559+0000 7f8d84cd0640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800ca860 0x7f8d8014b8e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d78034040 tx=0x7f8d78004a40 comp rx=0 tx=0).stop 2024-09-10T10:26:28.561 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.559+0000 7f8d84cd0640 1 -- 172.21.15.120:0/1773744565 shutdown_connections 2024-09-10T10:26:28.561 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.559+0000 7f8d84cd0640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8d6006bda0 0x7f8d6006e260 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.561 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.559+0000 7f8d84cd0640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d800ca860 0x7f8d8014b8e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.561 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.559+0000 7f8d84cd0640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d800c7040 0x7f8d801e1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.561 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.559+0000 7f8d84cd0640 1 --2- 172.21.15.120:0/1773744565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d800c6c60 0x7f8d801dfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:28.561 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.559+0000 7f8d84cd0640 1 -- 172.21.15.120:0/1773744565 >> 172.21.15.120:0/1773744565 conn(0x7f8d80072990 msgr2=0x7f8d80158430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:28.562 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.560+0000 7f8d84cd0640 1 -- 172.21.15.120:0/1773744565 shutdown_connections 2024-09-10T10:26:28.562 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:28.560+0000 7f8d84cd0640 1 -- 172.21.15.120:0/1773744565 wait complete. 2024-09-10T10:26:28.570 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T10:26:35.377 DEBUG:teuthology.orchestra.run.smithi120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T10:26:35.477 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.476+0000 7fe242dfd640 1 Processor -- start 2024-09-10T10:26:35.477 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.476+0000 7fe242dfd640 1 -- start start 2024-09-10T10:26:35.480 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.477+0000 7fe242dfd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe23c0cb770 0x7fe23c154730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.480 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.477+0000 7fe242dfd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe23c15e160 con 0x7fe23c0cb390 2024-09-10T10:26:35.480 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.477+0000 7fe242dfd640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe23c15e400 con 0x7fe23c15b5d0 2024-09-10T10:26:35.480 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.477+0000 7fe242dfd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe23c15e6e0 con 0x7fe23c0cb770 2024-09-10T10:26:35.480 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.477+0000 7fe240b99640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fe23c0cb390 0x7fe23c15aec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:60592/0 (socket says 172.21.15.120:60592) 2024-09-10T10:26:35.480 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.477+0000 7fe240b99640 1 -- 172.21.15.120:0/897625165 learned_addr learned my addr 172.21.15.120:0/897625165 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.477+0000 7fe23bfff640 1 --2- 172.21.15.120:0/897625165 >> v2:172.21.15.134:3300/0 conn(0x7fe23c0cb770 0x7fe23c154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.477+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1824926287 0 0) 0x7fe23c15e160 con 0x7fe23c0cb390 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.477+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe22c003520 con 0x7fe23c0cb390 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3326864518 0 0) 0x7fe22c003520 con 0x7fe23c0cb390 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 >> v2:172.21.15.134:3300/0 conn(0x7fe23c0cb770 msgr2=0x7fe23c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 --2- 172.21.15.120:0/897625165 >> v2:172.21.15.134:3300/0 conn(0x7fe23c0cb770 0x7fe23c154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 >> v1:172.21.15.134:6789/0 conn(0x7fe23c15b5d0 legacy=0x7fe23c154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c15e9c0 con 0x7fe23c0cb390 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7fe230001690 con 0x7fe23c0cb390 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 >> v1:172.21.15.120:6789/0 conn(0x7fe23c0cb390 legacy=0x7fe23c15aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe22c003a20 0x7fe22c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.481 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe22c004800 0x7fe22c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe22c0051c0 0x7fe22c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c15e6e0 con 0x7fe22c0051c0 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c00c0f0 con 0x7fe22c004800 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c00c2f0 con 0x7fe22c003a20 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe24139a640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe22c0051c0 0x7fe22c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe240b99640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe22c003a20 0x7fe22c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe23bfff640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe22c004800 0x7fe22c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe24139a640 1 -- 172.21.15.120:0/897625165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe22c003a20 msgr2=0x7fe22c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe24139a640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe22c003a20 0x7fe22c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe24139a640 1 -- 172.21.15.120:0/897625165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe22c004800 msgr2=0x7fe22c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe24139a640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe22c004800 0x7fe22c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.478+0000 7fe24139a640 1 -- 172.21.15.120:0/897625165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c003520 con 0x7fe22c0051c0 2024-09-10T10:26:35.482 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.479+0000 7fe24139a640 1 -- 172.21.15.120:0/897625165 reap_dead start 2024-09-10T10:26:35.483 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.479+0000 7fe24139a640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe22c0051c0 0x7fe22c00b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe234001e50 tx=0x7fe22c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a748576ddfa906ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:35.483 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.479+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234001a70 con 0x7fe22c0051c0 2024-09-10T10:26:35.483 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.479+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe234003970 con 0x7fe22c0051c0 2024-09-10T10:26:35.483 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.479+0000 7fe23b7fe640 1 -- 172.21.15.120:0/897625165 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234003d60 con 0x7fe22c0051c0 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.479+0000 7fe242dfd640 1 -- 172.21.15.120:0/897625165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe22c0051c0 msgr2=0x7fe22c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.479+0000 7fe242dfd640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe22c0051c0 0x7fe22c00b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe234001e50 tx=0x7fe22c005830 comp rx=0 tx=0).stop 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.479+0000 7fe242dfd640 1 -- 172.21.15.120:0/897625165 shutdown_connections 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.479+0000 7fe242dfd640 1 --2- 172.21.15.120:0/897625165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe22c0051c0 0x7fe22c00b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.479+0000 7fe242dfd640 1 -- 172.21.15.120:0/897625165 >> 172.21.15.120:0/897625165 conn(0x7fe23c067aa0 msgr2=0x7fe23c067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.480+0000 7fe242dfd640 1 -- 172.21.15.120:0/897625165 shutdown_connections 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.480+0000 7fe242dfd640 1 -- 172.21.15.120:0/897625165 wait complete. 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.481+0000 7fe242dfd640 1 Processor -- start 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.481+0000 7fe242dfd640 1 -- start start 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe242dfd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe22c0051c0 0x7fe23c157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.486 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe242dfd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe23c0d0ae0 0x7fe23c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe242dfd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23c156b90 0x7fe23c1f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe242dfd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c0cc710 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe242dfd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c0cc590 con 0x7fe22c0051c0 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe242dfd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe23c0cc890 con 0x7fe23c156b90 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe23bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe23c0d0ae0 0x7fe23c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe23bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe23c0d0ae0 0x7fe23c156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51708/0 (socket says 172.21.15.120:51708) 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe23bfff640 1 -- 172.21.15.120:0/3122745405 learned_addr learned my addr 172.21.15.120:0/3122745405 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe240b99640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe22c0051c0 0x7fe23c157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe24139a640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23c156b90 0x7fe23c1f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe23bfff640 1 -- 172.21.15.120:0/3122745405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23c156b90 msgr2=0x7fe23c1f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe23bfff640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23c156b90 0x7fe23c1f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe23bfff640 1 -- 172.21.15.120:0/3122745405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe22c0051c0 msgr2=0x7fe23c157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.487 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe23bfff640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe22c0051c0 0x7fe23c157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe23bfff640 1 -- 172.21.15.120:0/3122745405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c0ce430 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe240b99640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe22c0051c0 0x7fe23c157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe23bfff640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe23c0d0ae0 0x7fe23c156650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe2280118e0 tx=0x7fe228011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe2397fa640 1 -- 172.21.15.120:0/3122745405 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228005c60 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe2397fa640 1 -- 172.21.15.120:0/3122745405 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe228005e00 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe2397fa640 1 -- 172.21.15.120:0/3122745405 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22800fa60 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe242dfd640 1 -- 172.21.15.120:0/3122745405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23c0ce6d0 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.482+0000 7fe242dfd640 1 -- 172.21.15.120:0/3122745405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23c0cec90 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.483+0000 7fe242dfd640 1 -- 172.21.15.120:0/3122745405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe23c0cd460 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.484+0000 7fe2397fa640 1 -- 172.21.15.120:0/3122745405 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe228012040 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.484+0000 7fe2397fa640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe22006be30 0x7fe22006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.488 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.484+0000 7fe2397fa640 1 -- 172.21.15.120:0/3122745405 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe22808b2d0 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.489 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.484+0000 7fe240b99640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe22006be30 0x7fe22006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.489 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.485+0000 7fe240b99640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe22006be30 0x7fe22006e2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe23c15e330 tx=0x7fe230009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:35.490 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.489+0000 7fe2397fa640 1 -- 172.21.15.120:0/3122745405 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe228018030 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.656 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.655+0000 7fe242dfd640 1 -- 172.21.15.120:0/3122745405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe23c0b5f90 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.657 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.656+0000 7fe2397fa640 1 -- 172.21.15.120:0/3122745405 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12920 (secure 0 0 0) 0x7fe22801b360 con 0x7fe23c0d0ae0 2024-09-10T10:26:35.657 INFO:teuthology.misc.health.smithi120.stdout: 2024-09-10T10:26:35.657 INFO:teuthology.misc.health.smithi120.stdout:{"epoch":17,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:33.140570+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:26:35.659 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.657+0000 7fe242dfd640 1 -- 172.21.15.120:0/3122745405 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe22006be30 msgr2=0x7fe22006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.659 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe22006be30 0x7fe22006e2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe23c15e330 tx=0x7fe230009040 comp rx=0 tx=0).stop 2024-09-10T10:26:35.659 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 -- 172.21.15.120:0/3122745405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe23c0d0ae0 msgr2=0x7fe23c156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.659 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe23c0d0ae0 0x7fe23c156650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe2280118e0 tx=0x7fe228011da0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.659 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 -- 172.21.15.120:0/3122745405 shutdown_connections 2024-09-10T10:26:35.659 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe22006be30 0x7fe22006e2f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.659 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23c156b90 0x7fe23c1f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.659 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe23c0d0ae0 0x7fe23c156650 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.660 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 --2- 172.21.15.120:0/3122745405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe22c0051c0 0x7fe23c157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.660 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 -- 172.21.15.120:0/3122745405 >> 172.21.15.120:0/3122745405 conn(0x7fe23c067aa0 msgr2=0x7fe23c067e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:35.660 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 -- 172.21.15.120:0/3122745405 shutdown_connections 2024-09-10T10:26:35.660 INFO:teuthology.misc.health.smithi120.stderr:2024-09-10T10:26:35.658+0000 7fe242dfd640 1 -- 172.21.15.120:0/3122745405 wait complete. 2024-09-10T10:26:35.668 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T10:26:35.669 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T10:26:35.669 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:26:35.669 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:26:35.671 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:26:35.672 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:26:35.672 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:26:35.674 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:26:35.674 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-10T10:26:35.674 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-10T10:26:35.676 INFO:tasks.ceph.mds.e:Started 2024-09-10T10:26:35.676 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:26:35.676 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:26:35.679 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:26:35.679 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:26:35.679 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:26:35.682 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:26:35.682 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-10T10:26:35.682 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-10T10:26:35.684 INFO:tasks.ceph.mds.f:Started 2024-09-10T10:26:35.684 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T10:26:35.684 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:26:35.748 INFO:tasks.ceph.mds.a.smithi120.stdout:starting mds.a at 2024-09-10T10:26:35.748 INFO:tasks.ceph.mds.c.smithi120.stdout:starting mds.c at 2024-09-10T10:26:35.751 INFO:tasks.ceph.mds.e.smithi120.stdout:starting mds.e at 2024-09-10T10:26:35.768 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-10T10:26:35.769 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-10T10:26:35.769 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-09-10T10:26:35.767+0000 7f8914cf8640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-10T10:26:35.769 INFO:tasks.ceph.mds.f.smithi134.stdout:starting mds.f at 2024-09-10T10:26:35.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.788+0000 7fd5041bd640 1 Processor -- start 2024-09-10T10:26:35.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.789+0000 7fd5041bd640 1 -- start start 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.789+0000 7fd5041bd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd4fc0cb2f0 0x7fd4fc15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.789+0000 7fd5041bd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4fc15b1d0 con 0x7fd4fc0cc0c0 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.789+0000 7fd5041bd640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4fc15b4b0 con 0x7fd4fc0cc4a0 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.789+0000 7fd5041bd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd4fc0cbd00 con 0x7fd4fc0cb2f0 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.789+0000 7fd501f59640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fd4fc0cc0c0 0x7fd4fc15e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:60634/0 (socket says 172.21.15.120:60634) 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.789+0000 7fd501f59640 1 -- 172.21.15.120:0/2840143259 learned_addr learned my addr 172.21.15.120:0/2840143259 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3558399242 0 0) 0x7fd4fc15b1d0 con 0x7fd4fc0cc0c0 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4e8003520 con 0x7fd4fc0cc0c0 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd501758640 1 --2- 172.21.15.120:0/2840143259 >> v2:172.21.15.134:3300/0 conn(0x7fd4fc0cb2f0 0x7fd4fc15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1599950556 0 0) 0x7fd4e8003520 con 0x7fd4fc0cc0c0 2024-09-10T10:26:35.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 >> v2:172.21.15.134:3300/0 conn(0x7fd4fc0cb2f0 msgr2=0x7fd4fc15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 --2- 172.21.15.120:0/2840143259 >> v2:172.21.15.134:3300/0 conn(0x7fd4fc0cb2f0 0x7fd4fc15aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 >> v1:172.21.15.134:6789/0 conn(0x7fd4fc0cc4a0 legacy=0x7fd4fc165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4fc15b790 con 0x7fd4fc0cc0c0 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7fd4f8001670 con 0x7fd4fc0cc0c0 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 >> v1:172.21.15.120:6789/0 conn(0x7fd4fc0cc0c0 legacy=0x7fd4fc15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4e80039b0 0x7fd4e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4e8004800 0x7fd4e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80051c0 0x7fd4e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc0cbd00 con 0x7fd4e80051c0 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e800c060 con 0x7fd4e80039b0 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e800c260 con 0x7fd4e8004800 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd501f59640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4e80039b0 0x7fd4e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd50275a640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80051c0 0x7fd4e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd501758640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4e8004800 0x7fd4e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd501f59640 1 -- 172.21.15.120:0/2840143259 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4e8004800 msgr2=0x7fd4e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd501f59640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4e8004800 0x7fd4e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd501f59640 1 -- 172.21.15.120:0/2840143259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80051c0 msgr2=0x7fd4e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd501f59640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80051c0 0x7fd4e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd501f59640 1 -- 172.21.15.120:0/2840143259 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e8003520 con 0x7fd4e80039b0 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.790+0000 7fd50275a640 1 -- 172.21.15.120:0/2840143259 reap_dead start 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.791+0000 7fd501f59640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4e80039b0 0x7fd4e8003dd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800c240 tx=0x7fd4e8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7906678143063e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.791+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8007cf0 con 0x7fd4e80039b0 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.791+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f80058d0 con 0x7fd4e80039b0 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.791+0000 7fd500f57640 1 -- 172.21.15.120:0/2840143259 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8005ca0 con 0x7fd4e80039b0 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.792+0000 7fd5041bd640 1 -- 172.21.15.120:0/2840143259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4e80039b0 msgr2=0x7fd4e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.792+0000 7fd5041bd640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4e80039b0 0x7fd4e8003dd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800c240 tx=0x7fd4e8005830 comp rx=0 tx=0).stop 2024-09-10T10:26:35.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.792+0000 7fd5041bd640 1 -- 172.21.15.120:0/2840143259 shutdown_connections 2024-09-10T10:26:35.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.792+0000 7fd5041bd640 1 --2- 172.21.15.120:0/2840143259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4e80039b0 0x7fd4e8003dd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.792+0000 7fd5041bd640 1 -- 172.21.15.120:0/2840143259 >> 172.21.15.120:0/2840143259 conn(0x7fd4fc06d560 msgr2=0x7fd4fc06d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:35.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.792+0000 7fd5041bd640 1 -- 172.21.15.120:0/2840143259 shutdown_connections 2024-09-10T10:26:35.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.792+0000 7fd5041bd640 1 -- 172.21.15.120:0/2840143259 wait complete. 2024-09-10T10:26:35.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.793+0000 7fd5041bd640 1 Processor -- start 2024-09-10T10:26:35.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.793+0000 7fd5041bd640 1 -- start start 2024-09-10T10:26:35.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd5041bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80039b0 0x7fd4fc1ecce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd5041bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4fc0c9ea0 0x7fd4fc159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd5041bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4fc0c92d0 0x7fd4fc1f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd5041bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc15e540 con 0x7fd4e80039b0 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd5041bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc15ba70 con 0x7fd4fc0c92d0 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd5041bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd4fc0cc8b0 con 0x7fd4fc0c9ea0 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd501f59640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80039b0 0x7fd4fc1ecce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd501f59640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80039b0 0x7fd4fc1ecce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51774/0 (socket says 172.21.15.120:51774) 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd501f59640 1 -- 172.21.15.120:0/2067311364 learned_addr learned my addr 172.21.15.120:0/2067311364 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd50275a640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4fc0c92d0 0x7fd4fc1f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd501758640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4fc0c9ea0 0x7fd4fc159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd501f59640 1 -- 172.21.15.120:0/2067311364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4fc0c9ea0 msgr2=0x7fd4fc159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd501f59640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4fc0c9ea0 0x7fd4fc159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd501f59640 1 -- 172.21.15.120:0/2067311364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4fc0c92d0 msgr2=0x7fd4fc1f00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd501f59640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4fc0c92d0 0x7fd4fc1f00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.794+0000 7fd501f59640 1 -- 172.21.15.120:0/2067311364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4fc1f0620 con 0x7fd4e80039b0 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.795+0000 7fd501f59640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80039b0 0x7fd4fc1ecce0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800c240 tx=0x7fd4fc0b45c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.795+0000 7fd4f27fc640 1 -- 172.21.15.120:0/2067311364 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8007cf0 con 0x7fd4e80039b0 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.795+0000 7fd4f27fc640 1 -- 172.21.15.120:0/2067311364 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f8041070 con 0x7fd4e80039b0 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.795+0000 7fd4f27fc640 1 -- 172.21.15.120:0/2067311364 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8001c30 con 0x7fd4e80039b0 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.795+0000 7fd5041bd640 1 -- 172.21.15.120:0/2067311364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4fc0ca350 con 0x7fd4e80039b0 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.795+0000 7fd5041bd640 1 -- 172.21.15.120:0/2067311364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4fc0ca610 con 0x7fd4e80039b0 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.796+0000 7fd5041bd640 1 -- 172.21.15.120:0/2067311364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4fc0c5560 con 0x7fd4e80039b0 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.796+0000 7fd4f27fc640 1 -- 172.21.15.120:0/2067311364 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4f803b070 con 0x7fd4e80039b0 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.796+0000 7fd4f27fc640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd4e006beb0 0x7fd4e006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:35.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.797+0000 7fd4f27fc640 1 -- 172.21.15.120:0/2067311364 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd4f80aa0f0 con 0x7fd4e80039b0 2024-09-10T10:26:35.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.797+0000 7fd501758640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd4e006beb0 0x7fd4e006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:35.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.797+0000 7fd501758640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd4e006beb0 0x7fd4e006e370 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc1e6ad0 tx=0x7fd4ec009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:35.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.801+0000 7fd4f27fc640 1 -- 172.21.15.120:0/2067311364 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd4f8078880 con 0x7fd4e80039b0 2024-09-10T10:26:35.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.998+0000 7fd5041bd640 1 -- 172.21.15.120:0/2067311364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4fc0cdd40 con 0x7fd4e80039b0 2024-09-10T10:26:36.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:35.999+0000 7fd4f27fc640 1 -- 172.21.15.120:0/2067311364 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1437 (secure 0 0 0) 0x7fd4f807cb80 con 0x7fd4e80039b0 2024-09-10T10:26:36.000 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:26:36.000 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":2,"btime":"2024-09-10T10:26:35:752531+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T10:26:36.001 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 2 2024-09-10T10:26:36.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 -- 172.21.15.120:0/2067311364 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd4e006beb0 msgr2=0x7fd4e006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:36.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd4e006beb0 0x7fd4e006e370 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc1e6ad0 tx=0x7fd4ec009190 comp rx=0 tx=0).stop 2024-09-10T10:26:36.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 -- 172.21.15.120:0/2067311364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80039b0 msgr2=0x7fd4fc1ecce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:36.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80039b0 0x7fd4fc1ecce0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800c240 tx=0x7fd4fc0b45c0 comp rx=0 tx=0).stop 2024-09-10T10:26:36.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 -- 172.21.15.120:0/2067311364 shutdown_connections 2024-09-10T10:26:36.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd4e006beb0 0x7fd4e006e370 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:36.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4fc0c92d0 0x7fd4fc1f00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:36.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4fc0c9ea0 0x7fd4fc159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:36.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 --2- 172.21.15.120:0/2067311364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4e80039b0 0x7fd4fc1ecce0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:36.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 -- 172.21.15.120:0/2067311364 >> 172.21.15.120:0/2067311364 conn(0x7fd4fc06d560 msgr2=0x7fd4fc1500e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:36.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 -- 172.21.15.120:0/2067311364 shutdown_connections 2024-09-10T10:26:36.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:36.001+0000 7fd5041bd640 1 -- 172.21.15.120:0/2067311364 wait complete. 2024-09-10T10:26:38.013 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:26:38.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.122+0000 7fd57b89b640 1 Processor -- start 2024-09-10T10:26:38.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.123+0000 7fd57b89b640 1 -- start start 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.123+0000 7fd57b89b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd5740c6980 0x7fd5740cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.123+0000 7fd57b89b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5740cb410 0x7fd5740cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.123+0000 7fd57b89b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57415b1a0 con 0x7fd5740c7770 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.123+0000 7fd57b89b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fd57415b9e0 con 0x7fd5740c6980 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.123+0000 7fd57b89b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd57415b420 con 0x7fd5740cb410 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.124+0000 7fd579637640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd5740c6980 0x7fd5740cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.124+0000 7fd579637640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd5740c6980 0x7fd5740cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47298/0 (socket says 172.21.15.120:47298) 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.124+0000 7fd579637640 1 -- 172.21.15.120:0/1753629791 learned_addr learned my addr 172.21.15.120:0/1753629791 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.124+0000 7fd579e38640 1 --2- 172.21.15.120:0/1753629791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5740cb410 0x7fd5740cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.124+0000 7fd579637640 1 -- 172.21.15.120:0/1753629791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5740cb410 msgr2=0x7fd5740cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.124+0000 7fd579637640 1 --2- 172.21.15.120:0/1753629791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5740cb410 0x7fd5740cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.124+0000 7fd579637640 1 -- 172.21.15.120:0/1753629791 >> v1:172.21.15.134:6789/0 conn(0x7fd5740c7770 legacy=0x7fd57415aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.124+0000 7fd579637640 1 -- 172.21.15.120:0/1753629791 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57415b700 con 0x7fd5740c6980 2024-09-10T10:26:38.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd579637640 1 --2- 172.21.15.120:0/1753629791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c6980 0x7fd5740cd730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd570009870 tx=0x7fd57002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b87d36ff677d974 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:38.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd56bfff640 1 -- 172.21.15.120:0/1753629791 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570037070 con 0x7fd5740c6980 2024-09-10T10:26:38.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd56bfff640 1 -- 172.21.15.120:0/1753629791 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd570032440 con 0x7fd5740c6980 2024-09-10T10:26:38.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd56bfff640 1 -- 172.21.15.120:0/1753629791 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570032810 con 0x7fd5740c6980 2024-09-10T10:26:38.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd57b89b640 1 -- 172.21.15.120:0/1753629791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c6980 msgr2=0x7fd5740cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd57b89b640 1 --2- 172.21.15.120:0/1753629791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c6980 0x7fd5740cd730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd570009870 tx=0x7fd57002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd57b89b640 1 -- 172.21.15.120:0/1753629791 shutdown_connections 2024-09-10T10:26:38.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd57b89b640 1 --2- 172.21.15.120:0/1753629791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5740cb410 0x7fd5740cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd57b89b640 1 --2- 172.21.15.120:0/1753629791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c6980 0x7fd5740cd730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd57b89b640 1 -- 172.21.15.120:0/1753629791 >> 172.21.15.120:0/1753629791 conn(0x7fd57406cb70 msgr2=0x7fd57406e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:38.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd57b89b640 1 -- 172.21.15.120:0/1753629791 shutdown_connections 2024-09-10T10:26:38.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.125+0000 7fd57b89b640 1 -- 172.21.15.120:0/1753629791 wait complete. 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd57b89b640 1 Processor -- start 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd57b89b640 1 -- start start 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd57b89b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5740c6980 0x7fd5741f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd57b89b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c7770 0x7fd574157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd57b89b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5740cb410 0x7fd574153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd57b89b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd5740cd210 con 0x7fd5740c7770 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd57b89b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5740ccf30 con 0x7fd5740c6980 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd57b89b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd5740d0fe0 con 0x7fd5740cb410 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd578e36640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c7770 0x7fd574157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd578e36640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c7770 0x7fd574157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47308/0 (socket says 172.21.15.120:47308) 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.127+0000 7fd578e36640 1 -- 172.21.15.120:0/1419248911 learned_addr learned my addr 172.21.15.120:0/1419248911 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd579637640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5740c6980 0x7fd5741f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd578e36640 1 -- 172.21.15.120:0/1419248911 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5740cb410 msgr2=0x7fd574153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd579e38640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5740cb410 0x7fd574153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd578e36640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5740cb410 0x7fd574153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd578e36640 1 -- 172.21.15.120:0/1419248911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5740c6980 msgr2=0x7fd5741f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd578e36640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5740c6980 0x7fd5741f11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd578e36640 1 -- 172.21.15.120:0/1419248911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5741f1730 con 0x7fd5740c7770 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd578e36640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c7770 0x7fd574157840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c480 tx=0x7fd56400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd569ffb640 1 -- 172.21.15.120:0/1419248911 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640052d0 con 0x7fd5740c7770 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd569ffb640 1 -- 172.21.15.120:0/1419248911 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd564005470 con 0x7fd5740c7770 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd569ffb640 1 -- 172.21.15.120:0/1419248911 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564012870 con 0x7fd5740c7770 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd57b89b640 1 -- 172.21.15.120:0/1419248911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5740c58b0 con 0x7fd5740c7770 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.128+0000 7fd57b89b640 1 -- 172.21.15.120:0/1419248911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5740cf640 con 0x7fd5740c7770 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.129+0000 7fd57b89b640 1 -- 172.21.15.120:0/1419248911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5740cd730 con 0x7fd5740c7770 2024-09-10T10:26:38.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.129+0000 7fd569ffb640 1 -- 172.21.15.120:0/1419248911 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd564012a10 con 0x7fd5740c7770 2024-09-10T10:26:38.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.130+0000 7fd569ffb640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd55806be30 0x7fd55806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.130+0000 7fd569ffb640 1 -- 172.21.15.120:0/1419248911 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd56408a790 con 0x7fd5740c7770 2024-09-10T10:26:38.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.130+0000 7fd579637640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd55806be30 0x7fd55806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.130+0000 7fd579637640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd55806be30 0x7fd55806e2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd570034000 tx=0x7fd57002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:38.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.134+0000 7fd569ffb640 1 -- 172.21.15.120:0/1419248911 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd57415b1a0 con 0x7fd5740c7770 2024-09-10T10:26:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.333+0000 7fd57b89b640 1 -- 172.21.15.120:0/1419248911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5740cbdc0 con 0x7fd5740c7770 2024-09-10T10:26:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.334+0000 7fd569ffb640 1 -- 172.21.15.120:0/1419248911 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5563 (secure 0 0 0) 0x7fd564059490 con 0x7fd5740c7770 2024-09-10T10:26:38.336 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:26:38.336 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":3,"btime":"2024-09-10T10:26:36:753651+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T10:26:38.336 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 3 2024-09-10T10:26:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 -- 172.21.15.120:0/1419248911 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd55806be30 msgr2=0x7fd55806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd55806be30 0x7fd55806e2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd570034000 tx=0x7fd57002e000 comp rx=0 tx=0).stop 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 -- 172.21.15.120:0/1419248911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c7770 msgr2=0x7fd574157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c7770 0x7fd574157840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c480 tx=0x7fd56400c940 comp rx=0 tx=0).stop 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 -- 172.21.15.120:0/1419248911 shutdown_connections 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd55806be30 0x7fd55806e2f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5740cb410 0x7fd574153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5740c7770 0x7fd574157840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 --2- 172.21.15.120:0/1419248911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5740c6980 0x7fd5741f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 -- 172.21.15.120:0/1419248911 >> 172.21.15.120:0/1419248911 conn(0x7fd57406cb70 msgr2=0x7fd5740c4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 -- 172.21.15.120:0/1419248911 shutdown_connections 2024-09-10T10:26:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.336+0000 7fd57b89b640 1 -- 172.21.15.120:0/1419248911 wait complete. 2024-09-10T10:26:38.348 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:26:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.445+0000 7f271816d640 1 Processor -- start 2024-09-10T10:26:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.445+0000 7f271816d640 1 -- start start 2024-09-10T10:26:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f271816d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c6410 0x7f2710158040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f271816d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f27100c5280 0x7f27100c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f271816d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2710151e70 con 0x7f2710151a90 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f271816d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f27100c8130 con 0x7f27100c5280 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f271816d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27100c8410 con 0x7f27100c6410 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f2715708640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c6410 0x7f2710158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f271670a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f27100c5280 0x7f27100c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f2715f09640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f2710151a90 0x7f27100c36b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:59174/0 (socket says 172.21.15.120:59174) 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f2715708640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c6410 0x7f2710158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:38858/0 (socket says 172.21.15.120:38858) 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f2715f09640 1 -- 172.21.15.120:0/642258438 learned_addr learned my addr 172.21.15.120:0/642258438 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f2714f07640 1 -- 172.21.15.120:0/642258438 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 881218902 0 0) 0x7f2710151e70 con 0x7f2710151a90 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f2715708640 1 -- 172.21.15.120:0/642258438 >> v2:172.21.15.134:3300/0 conn(0x7f27100c5280 msgr2=0x7f27100c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f2715708640 1 --2- 172.21.15.120:0/642258438 >> v2:172.21.15.134:3300/0 conn(0x7f27100c5280 0x7f27100c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f2715708640 1 -- 172.21.15.120:0/642258438 >> v1:172.21.15.134:6789/0 conn(0x7f2710151a90 legacy=0x7f27100c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.446+0000 7f2715708640 1 -- 172.21.15.120:0/642258438 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27100c86f0 con 0x7f27100c6410 2024-09-10T10:26:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.447+0000 7f2715708640 1 --2- 172.21.15.120:0/642258438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c6410 0x7f2710158040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f270c0097a0 tx=0x7f270c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89d8fe8f62ff5cec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.447+0000 7f2714f07640 1 -- 172.21.15.120:0/642258438 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c037070 con 0x7f27100c6410 2024-09-10T10:26:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.447+0000 7f2714f07640 1 -- 172.21.15.120:0/642258438 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f270c032440 con 0x7f27100c6410 2024-09-10T10:26:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.447+0000 7f2714f07640 1 -- 172.21.15.120:0/642258438 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c032810 con 0x7f27100c6410 2024-09-10T10:26:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.448+0000 7f271816d640 1 -- 172.21.15.120:0/642258438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c6410 msgr2=0x7f2710158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.448+0000 7f271816d640 1 --2- 172.21.15.120:0/642258438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c6410 0x7f2710158040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f270c0097a0 tx=0x7f270c02a180 comp rx=0 tx=0).stop 2024-09-10T10:26:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.448+0000 7f271816d640 1 -- 172.21.15.120:0/642258438 shutdown_connections 2024-09-10T10:26:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.448+0000 7f271816d640 1 --2- 172.21.15.120:0/642258438 >> v2:172.21.15.134:3300/0 conn(0x7f27100c5280 0x7f27100c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.448+0000 7f271816d640 1 --2- 172.21.15.120:0/642258438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c6410 0x7f2710158040 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.448+0000 7f271816d640 1 -- 172.21.15.120:0/642258438 >> 172.21.15.120:0/642258438 conn(0x7f271006f3c0 msgr2=0x7f271006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.448+0000 7f271816d640 1 -- 172.21.15.120:0/642258438 shutdown_connections 2024-09-10T10:26:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.448+0000 7f271816d640 1 -- 172.21.15.120:0/642258438 wait complete. 2024-09-10T10:26:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.449+0000 7f271816d640 1 Processor -- start 2024-09-10T10:26:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.449+0000 7f271816d640 1 -- start start 2024-09-10T10:26:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.449+0000 7f271816d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c5280 0x7f271014d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.449+0000 7f271816d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27100c6410 0x7f27100cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f271816d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2710151a90 0x7f2710147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f271816d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2710157b20 con 0x7f27100c6410 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f271816d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2710157840 con 0x7f2710151a90 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f271816d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2710158580 con 0x7f27100c5280 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f2715708640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27100c6410 0x7f27100cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f2715708640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27100c6410 0x7f27100cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47324/0 (socket says 172.21.15.120:47324) 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f2715708640 1 -- 172.21.15.120:0/2623931787 learned_addr learned my addr 172.21.15.120:0/2623931787 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f2715f09640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c5280 0x7f271014d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f271670a640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2710151a90 0x7f2710147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f2715708640 1 -- 172.21.15.120:0/2623931787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c5280 msgr2=0x7f271014d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f2715708640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c5280 0x7f271014d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f2715708640 1 -- 172.21.15.120:0/2623931787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2710151a90 msgr2=0x7f2710147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f2715708640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2710151a90 0x7f2710147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f2715708640 1 -- 172.21.15.120:0/2623931787 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2710154530 con 0x7f27100c6410 2024-09-10T10:26:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f2715708640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27100c6410 0x7f27100cb3b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f270c035040 tx=0x7f270c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f27067fc640 1 -- 172.21.15.120:0/2623931787 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c037040 con 0x7f27100c6410 2024-09-10T10:26:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f27067fc640 1 -- 172.21.15.120:0/2623931787 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f270c004230 con 0x7f27100c6410 2024-09-10T10:26:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f271816d640 1 -- 172.21.15.120:0/2623931787 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271014c8e0 con 0x7f27100c6410 2024-09-10T10:26:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f27067fc640 1 -- 172.21.15.120:0/2623931787 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c003610 con 0x7f27100c6410 2024-09-10T10:26:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.450+0000 7f271816d640 1 -- 172.21.15.120:0/2623931787 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101e0e50 con 0x7f27100c6410 2024-09-10T10:26:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.451+0000 7f271816d640 1 -- 172.21.15.120:0/2623931787 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2710072980 con 0x7f27100c6410 2024-09-10T10:26:38.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.452+0000 7f27067fc640 1 -- 172.21.15.120:0/2623931787 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f270c0037b0 con 0x7f27100c6410 2024-09-10T10:26:38.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.452+0000 7f27067fc640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26f406be60 0x7f26f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.452+0000 7f27067fc640 1 -- 172.21.15.120:0/2623931787 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f270c0aba50 con 0x7f27100c6410 2024-09-10T10:26:38.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.452+0000 7f2715f09640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26f406be60 0x7f26f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.453+0000 7f2715f09640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26f406be60 0x7f26f406e320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2700000f60 tx=0x7f2700007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:38.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.457+0000 7f27067fc640 1 -- 172.21.15.120:0/2623931787 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2710072980 con 0x7f27100c6410 2024-09-10T10:26:38.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.622+0000 7f271816d640 1 -- 172.21.15.120:0/2623931787 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2710075400 con 0x7f27100c6410 2024-09-10T10:26:38.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.622+0000 7f27067fc640 1 -- 172.21.15.120:0/2623931787 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f270c07a2b0 con 0x7f27100c6410 2024-09-10T10:26:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:26:38.624 INFO:teuthology.orchestra.run.smithi120.stdout:[ 2024-09-10T10:26:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:26:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "poolnum": 1, 2024-09-10T10:26:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "poolname": ".mgr" 2024-09-10T10:26:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:26:38.624 INFO:teuthology.orchestra.run.smithi120.stdout:] 2024-09-10T10:26:38.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 -- 172.21.15.120:0/2623931787 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26f406be60 msgr2=0x7f26f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26f406be60 0x7f26f406e320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2700000f60 tx=0x7f2700007e40 comp rx=0 tx=0).stop 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 -- 172.21.15.120:0/2623931787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27100c6410 msgr2=0x7f27100cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27100c6410 0x7f27100cb3b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f270c035040 tx=0x7f270c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 -- 172.21.15.120:0/2623931787 shutdown_connections 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26f406be60 0x7f26f406e320 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2710151a90 0x7f2710147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27100c6410 0x7f27100cb3b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 --2- 172.21.15.120:0/2623931787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27100c5280 0x7f271014d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 -- 172.21.15.120:0/2623931787 >> 172.21.15.120:0/2623931787 conn(0x7f271006f3c0 msgr2=0x7f2710155390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 -- 172.21.15.120:0/2623931787 shutdown_connections 2024-09-10T10:26:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.624+0000 7f271816d640 1 -- 172.21.15.120:0/2623931787 wait complete. 2024-09-10T10:26:38.636 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:26:38.636 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:26:38.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.736+0000 7f3e625f1640 1 Processor -- start 2024-09-10T10:26:38.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.736+0000 7f3e625f1640 1 -- start start 2024-09-10T10:26:38.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.737+0000 7f3e625f1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3e5c0b2f20 0x7f3e5c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.737+0000 7f3e625f1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3e5c15ab30 0x7f3e5c154970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.737+0000 7f3e625f1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e5c0cc950 con 0x7f3e5c0cd480 2024-09-10T10:26:38.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.737+0000 7f3e625f1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f3e5c15e1d0 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.737+0000 7f3e625f1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3e5c0cc0b0 con 0x7f3e5c15ab30 2024-09-10T10:26:38.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.737+0000 7f3e5b7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3e5c0b2f20 0x7f3e5c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.737+0000 7f3e5b7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3e5c0b2f20 0x7f3e5c15e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47330/0 (socket says 172.21.15.120:47330) 2024-09-10T10:26:38.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.737+0000 7f3e5b7fe640 1 -- 172.21.15.120:0/2827333354 learned_addr learned my addr 172.21.15.120:0/2827333354 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:38.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.737+0000 7f3e5bfff640 1 --1- 172.21.15.120:0/2827333354 >> v1:172.21.15.120:6789/0 conn(0x7f3e5c0cd480 0x7f3e5c0cb320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57646/0 (socket says 172.21.15.120:57646) 2024-09-10T10:26:38.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.737+0000 7f3e60b8e640 1 --2- 172.21.15.120:0/2827333354 >> v2:172.21.15.134:3300/0 conn(0x7f3e5c15ab30 0x7f3e5c154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.738+0000 7f3e5b7fe640 1 -- 172.21.15.120:0/2827333354 >> v2:172.21.15.134:3300/0 conn(0x7f3e5c15ab30 msgr2=0x7f3e5c154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:38.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.738+0000 7f3e5b7fe640 1 --2- 172.21.15.120:0/2827333354 >> v2:172.21.15.134:3300/0 conn(0x7f3e5c15ab30 0x7f3e5c154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.738+0000 7f3e5b7fe640 1 -- 172.21.15.120:0/2827333354 >> v1:172.21.15.120:6789/0 conn(0x7f3e5c0cd480 legacy=0x7f3e5c0cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.738+0000 7f3e5b7fe640 1 -- 172.21.15.120:0/2827333354 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c0cc390 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.738+0000 7f3e5affd640 1 -- 172.21.15.120:0/2827333354 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 259755254 0 0) 0x7f3e5c0cc950 con 0x7f3e5c0cd480 2024-09-10T10:26:38.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.738+0000 7f3e5b7fe640 1 --2- 172.21.15.120:0/2827333354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 0x7f3e5c15e990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0b4010 tx=0x7f3e5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662e789cd128a3cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:38.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.738+0000 7f3e5affd640 1 -- 172.21.15.120:0/2827333354 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50037070 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.738+0000 7f3e5affd640 1 -- 172.21.15.120:0/2827333354 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e50032440 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.738+0000 7f3e5affd640 1 -- 172.21.15.120:0/2827333354 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50032810 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.739+0000 7f3e625f1640 1 -- 172.21.15.120:0/2827333354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 msgr2=0x7f3e5c15e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.739+0000 7f3e625f1640 1 --2- 172.21.15.120:0/2827333354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 0x7f3e5c15e990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0b4010 tx=0x7f3e5002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.739+0000 7f3e625f1640 1 -- 172.21.15.120:0/2827333354 shutdown_connections 2024-09-10T10:26:38.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.739+0000 7f3e625f1640 1 --2- 172.21.15.120:0/2827333354 >> v2:172.21.15.134:3300/0 conn(0x7f3e5c15ab30 0x7f3e5c154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.739+0000 7f3e625f1640 1 --2- 172.21.15.120:0/2827333354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 0x7f3e5c15e990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.739+0000 7f3e625f1640 1 -- 172.21.15.120:0/2827333354 >> 172.21.15.120:0/2827333354 conn(0x7f3e5c067b30 msgr2=0x7f3e5c067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:38.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.739+0000 7f3e625f1640 1 -- 172.21.15.120:0/2827333354 shutdown_connections 2024-09-10T10:26:38.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.739+0000 7f3e625f1640 1 -- 172.21.15.120:0/2827333354 wait complete. 2024-09-10T10:26:38.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.740+0000 7f3e625f1640 1 Processor -- start 2024-09-10T10:26:38.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.740+0000 7f3e625f1640 1 -- start start 2024-09-10T10:26:38.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e625f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 0x7f3e5c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e625f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c0cd480 0x7f3e5c1f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e625f1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e5c15ab30 0x7f3e5c1f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e625f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c15eed0 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e625f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c15e470 con 0x7f3e5c0cd480 2024-09-10T10:26:38.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e625f1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5c0cd860 con 0x7f3e5c15ab30 2024-09-10T10:26:38.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e5bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 0x7f3e5c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e5bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 0x7f3e5c14d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47336/0 (socket says 172.21.15.120:47336) 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e5bfff640 1 -- 172.21.15.120:0/3366941062 learned_addr learned my addr 172.21.15.120:0/3366941062 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e5bfff640 1 -- 172.21.15.120:0/3366941062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e5c15ab30 msgr2=0x7f3e5c1f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e5bfff640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e5c15ab30 0x7f3e5c1f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e5bfff640 1 -- 172.21.15.120:0/3366941062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c0cd480 msgr2=0x7f3e5c1f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e5bfff640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c0cd480 0x7f3e5c1f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e5bfff640 1 -- 172.21.15.120:0/3366941062 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c0c6870 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.741+0000 7f3e5bfff640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 0x7f3e5c14d920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c500 tx=0x7f3e4c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.742+0000 7f3e58ff9640 1 -- 172.21.15.120:0/3366941062 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c018070 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.742+0000 7f3e58ff9640 1 -- 172.21.15.120:0/3366941062 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e4c007580 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.742+0000 7f3e625f1640 1 -- 172.21.15.120:0/3366941062 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c0c6b50 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.742+0000 7f3e58ff9640 1 -- 172.21.15.120:0/3366941062 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c004a20 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.742+0000 7f3e625f1640 1 -- 172.21.15.120:0/3366941062 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c0cf100 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.743+0000 7f3e625f1640 1 -- 172.21.15.120:0/3366941062 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5c0694c0 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.743+0000 7f3e58ff9640 1 -- 172.21.15.120:0/3366941062 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e4c01b070 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.743+0000 7f3e58ff9640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3e4806be30 0x7f3e4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:38.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.743+0000 7f3e58ff9640 1 -- 172.21.15.120:0/3366941062 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f3e4c089e80 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.743+0000 7f3e5b7fe640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3e4806be30 0x7f3e4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:38.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.744+0000 7f3e5b7fe640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3e4806be30 0x7f3e4806e2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3e50034000 tx=0x7f3e5002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:38.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.748+0000 7f3e58ff9640 1 -- 172.21.15.120:0/3366941062 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3e5c0694c0 con 0x7f3e5c0b2f20 2024-09-10T10:26:38.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:38.918+0000 7f3e625f1640 1 -- 172.21.15.120:0/3366941062 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3e5c068290 con 0x7f3e5c0b2f20 2024-09-10T10:26:39.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.766+0000 7f3e58ff9640 1 -- 172.21.15.120:0/3366941062 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f3e4c058b80 con 0x7f3e5c0b2f20 2024-09-10T10:26:39.767 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_metadata' created 2024-09-10T10:26:39.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.769+0000 7f3e625f1640 1 -- 172.21.15.120:0/3366941062 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3e4806be30 msgr2=0x7f3e4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:39.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.769+0000 7f3e625f1640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3e4806be30 0x7f3e4806e2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3e50034000 tx=0x7f3e5002c000 comp rx=0 tx=0).stop 2024-09-10T10:26:39.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.769+0000 7f3e625f1640 1 -- 172.21.15.120:0/3366941062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 msgr2=0x7f3e5c14d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:39.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.769+0000 7f3e625f1640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 0x7f3e5c14d920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c500 tx=0x7f3e4c00c9c0 comp rx=0 tx=0).stop 2024-09-10T10:26:39.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.769+0000 7f3e625f1640 1 -- 172.21.15.120:0/3366941062 shutdown_connections 2024-09-10T10:26:39.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.769+0000 7f3e625f1640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3e4806be30 0x7f3e4806e2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:39.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.769+0000 7f3e625f1640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e5c15ab30 0x7f3e5c1f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:39.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.769+0000 7f3e625f1640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c0cd480 0x7f3e5c1f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:39.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.769+0000 7f3e625f1640 1 --2- 172.21.15.120:0/3366941062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e5c0b2f20 0x7f3e5c14d920 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:39.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.769+0000 7f3e625f1640 1 -- 172.21.15.120:0/3366941062 >> 172.21.15.120:0/3366941062 conn(0x7f3e5c067b30 msgr2=0x7f3e5c153f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:39.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.770+0000 7f3e625f1640 1 -- 172.21.15.120:0/3366941062 shutdown_connections 2024-09-10T10:26:39.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.770+0000 7f3e625f1640 1 -- 172.21.15.120:0/3366941062 wait complete. 2024-09-10T10:26:39.788 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T10:26:39.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.897+0000 7fb17c986640 1 Processor -- start 2024-09-10T10:26:39.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.897+0000 7fb17c986640 1 -- start start 2024-09-10T10:26:39.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.898+0000 7fb17c986640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1780c8d50 0x7fb1780c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:39.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.898+0000 7fb17c986640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb178073100 0x7fb178151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:39.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.898+0000 7fb17c986640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1780c5780 con 0x7fb178072d20 2024-09-10T10:26:39.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.898+0000 7fb17c986640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb1780c5a60 con 0x7fb178073100 2024-09-10T10:26:39.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.898+0000 7fb17c986640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1780c5d40 con 0x7fb1780c8d50 2024-09-10T10:26:39.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.899+0000 7fb176d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb178073100 0x7fb178151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:39.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.899+0000 7fb176d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb178073100 0x7fb178151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47344/0 (socket says 172.21.15.120:47344) 2024-09-10T10:26:39.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.899+0000 7fb176d9d640 1 -- 172.21.15.120:0/3298248449 learned_addr learned my addr 172.21.15.120:0/3298248449 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:39.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.899+0000 7fb175d9b640 1 --2- 172.21.15.120:0/3298248449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1780c8d50 0x7fb1780c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:39.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.899+0000 7fb176d9d640 1 -- 172.21.15.120:0/3298248449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1780c8d50 msgr2=0x7fb1780c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:39.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.899+0000 7fb176d9d640 1 --2- 172.21.15.120:0/3298248449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1780c8d50 0x7fb1780c37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:39.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.899+0000 7fb176d9d640 1 -- 172.21.15.120:0/3298248449 >> v1:172.21.15.134:6789/0 conn(0x7fb178072d20 legacy=0x7fb1780c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:39.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.899+0000 7fb176d9d640 1 -- 172.21.15.120:0/3298248449 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1780c75f0 con 0x7fb178073100 2024-09-10T10:26:39.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb176d9d640 1 --2- 172.21.15.120:0/3298248449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 0x7fb178151550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb170009e50 tx=0x7fb17002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525871530f91200 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:39.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17559a640 1 -- 172.21.15.120:0/3298248449 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170036070 con 0x7fb178073100 2024-09-10T10:26:39.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17559a640 1 -- 172.21.15.120:0/3298248449 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb170031600 con 0x7fb178073100 2024-09-10T10:26:39.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17559a640 1 -- 172.21.15.120:0/3298248449 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1700319d0 con 0x7fb178073100 2024-09-10T10:26:39.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17c986640 1 -- 172.21.15.120:0/3298248449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 msgr2=0x7fb178151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:39.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17c986640 1 --2- 172.21.15.120:0/3298248449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 0x7fb178151550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb170009e50 tx=0x7fb17002a750 comp rx=0 tx=0).stop 2024-09-10T10:26:39.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17c986640 1 -- 172.21.15.120:0/3298248449 shutdown_connections 2024-09-10T10:26:39.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17c986640 1 --2- 172.21.15.120:0/3298248449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 0x7fb178151550 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:39.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17c986640 1 --2- 172.21.15.120:0/3298248449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1780c8d50 0x7fb1780c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:39.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17c986640 1 -- 172.21.15.120:0/3298248449 >> 172.21.15.120:0/3298248449 conn(0x7fb17806e670 msgr2=0x7fb17806ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:39.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17c986640 1 -- 172.21.15.120:0/3298248449 shutdown_connections 2024-09-10T10:26:39.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.900+0000 7fb17c986640 1 -- 172.21.15.120:0/3298248449 wait complete. 2024-09-10T10:26:39.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb17c986640 1 Processor -- start 2024-09-10T10:26:39.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb17c986640 1 -- start start 2024-09-10T10:26:39.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb17c986640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb178072d20 0x7fb17814f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:39.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb17c986640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 0x7fb1781dd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:39.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb17c986640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1780c8d50 0x7fb178151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb17c986640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1780c3d00 con 0x7fb178073100 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb17c986640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1780c72c0 con 0x7fb1780c8d50 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb17c986640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1780c65e0 con 0x7fb178072d20 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb175d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 0x7fb1781dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb175d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 0x7fb1781dd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47354/0 (socket says 172.21.15.120:47354) 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.902+0000 7fb175d9b640 1 -- 172.21.15.120:0/4216567677 learned_addr learned my addr 172.21.15.120:0/4216567677 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb176d9d640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1780c8d50 0x7fb178151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb17659c640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb178072d20 0x7fb17814f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb175d9b640 1 -- 172.21.15.120:0/4216567677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb178072d20 msgr2=0x7fb17814f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb175d9b640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb178072d20 0x7fb17814f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb175d9b640 1 -- 172.21.15.120:0/4216567677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1780c8d50 msgr2=0x7fb178151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb175d9b640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1780c8d50 0x7fb178151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb175d9b640 1 -- 172.21.15.120:0/4216567677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17814fa40 con 0x7fb178073100 2024-09-10T10:26:39.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb176d9d640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1780c8d50 0x7fb178151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:26:39.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb17659c640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb178072d20 0x7fb17814f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:26:39.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb175d9b640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 0x7fb1781dd890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb16000ea70 tx=0x7fb16000ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:39.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb16effd640 1 -- 172.21.15.120:0/4216567677 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160004220 con 0x7fb178073100 2024-09-10T10:26:39.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb16effd640 1 -- 172.21.15.120:0/4216567677 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1600043c0 con 0x7fb178073100 2024-09-10T10:26:39.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb17c986640 1 -- 172.21.15.120:0/4216567677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1781e1660 con 0x7fb178073100 2024-09-10T10:26:39.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb16effd640 1 -- 172.21.15.120:0/4216567677 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160007b70 con 0x7fb178073100 2024-09-10T10:26:39.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.903+0000 7fb17c986640 1 -- 172.21.15.120:0/4216567677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1781501d0 con 0x7fb178073100 2024-09-10T10:26:39.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.904+0000 7fb17c986640 1 -- 172.21.15.120:0/4216567677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1780c6cc0 con 0x7fb178073100 2024-09-10T10:26:39.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.905+0000 7fb16effd640 1 -- 172.21.15.120:0/4216567677 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb16000b030 con 0x7fb178073100 2024-09-10T10:26:39.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.905+0000 7fb16effd640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb15806be30 0x7fb15806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:39.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.905+0000 7fb16effd640 1 -- 172.21.15.120:0/4216567677 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fb16008cfd0 con 0x7fb178073100 2024-09-10T10:26:39.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.905+0000 7fb17659c640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb15806be30 0x7fb15806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:39.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.905+0000 7fb17659c640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb15806be30 0x7fb15806e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb168003eb0 tx=0x7fb168009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:39.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:39.910+0000 7fb16effd640 1 -- 172.21.15.120:0/4216567677 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb160016330 con 0x7fb178073100 2024-09-10T10:26:40.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.076+0000 7fb17c986640 1 -- 172.21.15.120:0/4216567677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fb1780c4f80 con 0x7fb178073100 2024-09-10T10:26:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.770+0000 7fb16effd640 1 -- 172.21.15.120:0/4216567677 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7fb16005bae0 con 0x7fb178073100 2024-09-10T10:26:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_data' created 2024-09-10T10:26:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 -- 172.21.15.120:0/4216567677 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb15806be30 msgr2=0x7fb15806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb15806be30 0x7fb15806e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb168003eb0 tx=0x7fb168009040 comp rx=0 tx=0).stop 2024-09-10T10:26:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 -- 172.21.15.120:0/4216567677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 msgr2=0x7fb1781dd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 0x7fb1781dd890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb16000ea70 tx=0x7fb16000ef30 comp rx=0 tx=0).stop 2024-09-10T10:26:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 -- 172.21.15.120:0/4216567677 shutdown_connections 2024-09-10T10:26:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb15806be30 0x7fb15806e2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1780c8d50 0x7fb178151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb178073100 0x7fb1781dd890 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 --2- 172.21.15.120:0/4216567677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb178072d20 0x7fb17814f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 -- 172.21.15.120:0/4216567677 >> 172.21.15.120:0/4216567677 conn(0x7fb17806e670 msgr2=0x7fb1780c33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 -- 172.21.15.120:0/4216567677 shutdown_connections 2024-09-10T10:26:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.773+0000 7fb17c986640 1 -- 172.21.15.120:0/4216567677 wait complete. 2024-09-10T10:26:40.800 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:26:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.901+0000 7f45d9f9f640 1 Processor -- start 2024-09-10T10:26:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.901+0000 7f45d9f9f640 1 -- start start 2024-09-10T10:26:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.902+0000 7f45d9f9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d4154ac0 0x7f45d415b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.902+0000 7f45d9f9f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f45d415bc50 0x7f45d415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.902+0000 7f45d9f9f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d4154ea0 con 0x7f45d406d820 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.902+0000 7f45d9f9f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f45d415c620 con 0x7f45d415bc50 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.902+0000 7f45d9f9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f45d415b6d0 con 0x7f45d4154ac0 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.902+0000 7f45d979e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f45d415bc50 0x7f45d415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.902+0000 7f45d979e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f45d415bc50 0x7f45d415c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47364/0 (socket says 172.21.15.120:47364) 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.902+0000 7f45d979e640 1 -- 172.21.15.120:0/2546556640 learned_addr learned my addr 172.21.15.120:0/2546556640 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.903+0000 7f45d979e640 1 -- 172.21.15.120:0/2546556640 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d4154ac0 msgr2=0x7f45d415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.903+0000 7f45d8f9d640 1 --2- 172.21.15.120:0/2546556640 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d4154ac0 0x7f45d415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.903+0000 7f45d979e640 1 --2- 172.21.15.120:0/2546556640 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d4154ac0 0x7f45d415b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.903+0000 7f45d979e640 1 -- 172.21.15.120:0/2546556640 >> v1:172.21.15.134:6789/0 conn(0x7f45d406d820 legacy=0x7f45d40d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.903+0000 7f45d979e640 1 -- 172.21.15.120:0/2546556640 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d40c5340 con 0x7f45d415bc50 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.903+0000 7f45d979e640 1 --2- 172.21.15.120:0/2546556640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d415bc50 0x7f45d415c0e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009e80 tx=0x7f45cc02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04b47ca9b82b5c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.903+0000 7f45cb7fe640 1 -- 172.21.15.120:0/2546556640 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc036070 con 0x7f45d415bc50 2024-09-10T10:26:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.903+0000 7f45cb7fe640 1 -- 172.21.15.120:0/2546556640 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc031600 con 0x7f45d415bc50 2024-09-10T10:26:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.903+0000 7f45cb7fe640 1 -- 172.21.15.120:0/2546556640 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0319d0 con 0x7f45d415bc50 2024-09-10T10:26:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.904+0000 7f45d9f9f640 1 -- 172.21.15.120:0/2546556640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d415bc50 msgr2=0x7f45d415c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.904+0000 7f45d9f9f640 1 --2- 172.21.15.120:0/2546556640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d415bc50 0x7f45d415c0e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009e80 tx=0x7f45cc02a780 comp rx=0 tx=0).stop 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.904+0000 7f45d9f9f640 1 -- 172.21.15.120:0/2546556640 shutdown_connections 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.904+0000 7f45d9f9f640 1 --2- 172.21.15.120:0/2546556640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d415bc50 0x7f45d415c0e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.904+0000 7f45d9f9f640 1 --2- 172.21.15.120:0/2546556640 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d4154ac0 0x7f45d415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.904+0000 7f45d9f9f640 1 -- 172.21.15.120:0/2546556640 >> 172.21.15.120:0/2546556640 conn(0x7f45d406e400 msgr2=0x7f45d406e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.904+0000 7f45d9f9f640 1 -- 172.21.15.120:0/2546556640 shutdown_connections 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.904+0000 7f45d9f9f640 1 -- 172.21.15.120:0/2546556640 wait complete. 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.905+0000 7f45d9f9f640 1 Processor -- start 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.905+0000 7f45d9f9f640 1 -- start start 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45d9f9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d406d820 0x7f45d414efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45d9f9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d4154ac0 0x7f45d41f0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45d9f9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d415bc50 0x7f45d41f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45d9f9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45d40c5be0 con 0x7f45d4154ac0 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45d9f9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45d40c5900 con 0x7f45d415bc50 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45d9f9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f45d40c5ec0 con 0x7f45d406d820 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d4154ac0 0x7f45d41f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d4154ac0 0x7f45d41f0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47378/0 (socket says 172.21.15.120:47378) 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45cbfff640 1 -- 172.21.15.120:0/4078174567 learned_addr learned my addr 172.21.15.120:0/4078174567 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45d979e640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d415bc50 0x7f45d41f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45d8f9d640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d406d820 0x7f45d414efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45cbfff640 1 -- 172.21.15.120:0/4078174567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d406d820 msgr2=0x7f45d414efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45cbfff640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d406d820 0x7f45d414efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45cbfff640 1 -- 172.21.15.120:0/4078174567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d415bc50 msgr2=0x7f45d41f0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45cbfff640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d415bc50 0x7f45d41f0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:40.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45cbfff640 1 -- 172.21.15.120:0/4078174567 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d40ccff0 con 0x7f45d4154ac0 2024-09-10T10:26:40.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45d8f9d640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d406d820 0x7f45d414efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:26:40.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.906+0000 7f45cbfff640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d4154ac0 0x7f45d41f0620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f45c400c430 tx=0x7f45c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:40.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.907+0000 7f45c97fa640 1 -- 172.21.15.120:0/4078174567 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4011790 con 0x7f45d4154ac0 2024-09-10T10:26:40.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.907+0000 7f45c97fa640 1 -- 172.21.15.120:0/4078174567 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c4017070 con 0x7f45d4154ac0 2024-09-10T10:26:40.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.907+0000 7f45d9f9f640 1 -- 172.21.15.120:0/4078174567 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d40ca6e0 con 0x7f45d4154ac0 2024-09-10T10:26:40.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.907+0000 7f45c97fa640 1 -- 172.21.15.120:0/4078174567 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4011b70 con 0x7f45d4154ac0 2024-09-10T10:26:40.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.907+0000 7f45d9f9f640 1 -- 172.21.15.120:0/4078174567 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d40cb4e0 con 0x7f45d4154ac0 2024-09-10T10:26:40.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.908+0000 7f45d9f9f640 1 -- 172.21.15.120:0/4078174567 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d40c61a0 con 0x7f45d4154ac0 2024-09-10T10:26:40.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.908+0000 7f45c97fa640 1 -- 172.21.15.120:0/4078174567 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45c4011d10 con 0x7f45d4154ac0 2024-09-10T10:26:40.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.908+0000 7f45c97fa640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f45c006be60 0x7f45c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:40.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.909+0000 7f45c97fa640 1 -- 172.21.15.120:0/4078174567 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f45c408ab30 con 0x7f45d4154ac0 2024-09-10T10:26:40.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.909+0000 7f45d8f9d640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f45c006be60 0x7f45c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:40.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.909+0000 7f45d8f9d640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f45c006be60 0x7f45c006e320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f45d000a9d0 tx=0x7f45d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:40.913+0000 7f45c97fa640 1 -- 172.21.15.120:0/4078174567 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f45c40595d0 con 0x7f45d4154ac0 2024-09-10T10:26:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.112+0000 7f45d9f9f640 1 -- 172.21.15.120:0/4078174567 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f45d4154190 con 0x7f45d4154ac0 2024-09-10T10:26:41.178 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:26:41.181 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:26:41.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.772+0000 7f45c97fa640 1 -- 172.21.15.120:0/4078174567 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f45c4058ed0 con 0x7f45d4154ac0 2024-09-10T10:26:41.774 INFO:teuthology.orchestra.run.smithi120.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.775+0000 7f45d9f9f640 1 -- 172.21.15.120:0/4078174567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f45c006be60 msgr2=0x7f45c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.775+0000 7f45d9f9f640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f45c006be60 0x7f45c006e320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f45d000a9d0 tx=0x7f45d0008040 comp rx=0 tx=0).stop 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.775+0000 7f45d9f9f640 1 -- 172.21.15.120:0/4078174567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d4154ac0 msgr2=0x7f45d41f0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.775+0000 7f45d9f9f640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d4154ac0 0x7f45d41f0620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f45c400c430 tx=0x7f45c400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.775+0000 7f45d9f9f640 1 -- 172.21.15.120:0/4078174567 shutdown_connections 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.775+0000 7f45d9f9f640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f45c006be60 0x7f45c006e320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.775+0000 7f45d9f9f640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d415bc50 0x7f45d41f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.775+0000 7f45d9f9f640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d4154ac0 0x7f45d41f0620 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.775+0000 7f45d9f9f640 1 --2- 172.21.15.120:0/4078174567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45d406d820 0x7f45d414efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.776+0000 7f45d9f9f640 1 -- 172.21.15.120:0/4078174567 >> 172.21.15.120:0/4078174567 conn(0x7f45d406e400 msgr2=0x7f45d415abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:41.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.776+0000 7f45d9f9f640 1 -- 172.21.15.120:0/4078174567 shutdown_connections 2024-09-10T10:26:41.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.776+0000 7f45d9f9f640 1 -- 172.21.15.120:0/4078174567 wait complete. 2024-09-10T10:26:41.798 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T10:26:41.798 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T10:26:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.906+0000 7fa8c2053640 1 Processor -- start 2024-09-10T10:26:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.906+0000 7fa8c2053640 1 -- start start 2024-09-10T10:26:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.906+0000 7fa8c2053640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa8bc151a00 0x7fa8bc067950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.906+0000 7fa8c2053640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa8bc0709b0 0x7fa8bc06c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.906+0000 7fa8c2053640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8bc0c41d0 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.906+0000 7fa8c2053640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc0c3320 con 0x7fa8bc0709b0 2024-09-10T10:26:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.906+0000 7fa8c2053640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc0c3600 con 0x7fa8bc151a00 2024-09-10T10:26:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.907+0000 7fa8b3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa8bc0709b0 0x7fa8bc06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.907+0000 7fa8bbfff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fa8bc06ccd0 0x7fa8bc06d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57656/0 (socket says 172.21.15.120:57656) 2024-09-10T10:26:41.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.907+0000 7fa8b3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa8bc0709b0 0x7fa8bc06c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47394/0 (socket says 172.21.15.120:47394) 2024-09-10T10:26:41.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.907+0000 7fa8bbfff640 1 -- 172.21.15.120:0/1445800321 learned_addr learned my addr 172.21.15.120:0/1445800321 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:41.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.907+0000 7fa8bb7fe640 1 --2- 172.21.15.120:0/1445800321 >> v2:172.21.15.134:3300/0 conn(0x7fa8bc151a00 0x7fa8bc067950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:41.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.907+0000 7fa8b3fff640 1 -- 172.21.15.120:0/1445800321 >> v2:172.21.15.134:3300/0 conn(0x7fa8bc151a00 msgr2=0x7fa8bc067950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:41.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.907+0000 7fa8b3fff640 1 --2- 172.21.15.120:0/1445800321 >> v2:172.21.15.134:3300/0 conn(0x7fa8bc151a00 0x7fa8bc067950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:41.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.907+0000 7fa8b3fff640 1 -- 172.21.15.120:0/1445800321 >> v1:172.21.15.120:6789/0 conn(0x7fa8bc06ccd0 legacy=0x7fa8bc06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:41.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.907+0000 7fa8b3fff640 1 -- 172.21.15.120:0/1445800321 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc0c38e0 con 0x7fa8bc0709b0 2024-09-10T10:26:41.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.907+0000 7fa8baffd640 1 -- 172.21.15.120:0/1445800321 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2011107272 0 0) 0x7fa8bc0c41d0 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8b3fff640 1 --2- 172.21.15.120:0/1445800321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc0709b0 0x7fa8bc06c790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0743b0 tx=0x7fa8a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41d2f26ff935c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:41.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8baffd640 1 -- 172.21.15.120:0/1445800321 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4037070 con 0x7fa8bc0709b0 2024-09-10T10:26:41.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8baffd640 1 -- 172.21.15.120:0/1445800321 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a4032440 con 0x7fa8bc0709b0 2024-09-10T10:26:41.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8baffd640 1 -- 172.21.15.120:0/1445800321 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4032810 con 0x7fa8bc0709b0 2024-09-10T10:26:41.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8c2053640 1 -- 172.21.15.120:0/1445800321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc0709b0 msgr2=0x7fa8bc06c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8c2053640 1 --2- 172.21.15.120:0/1445800321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc0709b0 0x7fa8bc06c790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0743b0 tx=0x7fa8a402a230 comp rx=0 tx=0).stop 2024-09-10T10:26:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8c2053640 1 -- 172.21.15.120:0/1445800321 shutdown_connections 2024-09-10T10:26:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8c2053640 1 --2- 172.21.15.120:0/1445800321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc0709b0 0x7fa8bc06c790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8c2053640 1 --2- 172.21.15.120:0/1445800321 >> v2:172.21.15.134:3300/0 conn(0x7fa8bc151a00 0x7fa8bc067950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8c2053640 1 -- 172.21.15.120:0/1445800321 >> 172.21.15.120:0/1445800321 conn(0x7fa8bc06fc20 msgr2=0x7fa8bc070030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8c2053640 1 -- 172.21.15.120:0/1445800321 shutdown_connections 2024-09-10T10:26:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.908+0000 7fa8c2053640 1 -- 172.21.15.120:0/1445800321 wait complete. 2024-09-10T10:26:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8c2053640 1 Processor -- start 2024-09-10T10:26:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8c2053640 1 -- start start 2024-09-10T10:26:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8c2053640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc06ccd0 0x7fa8bc156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8c2053640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8bc0709b0 0x7fa8bc1ee340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8c2053640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8bc151a00 0x7fa8bc1ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8c2053640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc0c3e00 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8c2053640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc151de0 con 0x7fa8bc0709b0 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8c2053640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc0c3f80 con 0x7fa8bc151a00 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8bb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc06ccd0 0x7fa8bc156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8bb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc06ccd0 0x7fa8bc156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47402/0 (socket says 172.21.15.120:47402) 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.910+0000 7fa8bb7fe640 1 -- 172.21.15.120:0/2388155358 learned_addr learned my addr 172.21.15.120:0/2388155358 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8bb7fe640 1 -- 172.21.15.120:0/2388155358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8bc151a00 msgr2=0x7fa8bc1ee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8bb7fe640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8bc151a00 0x7fa8bc1ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8bb7fe640 1 -- 172.21.15.120:0/2388155358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8bc0709b0 msgr2=0x7fa8bc1ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8bb7fe640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8bc0709b0 0x7fa8bc1ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8bb7fe640 1 -- 172.21.15.120:0/2388155358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc1e2e30 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8bb7fe640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc06ccd0 0x7fa8bc156ee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00d5f0 tx=0x7fa8ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8b8ff9640 1 -- 172.21.15.120:0/2388155358 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0125f0 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8b8ff9640 1 -- 172.21.15.120:0/2388155358 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ac019070 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8c2053640 1 -- 172.21.15.120:0/2388155358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc1e3110 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8b8ff9640 1 -- 172.21.15.120:0/2388155358 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0129d0 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.911+0000 7fa8c2053640 1 -- 172.21.15.120:0/2388155358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc1e36d0 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.912+0000 7fa8c2053640 1 -- 172.21.15.120:0/2388155358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8bc1e38d0 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.913+0000 7fa8b8ff9640 1 -- 172.21.15.120:0/2388155358 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8ac012b70 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.913+0000 7fa8b8ff9640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa89c06be60 0x7fa89c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.913+0000 7fa8b3fff640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa89c06be60 0x7fa89c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.913+0000 7fa8b8ff9640 1 -- 172.21.15.120:0/2388155358 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa8ac08b1f0 con 0x7fa8bc06ccd0 2024-09-10T10:26:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.913+0000 7fa8b3fff640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa89c06be60 0x7fa89c06e320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4034000 tx=0x7fa8a402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:41.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:41.917+0000 7fa8b8ff9640 1 -- 172.21.15.120:0/2388155358 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa8ac059c30 con 0x7fa8bc06ccd0 2024-09-10T10:26:42.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.089+0000 7fa8c2053640 1 -- 172.21.15.120:0/2388155358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fa8bc072570 con 0x7fa8bc06ccd0 2024-09-10T10:26:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.774+0000 7fa8b8ff9640 1 -- 172.21.15.120:0/2388155358 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7fa8ac059530 con 0x7fa8bc06ccd0 2024-09-10T10:26:42.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 -- 172.21.15.120:0/2388155358 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa89c06be60 msgr2=0x7fa89c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:42.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa89c06be60 0x7fa89c06e320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4034000 tx=0x7fa8a402c000 comp rx=0 tx=0).stop 2024-09-10T10:26:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 -- 172.21.15.120:0/2388155358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc06ccd0 msgr2=0x7fa8bc156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc06ccd0 0x7fa8bc156ee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00d5f0 tx=0x7fa8ac00dab0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 -- 172.21.15.120:0/2388155358 shutdown_connections 2024-09-10T10:26:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa89c06be60 0x7fa89c06e320 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8bc151a00 0x7fa8bc1ee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8bc0709b0 0x7fa8bc1ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 --2- 172.21.15.120:0/2388155358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8bc06ccd0 0x7fa8bc156ee0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 -- 172.21.15.120:0/2388155358 >> 172.21.15.120:0/2388155358 conn(0x7fa8bc06fc20 msgr2=0x7fa8bc1543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 -- 172.21.15.120:0/2388155358 shutdown_connections 2024-09-10T10:26:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.777+0000 7fa8c2053640 1 -- 172.21.15.120:0/2388155358 wait complete. 2024-09-10T10:26:42.790 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:26:42.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.898+0000 7fe52e11a640 1 Processor -- start 2024-09-10T10:26:42.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.898+0000 7fe52e11a640 1 -- start start 2024-09-10T10:26:42.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe52e11a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe5280d0a00 0x7fe5280b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:42.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe52e11a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5280d0de0 0x7fe52815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:42.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe52e11a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe5280b4070 0x7fe52815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:42.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe52e11a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe5280b3640 con 0x7fe5280d0a00 2024-09-10T10:26:42.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe52e11a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe5280b4520 con 0x7fe5280b4070 2024-09-10T10:26:42.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe52e11a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe52815b5b0 con 0x7fe5280d0de0 2024-09-10T10:26:42.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe5277fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe5280d0a00 0x7fe5280b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:42.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe5277fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe5280d0a00 0x7fe5280b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47410/0 (socket says 172.21.15.120:47410) 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe5277fe640 1 -- 172.21.15.120:0/2191398451 learned_addr learned my addr 172.21.15.120:0/2191398451 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe527fff640 1 --2- 172.21.15.120:0/2191398451 >> v2:172.21.15.134:3300/0 conn(0x7fe5280b4070 0x7fe52815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe526ffd640 1 --2- 172.21.15.120:0/2191398451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5280d0de0 0x7fe52815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe5277fe640 1 -- 172.21.15.120:0/2191398451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5280d0de0 msgr2=0x7fe52815ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe5277fe640 1 --2- 172.21.15.120:0/2191398451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5280d0de0 0x7fe52815ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe5277fe640 1 -- 172.21.15.120:0/2191398451 >> v2:172.21.15.134:3300/0 conn(0x7fe5280b4070 msgr2=0x7fe52815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe5277fe640 1 --2- 172.21.15.120:0/2191398451 >> v2:172.21.15.134:3300/0 conn(0x7fe5280b4070 0x7fe52815ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe5277fe640 1 -- 172.21.15.120:0/2191398451 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52815b890 con 0x7fe5280d0a00 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.899+0000 7fe527fff640 1 --2- 172.21.15.120:0/2191398451 >> v2:172.21.15.134:3300/0 conn(0x7fe5280b4070 0x7fe52815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.900+0000 7fe5277fe640 1 --2- 172.21.15.120:0/2191398451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 0x7fe5280b3ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe52806c560 tx=0x7fe51802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d01fa8045c7590 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.900+0000 7fe5267fc640 1 -- 172.21.15.120:0/2191398451 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518037070 con 0x7fe5280d0a00 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.900+0000 7fe5267fc640 1 -- 172.21.15.120:0/2191398451 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe518032440 con 0x7fe5280d0a00 2024-09-10T10:26:42.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.900+0000 7fe5267fc640 1 -- 172.21.15.120:0/2191398451 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518032810 con 0x7fe5280d0a00 2024-09-10T10:26:42.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.901+0000 7fe52e11a640 1 -- 172.21.15.120:0/2191398451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 msgr2=0x7fe5280b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:42.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.901+0000 7fe52e11a640 1 --2- 172.21.15.120:0/2191398451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 0x7fe5280b3ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe52806c560 tx=0x7fe51802a270 comp rx=0 tx=0).stop 2024-09-10T10:26:42.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.901+0000 7fe52e11a640 1 -- 172.21.15.120:0/2191398451 shutdown_connections 2024-09-10T10:26:42.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.901+0000 7fe52e11a640 1 --2- 172.21.15.120:0/2191398451 >> v2:172.21.15.134:3300/0 conn(0x7fe5280b4070 0x7fe52815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.901+0000 7fe52e11a640 1 --2- 172.21.15.120:0/2191398451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5280d0de0 0x7fe52815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.901+0000 7fe52e11a640 1 --2- 172.21.15.120:0/2191398451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 0x7fe5280b3ad0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.901+0000 7fe52e11a640 1 -- 172.21.15.120:0/2191398451 >> 172.21.15.120:0/2191398451 conn(0x7fe52806c920 msgr2=0x7fe52806cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:42.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.901+0000 7fe52e11a640 1 -- 172.21.15.120:0/2191398451 shutdown_connections 2024-09-10T10:26:42.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.901+0000 7fe52e11a640 1 -- 172.21.15.120:0/2191398451 wait complete. 2024-09-10T10:26:42.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.902+0000 7fe52e11a640 1 Processor -- start 2024-09-10T10:26:42.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.902+0000 7fe52e11a640 1 -- start start 2024-09-10T10:26:42.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe52e11a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5280b4070 0x7fe5281f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:42.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe52e11a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 0x7fe5281f0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:42.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe52e11a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5280d0de0 0x7fe5281e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:42.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe52e11a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe52815b080 con 0x7fe5280d0a00 2024-09-10T10:26:42.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe52e11a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5280b32d0 con 0x7fe5280b4070 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe52e11a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe52815b200 con 0x7fe5280d0de0 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe526ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 0x7fe5281f0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe526ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 0x7fe5281f0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47426/0 (socket says 172.21.15.120:47426) 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe526ffd640 1 -- 172.21.15.120:0/2821564145 learned_addr learned my addr 172.21.15.120:0/2821564145 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe5277fe640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5280b4070 0x7fe5281f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe527fff640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5280d0de0 0x7fe5281e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe526ffd640 1 -- 172.21.15.120:0/2821564145 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5280d0de0 msgr2=0x7fe5281e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe526ffd640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5280d0de0 0x7fe5281e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe526ffd640 1 -- 172.21.15.120:0/2821564145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5280b4070 msgr2=0x7fe5281f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe526ffd640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5280b4070 0x7fe5281f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe526ffd640 1 -- 172.21.15.120:0/2821564145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5281e6470 con 0x7fe5280d0a00 2024-09-10T10:26:42.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe526ffd640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 0x7fe5281f0540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00c7f0 tx=0x7fe51c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe50ffff640 1 -- 172.21.15.120:0/2821564145 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c004230 con 0x7fe5280d0a00 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe50ffff640 1 -- 172.21.15.120:0/2821564145 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51c0043d0 con 0x7fe5280d0a00 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe52e11a640 1 -- 172.21.15.120:0/2821564145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5281f3b50 con 0x7fe5280d0a00 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe50ffff640 1 -- 172.21.15.120:0/2821564145 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c007840 con 0x7fe5280d0a00 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.903+0000 7fe52e11a640 1 -- 172.21.15.120:0/2821564145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5281e6a60 con 0x7fe5280d0a00 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.904+0000 7fe52e11a640 1 -- 172.21.15.120:0/2821564145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5280b3ab0 con 0x7fe5280d0a00 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.905+0000 7fe50ffff640 1 -- 172.21.15.120:0/2821564145 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe51c007bf0 con 0x7fe5280d0a00 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.905+0000 7fe50ffff640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe51406be30 0x7fe51406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.905+0000 7fe50ffff640 1 -- 172.21.15.120:0/2821564145 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7fe51c08b0d0 con 0x7fe5280d0a00 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.905+0000 7fe5277fe640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe51406be30 0x7fe51406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:42.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.906+0000 7fe5277fe640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe51406be30 0x7fe51406e2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe518034000 tx=0x7fe51802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:42.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:42.911+0000 7fe50ffff640 1 -- 172.21.15.120:0/2821564145 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe51c015020 con 0x7fe5280d0a00 2024-09-10T10:26:43.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:43.076+0000 7fe52e11a640 1 -- 172.21.15.120:0/2821564145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe52806cf50 con 0x7fe5280d0a00 2024-09-10T10:26:44.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.795+0000 7fe50ffff640 1 -- 172.21.15.120:0/2821564145 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7fe51c059300 con 0x7fe5280d0a00 2024-09-10T10:26:44.797 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_data_ec' created 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 -- 172.21.15.120:0/2821564145 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe51406be30 msgr2=0x7fe51406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe51406be30 0x7fe51406e2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe518034000 tx=0x7fe51802e000 comp rx=0 tx=0).stop 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 -- 172.21.15.120:0/2821564145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 msgr2=0x7fe5281f0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 0x7fe5281f0540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00c7f0 tx=0x7fe51c00ccb0 comp rx=0 tx=0).stop 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 -- 172.21.15.120:0/2821564145 shutdown_connections 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe51406be30 0x7fe51406e2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5280d0de0 0x7fe5281e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5280d0a00 0x7fe5281f0540 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 --2- 172.21.15.120:0/2821564145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5280b4070 0x7fe5281f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 -- 172.21.15.120:0/2821564145 >> 172.21.15.120:0/2821564145 conn(0x7fe52806c920 msgr2=0x7fe5280ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:44.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.798+0000 7fe52e11a640 1 -- 172.21.15.120:0/2821564145 shutdown_connections 2024-09-10T10:26:44.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.799+0000 7fe52e11a640 1 -- 172.21.15.120:0/2821564145 wait complete. 2024-09-10T10:26:44.819 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T10:26:44.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.924+0000 7f0579b28640 1 Processor -- start 2024-09-10T10:26:44.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.924+0000 7f0579b28640 1 -- start start 2024-09-10T10:26:44.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.925+0000 7f0579b28640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0574154df0 0x7f05740c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:44.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.925+0000 7f0579b28640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0574070390 con 0x7f057415df90 2024-09-10T10:26:44.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.925+0000 7f0579b28640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05740b0f60 con 0x7f057406d2e0 2024-09-10T10:26:44.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.925+0000 7f0579b28640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f057406ce50 con 0x7f0574154df0 2024-09-10T10:26:44.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.925+0000 7f0573fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f057415df90 0x7f057415e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57662/0 (socket says 172.21.15.120:57662) 2024-09-10T10:26:44.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.925+0000 7f0573fff640 1 -- 172.21.15.120:0/358394500 learned_addr learned my addr 172.21.15.120:0/358394500 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:44.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1027490907 0 0) 0x7f0574070390 con 0x7f057415df90 2024-09-10T10:26:44.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05737fe640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0574154df0 0x7f05740c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f055c003520 con 0x7f057415df90 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3674190142 0 0) 0x7f055c003520 con 0x7f057415df90 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0574154df0 msgr2=0x7f05740c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0574154df0 0x7f05740c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 >> v1:172.21.15.134:6789/0 conn(0x7f057406d2e0 legacy=0x7f057406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0574070630 con 0x7f057415df90 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f056c0019f0 con 0x7f057415df90 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 >> v1:172.21.15.120:6789/0 conn(0x7f057415df90 legacy=0x7f057415e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055c0039e0 0x7f055c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f055c004800 0x7f055c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0574154df0 0x7f055c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f057406ce50 con 0x7f055c0039e0 2024-09-10T10:26:44.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f055c00ba70 con 0x7f055c004800 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f055c00bc70 con 0x7f0574154df0 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05737fe640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055c0039e0 0x7f055c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f0573fff640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0574154df0 0x7f055c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f0572ffd640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f055c004800 0x7f055c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05737fe640 1 -- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0574154df0 msgr2=0x7f055c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05737fe640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0574154df0 0x7f055c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05737fe640 1 -- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f055c004800 msgr2=0x7f055c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05737fe640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f055c004800 0x7f055c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.926+0000 7f05737fe640 1 -- 172.21.15.120:0/358394500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c003520 con 0x7f055c0039e0 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.927+0000 7f05737fe640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055c0039e0 0x7f055c003e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0564009a90 tx=0x7f055c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d259908422daba69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.927+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564036070 con 0x7f055c0039e0 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.927+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0564031400 con 0x7f055c0039e0 2024-09-10T10:26:44.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.927+0000 7f05727fc640 1 -- 172.21.15.120:0/358394500 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564009690 con 0x7f055c0039e0 2024-09-10T10:26:44.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.928+0000 7f0579b28640 1 -- 172.21.15.120:0/358394500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055c0039e0 msgr2=0x7f055c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:44.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.928+0000 7f0579b28640 1 --2- 172.21.15.120:0/358394500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055c0039e0 0x7f055c003e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0564009a90 tx=0x7f055c005830 comp rx=0 tx=0).stop 2024-09-10T10:26:44.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.928+0000 7f0573fff640 1 -- 172.21.15.120:0/358394500 reap_dead start 2024-09-10T10:26:44.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.928+0000 7f0579b28640 1 -- 172.21.15.120:0/358394500 shutdown_connections 2024-09-10T10:26:44.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.928+0000 7f0579b28640 1 -- 172.21.15.120:0/358394500 >> 172.21.15.120:0/358394500 conn(0x7f0574071720 msgr2=0x7f0574071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:44.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.928+0000 7f0579b28640 1 -- 172.21.15.120:0/358394500 shutdown_connections 2024-09-10T10:26:44.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.928+0000 7f0579b28640 1 -- 172.21.15.120:0/358394500 wait complete. 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.929+0000 7f0579b28640 1 Processor -- start 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.929+0000 7f0579b28640 1 -- start start 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0579b28640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f055c0039e0 0x7f05740b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0579b28640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05740c7a10 0x7f05740af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0579b28640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05741f3fb0 0x7f05741f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0579b28640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0574070910 con 0x7f05740c7a10 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0579b28640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05741551d0 con 0x7f05741f3fb0 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0579b28640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f05740c90c0 con 0x7f055c0039e0 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0572ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05740c7a10 0x7f05740af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0572ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05740c7a10 0x7f05740af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47448/0 (socket says 172.21.15.120:47448) 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0572ffd640 1 -- 172.21.15.120:0/764913450 learned_addr learned my addr 172.21.15.120:0/764913450 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:44.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0573fff640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05741f3fb0 0x7f05741f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0572ffd640 1 -- 172.21.15.120:0/764913450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f055c0039e0 msgr2=0x7f05740b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f05737fe640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f055c0039e0 0x7f05740b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0572ffd640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f055c0039e0 0x7f05740b4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0572ffd640 1 -- 172.21.15.120:0/764913450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05741f3fb0 msgr2=0x7f05741f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0572ffd640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05741f3fb0 0x7f05741f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0572ffd640 1 -- 172.21.15.120:0/764913450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05741f0740 con 0x7f05740c7a10 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0573fff640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05741f3fb0 0x7f05741f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0572ffd640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05740c7a10 0x7f05740af130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f056800f7f0 tx=0x7f056800fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f055bfff640 1 -- 172.21.15.120:0/764913450 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05680050d0 con 0x7f05740c7a10 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f055bfff640 1 -- 172.21.15.120:0/764913450 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0568018070 con 0x7f05740c7a10 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f0579b28640 1 -- 172.21.15.120:0/764913450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05741f0a20 con 0x7f05740c7a10 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.930+0000 7f055bfff640 1 -- 172.21.15.120:0/764913450 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568005490 con 0x7f05740c7a10 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.931+0000 7f0579b28640 1 -- 172.21.15.120:0/764913450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05741f0fe0 con 0x7f05740c7a10 2024-09-10T10:26:44.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.932+0000 7f0579b28640 1 -- 172.21.15.120:0/764913450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05740ae9a0 con 0x7f05740c7a10 2024-09-10T10:26:44.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.932+0000 7f055bfff640 1 -- 172.21.15.120:0/764913450 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f056800d730 con 0x7f05740c7a10 2024-09-10T10:26:44.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.932+0000 7f055bfff640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f056006be60 0x7f056006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:44.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.933+0000 7f05737fe640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f056006be60 0x7f056006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:44.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.933+0000 7f055bfff640 1 -- 172.21.15.120:0/764913450 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f056808b100 con 0x7f05740c7a10 2024-09-10T10:26:44.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.933+0000 7f05737fe640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f056006be60 0x7f056006e320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0564033000 tx=0x7f0564008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:44.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:44.937+0000 7f055bfff640 1 -- 172.21.15.120:0/764913450 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0568059700 con 0x7f05740c7a10 2024-09-10T10:26:45.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.101+0000 7f0579b28640 1 -- 172.21.15.120:0/764913450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f05740cabe0 con 0x7f05740c7a10 2024-09-10T10:26:45.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.796+0000 7f055bfff640 1 -- 172.21.15.120:0/764913450 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f0568059000 con 0x7f05740c7a10 2024-09-10T10:26:45.798 INFO:teuthology.orchestra.run.smithi120.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T10:26:45.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.800+0000 7f0579b28640 1 -- 172.21.15.120:0/764913450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f056006be60 msgr2=0x7f056006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:45.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.800+0000 7f0579b28640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f056006be60 0x7f056006e320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0564033000 tx=0x7f0564008030 comp rx=0 tx=0).stop 2024-09-10T10:26:45.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.800+0000 7f0579b28640 1 -- 172.21.15.120:0/764913450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05740c7a10 msgr2=0x7f05740af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:45.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.800+0000 7f0579b28640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05740c7a10 0x7f05740af130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f056800f7f0 tx=0x7f056800fcb0 comp rx=0 tx=0).stop 2024-09-10T10:26:45.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.800+0000 7f0579b28640 1 -- 172.21.15.120:0/764913450 shutdown_connections 2024-09-10T10:26:45.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.800+0000 7f0579b28640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f056006be60 0x7f056006e320 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:45.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.800+0000 7f0579b28640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05741f3fb0 0x7f05741f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:45.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.800+0000 7f0579b28640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05740c7a10 0x7f05740af130 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:45.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.800+0000 7f0579b28640 1 --2- 172.21.15.120:0/764913450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f055c0039e0 0x7f05740b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:45.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.800+0000 7f0579b28640 1 -- 172.21.15.120:0/764913450 >> 172.21.15.120:0/764913450 conn(0x7f0574071720 msgr2=0x7f05740b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:45.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.809+0000 7f0579b28640 1 -- 172.21.15.120:0/764913450 shutdown_connections 2024-09-10T10:26:45.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.810+0000 7f0579b28640 1 -- 172.21.15.120:0/764913450 wait complete. 2024-09-10T10:26:45.834 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T10:26:45.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.934+0000 7efed034b640 1 Processor -- start 2024-09-10T10:26:45.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.934+0000 7efed034b640 1 -- start start 2024-09-10T10:26:45.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efed034b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efec80c8300 0x7efec80cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:45.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efed034b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efec80c35a0 0x7efec8163040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:45.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efed034b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec80c5310 con 0x7efec80c31c0 2024-09-10T10:26:45.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efed034b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7efec80cac30 con 0x7efec80c8300 2024-09-10T10:26:45.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efed034b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efec80ccd70 con 0x7efec80c35a0 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efecd8e6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efec80c8300 0x7efec80cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efecd8e6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efec80c8300 0x7efec80cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47458/0 (socket says 172.21.15.120:47458) 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efece0e7640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7efec80c31c0 0x7efec81586c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57676/0 (socket says 172.21.15.120:57676) 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efecd8e6640 1 -- 172.21.15.120:0/3147387586 learned_addr learned my addr 172.21.15.120:0/3147387586 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efecd8e6640 1 -- 172.21.15.120:0/3147387586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efec80c35a0 msgr2=0x7efec8163040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efece8e8640 1 --2- 172.21.15.120:0/3147387586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efec80c35a0 0x7efec8163040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efecd8e6640 1 --2- 172.21.15.120:0/3147387586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efec80c35a0 0x7efec8163040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efecd0e5640 1 -- 172.21.15.120:0/3147387586 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3890944679 0 0) 0x7efec80c5310 con 0x7efec80c31c0 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efecd8e6640 1 -- 172.21.15.120:0/3147387586 >> v1:172.21.15.120:6789/0 conn(0x7efec80c31c0 legacy=0x7efec81586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.935+0000 7efecd8e6640 1 -- 172.21.15.120:0/3147387586 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec80cd050 con 0x7efec80c8300 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.936+0000 7efecd8e6640 1 --2- 172.21.15.120:0/3147387586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c8300 0x7efec80cbfc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009870 tx=0x7efeb802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd98479ebaade0a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.936+0000 7efecd0e5640 1 -- 172.21.15.120:0/3147387586 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8037070 con 0x7efec80c8300 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.936+0000 7efecd0e5640 1 -- 172.21.15.120:0/3147387586 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb802ae50 con 0x7efec80c8300 2024-09-10T10:26:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.936+0000 7efecd0e5640 1 -- 172.21.15.120:0/3147387586 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8032860 con 0x7efec80c8300 2024-09-10T10:26:45.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.937+0000 7efed034b640 1 -- 172.21.15.120:0/3147387586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c8300 msgr2=0x7efec80cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:45.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.937+0000 7efed034b640 1 --2- 172.21.15.120:0/3147387586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c8300 0x7efec80cbfc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009870 tx=0x7efeb802a300 comp rx=0 tx=0).stop 2024-09-10T10:26:45.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.937+0000 7efed034b640 1 -- 172.21.15.120:0/3147387586 shutdown_connections 2024-09-10T10:26:45.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.937+0000 7efed034b640 1 --2- 172.21.15.120:0/3147387586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efec80c35a0 0x7efec8163040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.937+0000 7efed034b640 1 --2- 172.21.15.120:0/3147387586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c8300 0x7efec80cbfc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.937+0000 7efed034b640 1 -- 172.21.15.120:0/3147387586 >> 172.21.15.120:0/3147387586 conn(0x7efec8067950 msgr2=0x7efec8067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.937+0000 7efed034b640 1 -- 172.21.15.120:0/3147387586 shutdown_connections 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.937+0000 7efed034b640 1 -- 172.21.15.120:0/3147387586 wait complete. 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.938+0000 7efed034b640 1 Processor -- start 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.938+0000 7efed034b640 1 -- start start 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.938+0000 7efed034b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c31c0 0x7efec81f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.938+0000 7efed034b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efec80c35a0 0x7efec81f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.938+0000 7efed034b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efec80c8300 0x7efec81f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efed034b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efec80cb160 con 0x7efec80c31c0 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efed034b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efec80cae80 con 0x7efec80c8300 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efed034b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efec80cb680 con 0x7efec80c35a0 2024-09-10T10:26:45.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efece0e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c31c0 0x7efec81f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efece0e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c31c0 0x7efec81f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47464/0 (socket says 172.21.15.120:47464) 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efece0e7640 1 -- 172.21.15.120:0/3560738381 learned_addr learned my addr 172.21.15.120:0/3560738381 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efecd8e6640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efec80c35a0 0x7efec81f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efece8e8640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efec80c8300 0x7efec81f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efece0e7640 1 -- 172.21.15.120:0/3560738381 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efec80c35a0 msgr2=0x7efec81f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efece0e7640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efec80c35a0 0x7efec81f4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efece0e7640 1 -- 172.21.15.120:0/3560738381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efec80c8300 msgr2=0x7efec81f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efece0e7640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efec80c8300 0x7efec81f0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efece0e7640 1 -- 172.21.15.120:0/3560738381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec814b2d0 con 0x7efec80c31c0 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efece0e7640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c31c0 0x7efec81f4530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efec400c430 tx=0x7efec400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efebeffd640 1 -- 172.21.15.120:0/3560738381 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec40074e0 con 0x7efec80c31c0 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efebeffd640 1 -- 172.21.15.120:0/3560738381 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec40047e0 con 0x7efec80c31c0 2024-09-10T10:26:45.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efed034b640 1 -- 172.21.15.120:0/3560738381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec8149390 con 0x7efec80c31c0 2024-09-10T10:26:45.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efebeffd640 1 -- 172.21.15.120:0/3560738381 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4004bd0 con 0x7efec80c31c0 2024-09-10T10:26:45.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.939+0000 7efed034b640 1 -- 172.21.15.120:0/3560738381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec8149630 con 0x7efec80c31c0 2024-09-10T10:26:45.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.940+0000 7efed034b640 1 -- 172.21.15.120:0/3560738381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec80c7460 con 0x7efec80c31c0 2024-09-10T10:26:45.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.941+0000 7efebeffd640 1 -- 172.21.15.120:0/3560738381 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efec4011070 con 0x7efec80c31c0 2024-09-10T10:26:45.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.941+0000 7efebeffd640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efeac06be30 0x7efeac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:45.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.941+0000 7efecd8e6640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efeac06be30 0x7efeac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:45.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.941+0000 7efebeffd640 1 -- 172.21.15.120:0/3560738381 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7efec408ae20 con 0x7efec80c31c0 2024-09-10T10:26:45.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.942+0000 7efecd8e6640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efeac06be30 0x7efeac06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efeb8034000 tx=0x7efeb802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:45.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:45.946+0000 7efebeffd640 1 -- 172.21.15.120:0/3560738381 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efec80c7460 con 0x7efec80c31c0 2024-09-10T10:26:46.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.146+0000 7efed034b640 1 -- 172.21.15.120:0/3560738381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7efec80c39d0 con 0x7efec80c31c0 2024-09-10T10:26:46.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.826+0000 7efebeffd640 1 -- 172.21.15.120:0/3560738381 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v5) ==== 137+0+0 (secure 0 0 0) 0x7efec40594b0 con 0x7efec80c31c0 2024-09-10T10:26:46.829 INFO:teuthology.orchestra.run.smithi120.stderr:added data pool 4 to fsmap 2024-09-10T10:26:46.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 -- 172.21.15.120:0/3560738381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efeac06be30 msgr2=0x7efeac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:46.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efeac06be30 0x7efeac06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efeb8034000 tx=0x7efeb802e000 comp rx=0 tx=0).stop 2024-09-10T10:26:46.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 -- 172.21.15.120:0/3560738381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c31c0 msgr2=0x7efec81f4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:46.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c31c0 0x7efec81f4530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efec400c430 tx=0x7efec400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:26:46.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 -- 172.21.15.120:0/3560738381 shutdown_connections 2024-09-10T10:26:46.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efeac06be30 0x7efeac06e2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:46.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efec80c8300 0x7efec81f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:46.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efec80c35a0 0x7efec81f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:46.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 --2- 172.21.15.120:0/3560738381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efec80c31c0 0x7efec81f4530 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:46.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 -- 172.21.15.120:0/3560738381 >> 172.21.15.120:0/3560738381 conn(0x7efec8067950 msgr2=0x7efec8150eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:46.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 -- 172.21.15.120:0/3560738381 shutdown_connections 2024-09-10T10:26:46.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.829+0000 7efed034b640 1 -- 172.21.15.120:0/3560738381 wait complete. 2024-09-10T10:26:46.843 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:26:46.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.939+0000 7f7d912d0640 1 Processor -- start 2024-09-10T10:26:46.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.939+0000 7f7d912d0640 1 -- start start 2024-09-10T10:26:46.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.940+0000 7f7d912d0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7d8c0c8de0 0x7f7d8c070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:46.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.940+0000 7f7d912d0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8c15acd0 con 0x7f7d8c0c91c0 2024-09-10T10:26:46.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.940+0000 7f7d912d0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8c15afb0 con 0x7f7d8c15eab0 2024-09-10T10:26:46.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.940+0000 7f7d912d0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8c15ef80 con 0x7f7d8c0c8de0 2024-09-10T10:26:46.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.940+0000 7f7d8affd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7d8c0c8de0 0x7f7d8c070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:46.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.940+0000 7f7d8affd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7d8c0c8de0 0x7f7d8c070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47466/0 (socket says 172.21.15.120:47466) 2024-09-10T10:26:46.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.940+0000 7f7d8b7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f7d8c0c91c0 0x7f7d8c0b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57690/0 (socket says 172.21.15.120:57690) 2024-09-10T10:26:46.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.940+0000 7f7d8affd640 1 -- 172.21.15.120:0/2525349595 learned_addr learned my addr 172.21.15.120:0/2525349595 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:46.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.941+0000 7f7d8affd640 1 -- 172.21.15.120:0/2525349595 >> v1:172.21.15.134:6789/0 conn(0x7f7d8c15eab0 legacy=0x7f7d8c0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:46.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.941+0000 7f7d8affd640 1 -- 172.21.15.120:0/2525349595 >> v1:172.21.15.120:6789/0 conn(0x7f7d8c0c91c0 legacy=0x7f7d8c0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:46.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.941+0000 7f7d89ffb640 1 -- 172.21.15.120:0/2525349595 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1882042399 0 0) 0x7f7d8c15acd0 con 0x7f7d8c0c91c0 2024-09-10T10:26:46.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.941+0000 7f7d8affd640 1 -- 172.21.15.120:0/2525349595 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c15b290 con 0x7f7d8c0c8de0 2024-09-10T10:26:46.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.941+0000 7f7d8affd640 1 --2- 172.21.15.120:0/2525349595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c0c8de0 0x7f7d8c070be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009e50 tx=0x7f7d8402a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7a46c3b98f1d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:46.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.941+0000 7f7d89ffb640 1 -- 172.21.15.120:0/2525349595 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84036070 con 0x7f7d8c0c8de0 2024-09-10T10:26:46.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.941+0000 7f7d89ffb640 1 -- 172.21.15.120:0/2525349595 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d84031600 con 0x7f7d8c0c8de0 2024-09-10T10:26:46.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.941+0000 7f7d89ffb640 1 -- 172.21.15.120:0/2525349595 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840319d0 con 0x7f7d8c0c8de0 2024-09-10T10:26:46.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.942+0000 7f7d912d0640 1 -- 172.21.15.120:0/2525349595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c0c8de0 msgr2=0x7f7d8c070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:46.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.942+0000 7f7d912d0640 1 --2- 172.21.15.120:0/2525349595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c0c8de0 0x7f7d8c070be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009e50 tx=0x7f7d8402a7f0 comp rx=0 tx=0).stop 2024-09-10T10:26:46.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.942+0000 7f7d912d0640 1 -- 172.21.15.120:0/2525349595 shutdown_connections 2024-09-10T10:26:46.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.942+0000 7f7d912d0640 1 --2- 172.21.15.120:0/2525349595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c0c8de0 0x7f7d8c070be0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:46.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.942+0000 7f7d912d0640 1 -- 172.21.15.120:0/2525349595 >> 172.21.15.120:0/2525349595 conn(0x7f7d8c071550 msgr2=0x7f7d8c071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:46.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.942+0000 7f7d912d0640 1 -- 172.21.15.120:0/2525349595 shutdown_connections 2024-09-10T10:26:46.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.942+0000 7f7d912d0640 1 -- 172.21.15.120:0/2525349595 wait complete. 2024-09-10T10:26:46.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.943+0000 7f7d912d0640 1 Processor -- start 2024-09-10T10:26:46.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.943+0000 7f7d912d0640 1 -- start start 2024-09-10T10:26:46.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d912d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c0c8de0 0x7f7d8c0c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:46.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d912d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d8c0c91c0 0x7f7d8c0c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:46.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d912d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c15eab0 0x7f7d8c06d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:46.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d912d0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c0cad50 con 0x7f7d8c15eab0 2024-09-10T10:26:46.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d912d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c15b8e0 con 0x7f7d8c0c91c0 2024-09-10T10:26:46.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d912d0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c0caed0 con 0x7f7d8c0c8de0 2024-09-10T10:26:46.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c15eab0 0x7f7d8c06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:46.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c15eab0 0x7f7d8c06d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47474/0 (socket says 172.21.15.120:47474) 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8b7fe640 1 -- 172.21.15.120:0/253952205 learned_addr learned my addr 172.21.15.120:0/253952205 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8affd640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c0c8de0 0x7f7d8c0c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8a7fc640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d8c0c91c0 0x7f7d8c0c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8b7fe640 1 -- 172.21.15.120:0/253952205 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c0c8de0 msgr2=0x7f7d8c0c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8b7fe640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c0c8de0 0x7f7d8c0c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8b7fe640 1 -- 172.21.15.120:0/253952205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d8c0c91c0 msgr2=0x7f7d8c0c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8b7fe640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d8c0c91c0 0x7f7d8c0c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8b7fe640 1 -- 172.21.15.120:0/253952205 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c157f60 con 0x7f7d8c15eab0 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8affd640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c0c8de0 0x7f7d8c0c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.944+0000 7f7d8b7fe640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c15eab0 0x7f7d8c06d830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00c480 tx=0x7f7d7c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.945+0000 7f7d837fe640 1 -- 172.21.15.120:0/253952205 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c018070 con 0x7f7d8c15eab0 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.945+0000 7f7d837fe640 1 -- 172.21.15.120:0/253952205 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7c0074e0 con 0x7f7d8c15eab0 2024-09-10T10:26:46.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.945+0000 7f7d837fe640 1 -- 172.21.15.120:0/253952205 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c004a20 con 0x7f7d8c15eab0 2024-09-10T10:26:46.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.945+0000 7f7d912d0640 1 -- 172.21.15.120:0/253952205 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c1f47a0 con 0x7f7d8c15eab0 2024-09-10T10:26:46.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.945+0000 7f7d912d0640 1 -- 172.21.15.120:0/253952205 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c0c1720 con 0x7f7d8c15eab0 2024-09-10T10:26:46.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.945+0000 7f7d912d0640 1 -- 172.21.15.120:0/253952205 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8c06c6a0 con 0x7f7d8c15eab0 2024-09-10T10:26:46.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.946+0000 7f7d837fe640 1 -- 172.21.15.120:0/253952205 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d7c01b070 con 0x7f7d8c15eab0 2024-09-10T10:26:46.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.946+0000 7f7d837fe640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 0x7f7d6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:46.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.947+0000 7f7d837fe640 1 -- 172.21.15.120:0/253952205 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7d7c08a410 con 0x7f7d8c15eab0 2024-09-10T10:26:46.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.947+0000 7f7d8affd640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 0x7f7d6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:46.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.947+0000 7f7d8affd640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 0x7f7d6c06e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0c96b0 tx=0x7f7d84009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:46.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:46.951+0000 7f7d837fe640 1 -- 172.21.15.120:0/253952205 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7d7c058b10 con 0x7f7d8c15eab0 2024-09-10T10:26:47.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.153+0000 7f7d912d0640 1 -- 172.21.15.120:0/253952205 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d8c0c9890 con 0x7f7d8c15eab0 2024-09-10T10:26:47.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.153+0000 7f7d837fe640 1 -- 172.21.15.120:0/253952205 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+6876 (secure 0 0 0) 0x7f7d7c058410 con 0x7f7d8c15eab0 2024-09-10T10:26:47.156 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:26:47.156 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":6,"btime":"2024-09-10T10:26:46:830817+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:26:41.113510+0000","modified":"2024-09-10T10:26:46.822339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-10T10:26:47.156 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 6 2024-09-10T10:26:47.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.155+0000 7f7d912d0640 1 -- 172.21.15.120:0/253952205 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 msgr2=0x7f7d6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.155+0000 7f7d912d0640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 0x7f7d6c06e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0c96b0 tx=0x7f7d84009000 comp rx=0 tx=0).stop 2024-09-10T10:26:47.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.155+0000 7f7d912d0640 1 -- 172.21.15.120:0/253952205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c15eab0 msgr2=0x7f7d8c06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.155+0000 7f7d912d0640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c15eab0 0x7f7d8c06d830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00c480 tx=0x7f7d7c00c940 comp rx=0 tx=0).stop 2024-09-10T10:26:47.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.155+0000 7f7d912d0640 1 -- 172.21.15.120:0/253952205 shutdown_connections 2024-09-10T10:26:47.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.155+0000 7f7d912d0640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 0x7f7d6c06e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.155+0000 7f7d912d0640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c15eab0 0x7f7d8c06d830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.155+0000 7f7d912d0640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d8c0c91c0 0x7f7d8c0c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.155+0000 7f7d912d0640 1 --2- 172.21.15.120:0/253952205 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c0c8de0 0x7f7d8c0c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.155+0000 7f7d912d0640 1 -- 172.21.15.120:0/253952205 >> 172.21.15.120:0/253952205 conn(0x7f7d8c071550 msgr2=0x7f7d8c071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:47.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.156+0000 7f7d912d0640 1 -- 172.21.15.120:0/253952205 shutdown_connections 2024-09-10T10:26:47.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.156+0000 7f7d912d0640 1 -- 172.21.15.120:0/253952205 wait complete. 2024-09-10T10:26:47.169 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:26:47.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.264+0000 7f29dc931640 1 Processor -- start 2024-09-10T10:26:47.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.264+0000 7f29dc931640 1 -- start start 2024-09-10T10:26:47.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29dc931640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d806cf90 0x7f29d80c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29dc931640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f29d8151c00 0x7f29d8161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29dc931640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d80c4890 con 0x7f29d806d370 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29dc931640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f29d80c8fc0 con 0x7f29d8151c00 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29dc931640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f29d80c4630 con 0x7f29d806cf90 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29d6d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f29d8151c00 0x7f29d8161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29d6d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f29d8151c00 0x7f29d8161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47478/0 (socket says 172.21.15.120:47478) 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29d6d9d640 1 -- 172.21.15.120:0/1442099785 learned_addr learned my addr 172.21.15.120:0/1442099785 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29d659c640 1 --2- 172.21.15.120:0/1442099785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d806cf90 0x7f29d80c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29d6d9d640 1 -- 172.21.15.120:0/1442099785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d806cf90 msgr2=0x7f29d80c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29d6d9d640 1 --2- 172.21.15.120:0/1442099785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d806cf90 0x7f29d80c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29d6d9d640 1 -- 172.21.15.120:0/1442099785 >> v1:172.21.15.134:6789/0 conn(0x7f29d806d370 legacy=0x7f29d81514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.265+0000 7f29d6d9d640 1 -- 172.21.15.120:0/1442099785 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d8070d90 con 0x7f29d8151c00 2024-09-10T10:26:47.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29d6d9d640 1 --2- 172.21.15.120:0/1442099785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 0x7f29d8161f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29d80c9140 tx=0x7f29d002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec45f6bd197fce20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29d5d9b640 1 -- 172.21.15.120:0/1442099785 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0036070 con 0x7f29d8151c00 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29d5d9b640 1 -- 172.21.15.120:0/1442099785 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d0031600 con 0x7f29d8151c00 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29d5d9b640 1 -- 172.21.15.120:0/1442099785 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d00319d0 con 0x7f29d8151c00 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29dc931640 1 -- 172.21.15.120:0/1442099785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 msgr2=0x7f29d8161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29dc931640 1 --2- 172.21.15.120:0/1442099785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 0x7f29d8161f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29d80c9140 tx=0x7f29d002a6c0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29dc931640 1 -- 172.21.15.120:0/1442099785 shutdown_connections 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29dc931640 1 --2- 172.21.15.120:0/1442099785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 0x7f29d8161f70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29dc931640 1 --2- 172.21.15.120:0/1442099785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d806cf90 0x7f29d80c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29dc931640 1 -- 172.21.15.120:0/1442099785 >> 172.21.15.120:0/1442099785 conn(0x7f29d8067d40 msgr2=0x7f29d8070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29dc931640 1 -- 172.21.15.120:0/1442099785 shutdown_connections 2024-09-10T10:26:47.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.266+0000 7f29dc931640 1 -- 172.21.15.120:0/1442099785 wait complete. 2024-09-10T10:26:47.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.268+0000 7f29dc931640 1 Processor -- start 2024-09-10T10:26:47.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.268+0000 7f29dc931640 1 -- start start 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.268+0000 7f29dc931640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d806cf90 0x7f29d81e5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.268+0000 7f29dc931640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d806d370 0x7f29d80cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.268+0000 7f29dc931640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 0x7f29d814c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.268+0000 7f29dc931640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29d80c8660 con 0x7f29d8151c00 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29dc931640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29d80c8380 con 0x7f29d806cf90 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29dc931640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f29d80c8940 con 0x7f29d806d370 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29d6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 0x7f29d814c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29d6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 0x7f29d814c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47482/0 (socket says 172.21.15.120:47482) 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29d6d9d640 1 -- 172.21.15.120:0/2370475430 learned_addr learned my addr 172.21.15.120:0/2370475430 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29d659c640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d806cf90 0x7f29d81e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29cdd9b640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d806d370 0x7f29d80cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29d6d9d640 1 -- 172.21.15.120:0/2370475430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d806d370 msgr2=0x7f29d80cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29d6d9d640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d806d370 0x7f29d80cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29d6d9d640 1 -- 172.21.15.120:0/2370475430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d806cf90 msgr2=0x7f29d81e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29d6d9d640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d806cf90 0x7f29d81e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29d6d9d640 1 -- 172.21.15.120:0/2370475430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d8143110 con 0x7f29d8151c00 2024-09-10T10:26:47.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29d6d9d640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 0x7f29d814c000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29d002acb0 tx=0x7f29d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:47.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29cf7fe640 1 -- 172.21.15.120:0/2370475430 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0036050 con 0x7f29d8151c00 2024-09-10T10:26:47.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29cf7fe640 1 -- 172.21.15.120:0/2370475430 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d0042070 con 0x7f29d8151c00 2024-09-10T10:26:47.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29dc931640 1 -- 172.21.15.120:0/2370475430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d81425b0 con 0x7f29d8151c00 2024-09-10T10:26:47.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29cf7fe640 1 -- 172.21.15.120:0/2370475430 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0009510 con 0x7f29d8151c00 2024-09-10T10:26:47.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.269+0000 7f29dc931640 1 -- 172.21.15.120:0/2370475430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d8142b50 con 0x7f29d8151c00 2024-09-10T10:26:47.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.270+0000 7f29dc931640 1 -- 172.21.15.120:0/2370475430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d80cd080 con 0x7f29d8151c00 2024-09-10T10:26:47.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.271+0000 7f29cf7fe640 1 -- 172.21.15.120:0/2370475430 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29d0007bd0 con 0x7f29d8151c00 2024-09-10T10:26:47.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.271+0000 7f29cf7fe640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f29b806be60 0x7f29b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.271+0000 7f29cf7fe640 1 -- 172.21.15.120:0/2370475430 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f29d00abe70 con 0x7f29d8151c00 2024-09-10T10:26:47.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.271+0000 7f29d659c640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f29b806be60 0x7f29b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.272+0000 7f29d659c640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f29b806be60 0x7f29b806e320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f29c8006fd0 tx=0x7f29c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:47.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.276+0000 7f29cf7fe640 1 -- 172.21.15.120:0/2370475430 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f29d0079e90 con 0x7f29d8151c00 2024-09-10T10:26:47.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.442+0000 7f29dc931640 1 -- 172.21.15.120:0/2370475430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29d81e3620 con 0x7f29d8151c00 2024-09-10T10:26:47.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.443+0000 7f29cf7fe640 1 -- 172.21.15.120:0/2370475430 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18635 (secure 0 0 0) 0x7f29d007e190 con 0x7f29d8151c00 2024-09-10T10:26:47.445 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:26:47.445 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":25,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:46.148813+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:26:47.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 -- 172.21.15.120:0/2370475430 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f29b806be60 msgr2=0x7f29b806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f29b806be60 0x7f29b806e320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f29c8006fd0 tx=0x7f29c8008040 comp rx=0 tx=0).stop 2024-09-10T10:26:47.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 -- 172.21.15.120:0/2370475430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 msgr2=0x7f29d814c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 0x7f29d814c000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29d002acb0 tx=0x7f29d0004a40 comp rx=0 tx=0).stop 2024-09-10T10:26:47.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 -- 172.21.15.120:0/2370475430 shutdown_connections 2024-09-10T10:26:47.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f29b806be60 0x7f29b806e320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29d8151c00 0x7f29d814c000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29d806d370 0x7f29d80cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 --2- 172.21.15.120:0/2370475430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29d806cf90 0x7f29d81e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 -- 172.21.15.120:0/2370475430 >> 172.21.15.120:0/2370475430 conn(0x7f29d8067d40 msgr2=0x7f29d81512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:47.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 -- 172.21.15.120:0/2370475430 shutdown_connections 2024-09-10T10:26:47.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.445+0000 7f29dc931640 1 -- 172.21.15.120:0/2370475430 wait complete. 2024-09-10T10:26:47.458 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:26:47.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.553+0000 7f151a877640 1 Processor -- start 2024-09-10T10:26:47.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.553+0000 7f151a877640 1 -- start start 2024-09-10T10:26:47.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.554+0000 7f151a877640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1514152810 0x7f15140c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.554+0000 7f151a877640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f151415cdc0 con 0x7f151406d940 2024-09-10T10:26:47.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.554+0000 7f151a877640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f151406c320 con 0x7f151406ea60 2024-09-10T10:26:47.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.554+0000 7f151a877640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f151415be40 con 0x7f1514152810 2024-09-10T10:26:47.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.554+0000 7f15137fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f151406d940 0x7f151406bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:46234/0 (socket says 172.21.15.120:46234) 2024-09-10T10:26:47.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.554+0000 7f15137fe640 1 -- 172.21.15.120:0/2002316933 learned_addr learned my addr 172.21.15.120:0/2002316933 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:47.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.555+0000 7f1513fff640 1 --2- 172.21.15.120:0/2002316933 >> v2:172.21.15.134:3300/0 conn(0x7f1514152810 0x7f15140c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.555+0000 7f1512ffd640 1 -- 172.21.15.120:0/2002316933 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884561217 0 0) 0x7f151415cdc0 con 0x7f151406d940 2024-09-10T10:26:47.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.555+0000 7f1512ffd640 1 -- 172.21.15.120:0/2002316933 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14fc003520 con 0x7f151406d940 2024-09-10T10:26:47.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.555+0000 7f1513fff640 1 -- 172.21.15.120:0/2002316933 >> v1:172.21.15.134:6789/0 conn(0x7f151406ea60 legacy=0x7f15141631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.555+0000 7f1513fff640 1 -- 172.21.15.120:0/2002316933 >> v1:172.21.15.120:6789/0 conn(0x7f151406d940 legacy=0x7f151406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.555+0000 7f1513fff640 1 -- 172.21.15.120:0/2002316933 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15141593e0 con 0x7f1514152810 2024-09-10T10:26:47.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f1513fff640 1 --2- 172.21.15.120:0/2002316933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1514152810 0x7f15140c79c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f15040097a0 tx=0x7f150402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3e6fd1c344d8577 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:47.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f1512ffd640 1 -- 172.21.15.120:0/2002316933 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504037070 con 0x7f1514152810 2024-09-10T10:26:47.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f1512ffd640 1 -- 172.21.15.120:0/2002316933 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1504032640 con 0x7f1514152810 2024-09-10T10:26:47.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f1512ffd640 1 -- 172.21.15.120:0/2002316933 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15040329f0 con 0x7f1514152810 2024-09-10T10:26:47.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f151a877640 1 -- 172.21.15.120:0/2002316933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1514152810 msgr2=0x7f15140c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f151a877640 1 --2- 172.21.15.120:0/2002316933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1514152810 0x7f15140c79c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f15040097a0 tx=0x7f150402a2c0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f151a877640 1 -- 172.21.15.120:0/2002316933 shutdown_connections 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f151a877640 1 --2- 172.21.15.120:0/2002316933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1514152810 0x7f15140c79c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f151a877640 1 -- 172.21.15.120:0/2002316933 >> 172.21.15.120:0/2002316933 conn(0x7f1514067950 msgr2=0x7f1514067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f151a877640 1 -- 172.21.15.120:0/2002316933 shutdown_connections 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.556+0000 7f151a877640 1 -- 172.21.15.120:0/2002316933 wait complete. 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.557+0000 7f151a877640 1 Processor -- start 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f151a877640 1 -- start start 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f151a877640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151406d940 0x7f15140bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f151a877640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f151406ea60 0x7f15140b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f151a877640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1514152810 0x7f15140ba230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f151a877640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15140cd070 con 0x7f151406ea60 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f151a877640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15140c7f00 con 0x7f1514152810 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f151a877640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f15140cd2c0 con 0x7f151406d940 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f15137fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f151406ea60 0x7f15140b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f15137fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f151406ea60 0x7f15140b9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39908/0 (socket says 172.21.15.120:39908) 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f15137fe640 1 -- 172.21.15.120:0/1637249763 learned_addr learned my addr 172.21.15.120:0/1637249763 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.558+0000 7f1513fff640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151406d940 0x7f15140bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f1518e14640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1514152810 0x7f15140ba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f15137fe640 1 -- 172.21.15.120:0/1637249763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151406d940 msgr2=0x7f15140bcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f15137fe640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151406d940 0x7f15140bcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f15137fe640 1 -- 172.21.15.120:0/1637249763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1514152810 msgr2=0x7f15140ba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f15137fe640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1514152810 0x7f15140ba230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f15137fe640 1 -- 172.21.15.120:0/1637249763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15140b8fe0 con 0x7f151406ea60 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f15137fe640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f151406ea60 0x7f15140b9cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f150800c530 tx=0x7f150800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:47.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f1510ff9640 1 -- 172.21.15.120:0/1637249763 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508017070 con 0x7f151406ea60 2024-09-10T10:26:47.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f1510ff9640 1 -- 172.21.15.120:0/1637249763 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1508005ce0 con 0x7f151406ea60 2024-09-10T10:26:47.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f151a877640 1 -- 172.21.15.120:0/1637249763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15140c6140 con 0x7f151406ea60 2024-09-10T10:26:47.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f1510ff9640 1 -- 172.21.15.120:0/1637249763 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080072b0 con 0x7f151406ea60 2024-09-10T10:26:47.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.559+0000 7f151a877640 1 -- 172.21.15.120:0/1637249763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15140bbb40 con 0x7f151406ea60 2024-09-10T10:26:47.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.560+0000 7f151a877640 1 -- 172.21.15.120:0/1637249763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15140cd5a0 con 0x7f151406ea60 2024-09-10T10:26:47.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.560+0000 7f1510ff9640 1 -- 172.21.15.120:0/1637249763 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f150801a070 con 0x7f151406ea60 2024-09-10T10:26:47.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.561+0000 7f1510ff9640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f150006be30 0x7f150006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.561+0000 7f1510ff9640 1 -- 172.21.15.120:0/1637249763 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f150808a500 con 0x7f151406ea60 2024-09-10T10:26:47.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.561+0000 7f1513fff640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f150006be30 0x7f150006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.561+0000 7f1513fff640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f150006be30 0x7f150006e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1504034000 tx=0x7f150402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:47.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.565+0000 7f1510ff9640 1 -- 172.21.15.120:0/1637249763 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f15080585e0 con 0x7f151406ea60 2024-09-10T10:26:47.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.732+0000 7f151a877640 1 -- 172.21.15.120:0/1637249763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15140c6d70 con 0x7f151406ea60 2024-09-10T10:26:47.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.733+0000 7f1510ff9640 1 -- 172.21.15.120:0/1637249763 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18635 (secure 0 0 0) 0x7f150805c8e0 con 0x7f151406ea60 2024-09-10T10:26:47.736 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:26:47.736 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":25,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:46.148813+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:26:47.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 -- 172.21.15.120:0/1637249763 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f150006be30 msgr2=0x7f150006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f150006be30 0x7f150006e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1504034000 tx=0x7f150402e000 comp rx=0 tx=0).stop 2024-09-10T10:26:47.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 -- 172.21.15.120:0/1637249763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f151406ea60 msgr2=0x7f15140b9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f151406ea60 0x7f15140b9cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f150800c530 tx=0x7f150800c9f0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 -- 172.21.15.120:0/1637249763 shutdown_connections 2024-09-10T10:26:47.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f150006be30 0x7f150006e2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1514152810 0x7f15140ba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f151406ea60 0x7f15140b9cf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 --2- 172.21.15.120:0/1637249763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151406d940 0x7f15140bcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 -- 172.21.15.120:0/1637249763 >> 172.21.15.120:0/1637249763 conn(0x7f1514067950 msgr2=0x7f151406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:47.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.735+0000 7f151a877640 1 -- 172.21.15.120:0/1637249763 shutdown_connections 2024-09-10T10:26:47.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.736+0000 7f151a877640 1 -- 172.21.15.120:0/1637249763 wait complete. 2024-09-10T10:26:47.764 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:26:47.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.861+0000 7f4b9cb4b640 1 Processor -- start 2024-09-10T10:26:47.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.861+0000 7f4b9cb4b640 1 -- start start 2024-09-10T10:26:47.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.862+0000 7f4b9cb4b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cb2f0 0x7f4b9815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.862+0000 7f4b9cb4b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b9815b1d0 con 0x7f4b980cc4a0 2024-09-10T10:26:47.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.862+0000 7f4b9cb4b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b9815b4b0 con 0x7f4b980cc0c0 2024-09-10T10:26:47.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.862+0000 7f4b9cb4b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4b980cbd00 con 0x7f4b980cb2f0 2024-09-10T10:26:47.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.862+0000 7f4b96d9d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f4b980cc4a0 0x7f4b98165430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:46244/0 (socket says 172.21.15.120:46244) 2024-09-10T10:26:47.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.862+0000 7f4b96d9d640 1 -- 172.21.15.120:0/970082075 learned_addr learned my addr 172.21.15.120:0/970082075 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:47.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b95d9b640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cb2f0 0x7f4b9815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1243036538 0 0) 0x7f4b9815b1d0 con 0x7f4b980cc4a0 2024-09-10T10:26:47.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b80003520 con 0x7f4b980cc4a0 2024-09-10T10:26:47.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 47905214 0 0) 0x7f4b9815b4b0 con 0x7f4b980cc0c0 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b9815b1d0 con 0x7f4b980cc0c0 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2518620846 0 0) 0x7f4b80003520 con 0x7f4b980cc4a0 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cb2f0 msgr2=0x7f4b9815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cb2f0 0x7f4b9815aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 >> v1:172.21.15.134:6789/0 conn(0x7f4b980cc0c0 legacy=0x7f4b9815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9815b790 con 0x7f4b980cc4a0 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f4b90001670 con 0x7f4b980cc4a0 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 >> v1:172.21.15.120:6789/0 conn(0x7f4b980cc4a0 legacy=0x7f4b98165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b80003b70 0x7f4b80003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cb2f0 0x7f4b80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 0x7f4b800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b980cbd00 con 0x7f4b80004d40 2024-09-10T10:26:47.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8000ba70 con 0x7f4b80003b70 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8000bc70 con 0x7f4b980cb2f0 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b96d9d640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 0x7f4b800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b9659c640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b80003b70 0x7f4b80003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b95d9b640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cb2f0 0x7f4b80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b96d9d640 1 -- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cb2f0 msgr2=0x7f4b80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b96d9d640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cb2f0 0x7f4b80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b96d9d640 1 -- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b80003b70 msgr2=0x7f4b80003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b96d9d640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b80003b70 0x7f4b80003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.863+0000 7f4b96d9d640 1 -- 172.21.15.120:0/970082075 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b80003520 con 0x7f4b80004d40 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.864+0000 7f4b96d9d640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 0x7f4b800051c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b90007fc0 tx=0x7f4b80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4663b433de09c02f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.864+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90007be0 con 0x7f4b80004d40 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.864+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b90005af0 con 0x7f4b80004d40 2024-09-10T10:26:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.864+0000 7f4b9559a640 1 -- 172.21.15.120:0/970082075 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90007070 con 0x7f4b80004d40 2024-09-10T10:26:47.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.865+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/970082075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 msgr2=0x7f4b800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.865+0000 7f4b9cb4b640 1 --2- 172.21.15.120:0/970082075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 0x7f4b800051c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b90007fc0 tx=0x7f4b80005830 comp rx=0 tx=0).stop 2024-09-10T10:26:47.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.865+0000 7f4b96d9d640 1 -- 172.21.15.120:0/970082075 reap_dead start 2024-09-10T10:26:47.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.865+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/970082075 shutdown_connections 2024-09-10T10:26:47.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.865+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/970082075 >> 172.21.15.120:0/970082075 conn(0x7f4b9806d560 msgr2=0x7f4b9806d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:47.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.865+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/970082075 shutdown_connections 2024-09-10T10:26:47.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.865+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/970082075 wait complete. 2024-09-10T10:26:47.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.866+0000 7f4b9cb4b640 1 Processor -- start 2024-09-10T10:26:47.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.866+0000 7f4b9cb4b640 1 -- start start 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9cb4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 0x7f4b980d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9cb4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b980c7ec0 0x7f4b980d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9cb4b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cf4d0 0x7f4b980cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9cb4b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9815e540 con 0x7f4b80004d40 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9cb4b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9815ba70 con 0x7f4b980c7ec0 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9cb4b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4b980cc8b0 con 0x7f4b980cf4d0 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 0x7f4b980d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 0x7f4b980d07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39926/0 (socket says 172.21.15.120:39926) 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9659c640 1 -- 172.21.15.120:0/124499898 learned_addr learned my addr 172.21.15.120:0/124499898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b95d9b640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b980c7ec0 0x7f4b980d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b96d9d640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cf4d0 0x7f4b980cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9659c640 1 -- 172.21.15.120:0/124499898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cf4d0 msgr2=0x7f4b980cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9659c640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cf4d0 0x7f4b980cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9659c640 1 -- 172.21.15.120:0/124499898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b980c7ec0 msgr2=0x7f4b980d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9659c640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b980c7ec0 0x7f4b980d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9659c640 1 -- 172.21.15.120:0/124499898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b980c6ba0 con 0x7f4b80004d40 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9659c640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 0x7f4b980d07a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800fb10 tx=0x7f4b8800ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b86ffd640 1 -- 172.21.15.120:0/124499898 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8801a070 con 0x7f4b80004d40 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b86ffd640 1 -- 172.21.15.120:0/124499898 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b88006d60 con 0x7f4b80004d40 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/124499898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b981f0680 con 0x7f4b80004d40 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b86ffd640 1 -- 172.21.15.120:0/124499898 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88003b90 con 0x7f4b80004d40 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.867+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/124499898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b980ce9f0 con 0x7f4b80004d40 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.868+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/124499898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b980c5560 con 0x7f4b80004d40 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.869+0000 7f4b86ffd640 1 -- 172.21.15.120:0/124499898 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b8801d070 con 0x7f4b80004d40 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.869+0000 7f4b86ffd640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4b7806be60 0x7f4b7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:47.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.869+0000 7f4b86ffd640 1 -- 172.21.15.120:0/124499898 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4b8808ac60 con 0x7f4b80004d40 2024-09-10T10:26:47.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.869+0000 7f4b95d9b640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4b7806be60 0x7f4b7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:47.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.870+0000 7f4b95d9b640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4b7806be60 0x7f4b7806e320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b981ee0d0 tx=0x7f4b8c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:47.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:47.874+0000 7f4b86ffd640 1 -- 172.21.15.120:0/124499898 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b88058c80 con 0x7f4b80004d40 2024-09-10T10:26:48.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.072+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/124499898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b980cbea0 con 0x7f4b80004d40 2024-09-10T10:26:48.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.073+0000 7f4b86ffd640 1 -- 172.21.15.120:0/124499898 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+6876 (secure 0 0 0) 0x7f4b8805cf80 con 0x7f4b80004d40 2024-09-10T10:26:48.075 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:26:48.075 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":6,"btime":"2024-09-10T10:26:46:830817+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:26:41.113510+0000","modified":"2024-09-10T10:26:46.822339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-10T10:26:48.075 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 6 2024-09-10T10:26:48.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/124499898 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4b7806be60 msgr2=0x7f4b7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:48.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4b7806be60 0x7f4b7806e320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b981ee0d0 tx=0x7f4b8c009190 comp rx=0 tx=0).stop 2024-09-10T10:26:48.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/124499898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 msgr2=0x7f4b980d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:48.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 0x7f4b980d07a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800fb10 tx=0x7f4b8800ff10 comp rx=0 tx=0).stop 2024-09-10T10:26:48.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/124499898 shutdown_connections 2024-09-10T10:26:48.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4b7806be60 0x7f4b7806e320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b980cf4d0 0x7f4b980cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b980c7ec0 0x7f4b980d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 --2- 172.21.15.120:0/124499898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80004d40 0x7f4b980d07a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/124499898 >> 172.21.15.120:0/124499898 conn(0x7f4b9806d560 msgr2=0x7f4b98150c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:48.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/124499898 shutdown_connections 2024-09-10T10:26:48.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.075+0000 7f4b9cb4b640 1 -- 172.21.15.120:0/124499898 wait complete. 2024-09-10T10:26:48.088 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:26:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.183+0000 7f39d16d0640 1 Processor -- start 2024-09-10T10:26:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.183+0000 7f39d16d0640 1 -- start start 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39d16d0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f39cc0cb2f0 0x7f39cc15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39d16d0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39cc15b1d0 con 0x7f39cc0cc0c0 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39d16d0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39cc15b4b0 con 0x7f39cc0cc4a0 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39d16d0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f39cc0cbd00 con 0x7f39cc0cb2f0 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39caffd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f39cc0cc0c0 0x7f39cc15e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:46248/0 (socket says 172.21.15.120:46248) 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39caffd640 1 -- 172.21.15.120:0/3236643644 learned_addr learned my addr 172.21.15.120:0/3236643644 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39c27fc640 1 --2- 172.21.15.120:0/3236643644 >> v2:172.21.15.134:3300/0 conn(0x7f39cc0cb2f0 0x7f39cc15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1547224248 0 0) 0x7f39cc15b1d0 con 0x7f39cc0cc0c0 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39b8003520 con 0x7f39cc0cc0c0 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1677687336 0 0) 0x7f39b8003520 con 0x7f39cc0cc0c0 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 >> v2:172.21.15.134:3300/0 conn(0x7f39cc0cb2f0 msgr2=0x7f39cc15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39ca7fc640 1 --2- 172.21.15.120:0/3236643644 >> v2:172.21.15.134:3300/0 conn(0x7f39cc0cb2f0 0x7f39cc15aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 >> v1:172.21.15.134:6789/0 conn(0x7f39cc0cc4a0 legacy=0x7f39cc165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc15b790 con 0x7f39cc0cc0c0 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f39bc001670 con 0x7f39cc0cc0c0 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 >> v1:172.21.15.120:6789/0 conn(0x7f39cc0cc0c0 legacy=0x7f39cc15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.184+0000 7f39ca7fc640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b80039b0 0x7f39b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39ca7fc640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39b8004800 0x7f39b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39ca7fc640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39b80051c0 0x7f39b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc0cbd00 con 0x7f39b8004800 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39b800c060 con 0x7f39b80051c0 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f39b800c260 con 0x7f39b80039b0 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39c27fc640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39b8004800 0x7f39b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39cb7fe640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39b80051c0 0x7f39b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39caffd640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b80039b0 0x7f39b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39c27fc640 1 -- 172.21.15.120:0/3236643644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b80039b0 msgr2=0x7f39b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39c27fc640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b80039b0 0x7f39b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39c27fc640 1 -- 172.21.15.120:0/3236643644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39b80051c0 msgr2=0x7f39b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39c27fc640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39b80051c0 0x7f39b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39cb7fe640 1 -- 172.21.15.120:0/3236643644 reap_dead start 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.185+0000 7f39c27fc640 1 -- 172.21.15.120:0/3236643644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b8003520 con 0x7f39b8004800 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39c27fc640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39b8004800 0x7f39b8004c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009ac0 tx=0x7f39b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a462ff362eca30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4036070 con 0x7f39b8004800 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b4031400 con 0x7f39b8004800 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39ca7fc640 1 -- 172.21.15.120:0/3236643644 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4009690 con 0x7f39b8004800 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39d16d0640 1 -- 172.21.15.120:0/3236643644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39b8004800 msgr2=0x7f39b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39d16d0640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39b8004800 0x7f39b8004c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009ac0 tx=0x7f39b8005830 comp rx=0 tx=0).stop 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39d16d0640 1 -- 172.21.15.120:0/3236643644 shutdown_connections 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39d16d0640 1 --2- 172.21.15.120:0/3236643644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39b8004800 0x7f39b8004c80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39d16d0640 1 -- 172.21.15.120:0/3236643644 >> 172.21.15.120:0/3236643644 conn(0x7f39cc06d560 msgr2=0x7f39cc06d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39d16d0640 1 -- 172.21.15.120:0/3236643644 shutdown_connections 2024-09-10T10:26:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.186+0000 7f39d16d0640 1 -- 172.21.15.120:0/3236643644 wait complete. 2024-09-10T10:26:48.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.187+0000 7f39d16d0640 1 Processor -- start 2024-09-10T10:26:48.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.188+0000 7f39d16d0640 1 -- start start 2024-09-10T10:26:48.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.188+0000 7f39d16d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b8004800 0x7f39cc14ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:48.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.188+0000 7f39d16d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39cc1f11e0 0x7f39cc147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.188+0000 7f39d16d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39cc148380 0x7f39cc0b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.188+0000 7f39d16d0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc15e540 con 0x7f39cc1f11e0 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.188+0000 7f39d16d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc15ba70 con 0x7f39cc148380 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.188+0000 7f39d16d0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f39cc0cc8b0 con 0x7f39b8004800 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.188+0000 7f39c27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39cc1f11e0 0x7f39cc147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.188+0000 7f39c27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39cc1f11e0 0x7f39cc147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39942/0 (socket says 172.21.15.120:39942) 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.188+0000 7f39c27fc640 1 -- 172.21.15.120:0/4174527361 learned_addr learned my addr 172.21.15.120:0/4174527361 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39c27fc640 1 -- 172.21.15.120:0/4174527361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b8004800 msgr2=0x7f39cc14ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39caffd640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b8004800 0x7f39cc14ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39cb7fe640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39cc148380 0x7f39cc0b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39c27fc640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b8004800 0x7f39cc14ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39c27fc640 1 -- 172.21.15.120:0/4174527361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39cc148380 msgr2=0x7f39cc0b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39c27fc640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39cc148380 0x7f39cc0b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39c27fc640 1 -- 172.21.15.120:0/4174527361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc158790 con 0x7f39cc1f11e0 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39cb7fe640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39cc148380 0x7f39cc0b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39c27fc640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39cc1f11e0 0x7f39cc147e40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009be0 tx=0x7f39b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39c3fff640 1 -- 172.21.15.120:0/4174527361 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b403f020 con 0x7f39cc1f11e0 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39c3fff640 1 -- 172.21.15.120:0/4174527361 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b4037040 con 0x7f39cc1f11e0 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39d16d0640 1 -- 172.21.15.120:0/4174527361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39cc158a30 con 0x7f39cc1f11e0 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39c3fff640 1 -- 172.21.15.120:0/4174527361 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4036070 con 0x7f39cc1f11e0 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.189+0000 7f39d16d0640 1 -- 172.21.15.120:0/4174527361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39cc158ff0 con 0x7f39cc1f11e0 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.190+0000 7f39d16d0640 1 -- 172.21.15.120:0/4174527361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39cc0c5560 con 0x7f39cc1f11e0 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.190+0000 7f39c3fff640 1 -- 172.21.15.120:0/4174527361 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f39b4042020 con 0x7f39cc1f11e0 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.191+0000 7f39c3fff640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39ac06be30 0x7f39ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.191+0000 7f39c3fff640 1 -- 172.21.15.120:0/4174527361 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f39b40ab2c0 con 0x7f39cc1f11e0 2024-09-10T10:26:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.191+0000 7f39caffd640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39ac06be30 0x7f39ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:48.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.191+0000 7f39caffd640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39ac06be30 0x7f39ac06e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39bc001eb0 tx=0x7f39bc000fe0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:48.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.195+0000 7f39c3fff640 1 -- 172.21.15.120:0/4174527361 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f39b4049030 con 0x7f39cc1f11e0 2024-09-10T10:26:48.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.364+0000 7f39d16d0640 1 -- 172.21.15.120:0/4174527361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39cc0cbea0 con 0x7f39cc1f11e0 2024-09-10T10:26:48.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.365+0000 7f39c3fff640 1 -- 172.21.15.120:0/4174527361 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f39b40799f0 con 0x7f39cc1f11e0 2024-09-10T10:26:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:26:48.368 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:47.823215+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:26:48.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 -- 172.21.15.120:0/4174527361 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39ac06be30 msgr2=0x7f39ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:48.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39ac06be30 0x7f39ac06e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39bc001eb0 tx=0x7f39bc000fe0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 -- 172.21.15.120:0/4174527361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39cc1f11e0 msgr2=0x7f39cc147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:48.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39cc1f11e0 0x7f39cc147e40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009be0 tx=0x7f39b4004730 comp rx=0 tx=0).stop 2024-09-10T10:26:48.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 -- 172.21.15.120:0/4174527361 shutdown_connections 2024-09-10T10:26:48.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39ac06be30 0x7f39ac06e2f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39cc148380 0x7f39cc0b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39cc1f11e0 0x7f39cc147e40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 --2- 172.21.15.120:0/4174527361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b8004800 0x7f39cc14ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:48.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 -- 172.21.15.120:0/4174527361 >> 172.21.15.120:0/4174527361 conn(0x7f39cc06d560 msgr2=0x7f39cc0c6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:48.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 -- 172.21.15.120:0/4174527361 shutdown_connections 2024-09-10T10:26:48.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:48.367+0000 7f39d16d0640 1 -- 172.21.15.120:0/4174527361 wait complete. 2024-09-10T10:26:48.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:26:41.113510+0000', 'modified': '2024-09-10T10:26:46.822339+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6835/1434442356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6833', 'nonce': 1434442356}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1434442356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4306, 'qdb_cluster': [4306]} 2024-09-10T10:26:48.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:26:48.984 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T10:26:49.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.083+0000 7f65cd0a5640 1 Processor -- start 2024-09-10T10:26:49.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.083+0000 7f65cd0a5640 1 -- start start 2024-09-10T10:26:49.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.083+0000 7f65cd0a5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65c8152ae0 0x7f65c8159170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.083+0000 7f65cd0a5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 0x7f65c8163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.083+0000 7f65cd0a5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c815cc60 con 0x7f65c80c9580 2024-09-10T10:26:49.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.083+0000 7f65cd0a5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f65c81596b0 con 0x7f65c8152ae0 2024-09-10T10:26:49.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.083+0000 7f65cd0a5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f65c8152ec0 con 0x7f65c80c5140 2024-09-10T10:26:49.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.084+0000 7f65c759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 0x7f65c8163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.084+0000 7f65c759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 0x7f65c8163100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:55276/0 (socket says 172.21.15.120:55276) 2024-09-10T10:26:49.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.084+0000 7f65c6d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65c8152ae0 0x7f65c8159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.084+0000 7f65c759e640 1 -- 172.21.15.120:0/708998772 learned_addr learned my addr 172.21.15.120:0/708998772 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:49.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.084+0000 7f65c759e640 1 -- 172.21.15.120:0/708998772 >> v2:172.21.15.134:3300/0 conn(0x7f65c8152ae0 msgr2=0x7f65c8159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:49.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.084+0000 7f65c759e640 1 --2- 172.21.15.120:0/708998772 >> v2:172.21.15.134:3300/0 conn(0x7f65c8152ae0 0x7f65c8159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.084+0000 7f65c759e640 1 -- 172.21.15.120:0/708998772 >> v1:172.21.15.134:6789/0 conn(0x7f65c80c9580 legacy=0x7f65c80c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:49.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.084+0000 7f65c759e640 1 -- 172.21.15.120:0/708998772 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c80c9a70 con 0x7f65c80c5140 2024-09-10T10:26:49.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.084+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/708998772 >> v2:172.21.15.134:3300/0 conn(0x7f65c8152ae0 0x7f65c8159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:49.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.085+0000 7f65c759e640 1 --2- 172.21.15.120:0/708998772 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 0x7f65c8163100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009870 tx=0x7f65b802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1717c033d471b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:49.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.085+0000 7f65c5d9b640 1 -- 172.21.15.120:0/708998772 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8037070 con 0x7f65c80c5140 2024-09-10T10:26:49.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.085+0000 7f65c5d9b640 1 -- 172.21.15.120:0/708998772 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b8032440 con 0x7f65c80c5140 2024-09-10T10:26:49.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.085+0000 7f65c5d9b640 1 -- 172.21.15.120:0/708998772 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8032810 con 0x7f65c80c5140 2024-09-10T10:26:49.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.086+0000 7f65cd0a5640 1 -- 172.21.15.120:0/708998772 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 msgr2=0x7f65c8163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:49.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.086+0000 7f65cd0a5640 1 --2- 172.21.15.120:0/708998772 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 0x7f65c8163100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009870 tx=0x7f65b802a270 comp rx=0 tx=0).stop 2024-09-10T10:26:49.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.086+0000 7f65cd0a5640 1 -- 172.21.15.120:0/708998772 shutdown_connections 2024-09-10T10:26:49.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.086+0000 7f65cd0a5640 1 --2- 172.21.15.120:0/708998772 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 0x7f65c8163100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.086+0000 7f65cd0a5640 1 --2- 172.21.15.120:0/708998772 >> v2:172.21.15.134:3300/0 conn(0x7f65c8152ae0 0x7f65c8159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.086+0000 7f65cd0a5640 1 -- 172.21.15.120:0/708998772 >> 172.21.15.120:0/708998772 conn(0x7f65c806bc10 msgr2=0x7f65c806fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:49.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.086+0000 7f65cd0a5640 1 -- 172.21.15.120:0/708998772 shutdown_connections 2024-09-10T10:26:49.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.086+0000 7f65cd0a5640 1 -- 172.21.15.120:0/708998772 wait complete. 2024-09-10T10:26:49.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.087+0000 7f65cd0a5640 1 Processor -- start 2024-09-10T10:26:49.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.087+0000 7f65cd0a5640 1 -- start start 2024-09-10T10:26:49.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65cd0a5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 0x7f65c8150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65cd0a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c80c9580 0x7f65c80c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65cd0a5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c8152ae0 0x7f65c81f0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65cd0a5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65c8158970 con 0x7f65c8152ae0 2024-09-10T10:26:49.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65cd0a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80ca030 con 0x7f65c80c9580 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65cd0a5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f65c8158c50 con 0x7f65c80c5140 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65c759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c8152ae0 0x7f65c81f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65c759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c8152ae0 0x7f65c81f0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39944/0 (socket says 172.21.15.120:39944) 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65c759e640 1 -- 172.21.15.120:0/3763454371 learned_addr learned my addr 172.21.15.120:0/3763454371 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65c659c640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c80c9580 0x7f65c80c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 0x7f65c8150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65c759e640 1 -- 172.21.15.120:0/3763454371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 msgr2=0x7f65c8150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65c759e640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 0x7f65c8150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65c759e640 1 -- 172.21.15.120:0/3763454371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c80c9580 msgr2=0x7f65c80c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65c759e640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c80c9580 0x7f65c80c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.088+0000 7f65c759e640 1 -- 172.21.15.120:0/3763454371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c814e2e0 con 0x7f65c8152ae0 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.089+0000 7f65c759e640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c8152ae0 0x7f65c81f0040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009870 tx=0x7f65b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:49.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.089+0000 7f65bf7fe640 1 -- 172.21.15.120:0/3763454371 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8041020 con 0x7f65c8152ae0 2024-09-10T10:26:49.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.089+0000 7f65bf7fe640 1 -- 172.21.15.120:0/3763454371 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b803d640 con 0x7f65c8152ae0 2024-09-10T10:26:49.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.089+0000 7f65cd0a5640 1 -- 172.21.15.120:0/3763454371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c814e580 con 0x7f65c8152ae0 2024-09-10T10:26:49.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.089+0000 7f65bf7fe640 1 -- 172.21.15.120:0/3763454371 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8037040 con 0x7f65c8152ae0 2024-09-10T10:26:49.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.089+0000 7f65cd0a5640 1 -- 172.21.15.120:0/3763454371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c8151b10 con 0x7f65c8152ae0 2024-09-10T10:26:49.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.090+0000 7f65bf7fe640 1 -- 172.21.15.120:0/3763454371 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65b8044020 con 0x7f65c8152ae0 2024-09-10T10:26:49.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.091+0000 7f65bf7fe640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f65a806be60 0x7f65a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.091+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f65a806be60 0x7f65a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.091+0000 7f65bf7fe640 1 -- 172.21.15.120:0/3763454371 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f65b80ab820 con 0x7f65c8152ae0 2024-09-10T10:26:49.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.091+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f65a806be60 0x7f65a806e320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009bd0 tx=0x7f65c000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:49.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.092+0000 7f65cfe45740 1 Processor -- start 2024-09-10T10:26:49.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.092+0000 7f65cfe45740 1 -- start start 2024-09-10T10:26:49.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221750 0x55e4f92599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f921ef80 0x55e4f9259f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f921da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65cfe45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4f9182520 con 0x55e4f9221b30 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65cfe45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4f912a3e0 con 0x55e4f9221750 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65cfe45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55e4f917e4e0 con 0x55e4f921ef80 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f921da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f921da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39952/0 (socket says 172.21.15.120:39952) 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c759e640 1 -- 172.21.15.120:0/2846876033 learned_addr learned my addr 172.21.15.120:0/2846876033 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/2846876033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221750 0x55e4f92599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c659c640 1 --2- 172.21.15.120:0/2846876033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f921ef80 0x55e4f9259f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c759e640 1 -- 172.21.15.120:0/2846876033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f921ef80 msgr2=0x55e4f9259f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c759e640 1 --2- 172.21.15.120:0/2846876033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f921ef80 0x55e4f9259f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c759e640 1 -- 172.21.15.120:0/2846876033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221750 msgr2=0x55e4f92599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c759e640 1 --2- 172.21.15.120:0/2846876033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221750 0x55e4f92599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c759e640 1 -- 172.21.15.120:0/2846876033 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4f91452f0 con 0x55e4f9221b30 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65c759e640 1 --2- 172.21.15.120:0/2846876033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f921da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f65b8081e40 tx=0x7f65b8079e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46af15bd06bd4ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65bcff9640 1 -- 172.21.15.120:0/2846876033 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8082370 con 0x55e4f9221b30 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65bcff9640 1 -- 172.21.15.120:0/2846876033 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b8077a00 con 0x55e4f9221b30 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.093+0000 7f65bcff9640 1 -- 172.21.15.120:0/2846876033 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8081ee0 con 0x55e4f9221b30 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.094+0000 7f65cfe45740 1 -- 172.21.15.120:0/2846876033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 msgr2=0x55e4f921da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.094+0000 7f65cfe45740 1 --2- 172.21.15.120:0/2846876033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f921da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f65b8081e40 tx=0x7f65b8079e00 comp rx=0 tx=0).stop 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.094+0000 7f65cfe45740 1 -- 172.21.15.120:0/2846876033 shutdown_connections 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.094+0000 7f65cfe45740 1 --2- 172.21.15.120:0/2846876033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f921da60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.094+0000 7f65cfe45740 1 --2- 172.21.15.120:0/2846876033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f921ef80 0x55e4f9259f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.094+0000 7f65cfe45740 1 --2- 172.21.15.120:0/2846876033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221750 0x55e4f92599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.094+0000 7f65cfe45740 1 -- 172.21.15.120:0/2846876033 >> 172.21.15.120:0/2846876033 conn(0x55e4f921fe70 msgr2=0x55e4f9220270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.094+0000 7f65cfe45740 1 -- 172.21.15.120:0/2846876033 shutdown_connections 2024-09-10T10:26:49.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.094+0000 7f65cfe45740 1 -- 172.21.15.120:0/2846876033 wait complete. 2024-09-10T10:26:49.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.095+0000 7f65cfe45740 1 -- start start 2024-09-10T10:26:49.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.095+0000 7f65cfe45740 1 Processor -- start 2024-09-10T10:26:49.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.095+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f921ef80 0x55e4f9271510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.095+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 0x55e4f9244fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.095+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f924b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.095+0000 7f65cfe45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4f9182520 con 0x55e4f9221b30 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.095+0000 7f65cfe45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4f910b450 con 0x55e4f921ef80 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.095+0000 7f65cfe45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55e4f9220fe0 con 0x55e4f9221750 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65cfe45740 10 client.0 _ll_register_callbacks cb 0x55e4f8e94ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f924b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f924b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39966/0 (socket says 172.21.15.120:39966) 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c759e640 1 -- 172.21.15.120:0/3493858906 learned_addr learned my addr 172.21.15.120:0/3493858906 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f921ef80 0x55e4f9271510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c659c640 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 0x55e4f9244fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c759e640 1 -- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 msgr2=0x55e4f9244fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c759e640 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 0x55e4f9244fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c759e640 1 -- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f921ef80 msgr2=0x55e4f9271510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c759e640 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f921ef80 0x55e4f9271510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c759e640 1 -- 172.21.15.120:0/3493858906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4f8e952d0 con 0x55e4f9221b30 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65c759e640 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f924b080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65b80a9a60 tx=0x7f65b8006ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4741b8fd3aaa3fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65a5ffb640 1 -- 172.21.15.120:0/3493858906 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8096060 con 0x55e4f9221b30 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65a5ffb640 1 -- 172.21.15.120:0/3493858906 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f65b803db50 con 0x55e4f9221b30 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65cfe45740 1 -- 172.21.15.120:0/3493858906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b8077a00 con 0x55e4f9221b30 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65a5ffb640 1 -- 172.21.15.120:0/3493858906 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f65b809d050 con 0x55e4f9221b30 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65cfe45740 10 client.4410 fetch_fsmap learned FSMap version 6 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65cfe45740 1 -- 172.21.15.120:0/3493858906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55e4f8f5f6b0 con 0x55e4f9221b30 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65a5ffb640 1 -- 172.21.15.120:0/3493858906 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b8077a00 con 0x55e4f9221b30 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.096+0000 7f65a5ffb640 1 -- 172.21.15.120:0/3493858906 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 6) ==== 4720+0+0 (secure 0 0 0) 0x7f65b803a030 con 0x55e4f9221b30 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.097+0000 7f65cfe45740 10 client.4410 fetch_fsmap finished waiting for FSMap version 6 2024-09-10T10:26:49.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.097+0000 7f65cfe45740 10 client.4410 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:26:49.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.097+0000 7f65cfe45740 20 client.4410 populate_metadata read hostname 'smithi120' 2024-09-10T10:26:49.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.097+0000 7f65cfe45740 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x55e4f924f440 0x55e4f927eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:49.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.097+0000 7f65cfe45740 4 client.4410 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:26:49.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.097+0000 7f65cfe45740 1 -- 172.21.15.120:0/3493858906 --> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4f918fc50 con 0x55e4f924f440 2024-09-10T10:26:49.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.097+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x55e4f924f440 0x55e4f927eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:49.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.097+0000 7f65a5ffb640 10 client.4410 ms_handle_connect on v2:172.21.15.134:6833/1434442356 2024-09-10T10:26:49.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:49.097+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x55e4f924f440 0x55e4f927eb60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6485c7dd83f3cde2 server_cookie=3191ac3c5f3f7522 in_seq=0 out_seq=0 2024-09-10T10:26:51.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65a5ffb640 1 -- 172.21.15.120:0/3493858906 <== mds.0 v2:172.21.15.134:6833/1434442356 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4f918fc50 con 0x55e4f924f440 2024-09-10T10:26:51.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65a5ffb640 10 client.4410 handle_command_reply: tid=0 2024-09-10T10:26:51.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 client.4410 shutdown 2024-09-10T10:26:51.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 -- 172.21.15.120:0/3493858906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 msgr2=0x55e4f924b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:51.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f924b080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65b80a9a60 tx=0x7f65b8006ed0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 -- 172.21.15.120:0/3493858906 shutdown_connections 2024-09-10T10:26:51.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x55e4f924f440 0x55e4f927eb60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f9221b30 0x55e4f924b080 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 0x55e4f9244fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 --2- 172.21.15.120:0/3493858906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f921ef80 0x55e4f9271510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 -- 172.21.15.120:0/3493858906 >> 172.21.15.120:0/3493858906 conn(0x55e4f921fe70 msgr2=0x55e4f9259950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:51.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65a5ffb640 0 client.4410 ms_handle_reset on v2:172.21.15.134:6833/1434442356 2024-09-10T10:26:51.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 -- 172.21.15.120:0/3493858906 shutdown_connections 2024-09-10T10:26:51.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.829+0000 7f65cfe45740 1 -- 172.21.15.120:0/3493858906 wait complete. 2024-09-10T10:26:51.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.830+0000 7f65cfe45740 20 client.4410 trim_cache size 0 max 16384 2024-09-10T10:26:51.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.848+0000 7f65cfe45740 1 Processor -- start 2024-09-10T10:26:51.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.849+0000 7f65cfe45740 1 -- start start 2024-09-10T10:26:51.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.849+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f921ef80 0x55e4f92503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.849+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 0x55e4f9250900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.849+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221b30 0x55e4f9263a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.849+0000 7f65cfe45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81fb370 con 0x55e4f921ef80 2024-09-10T10:26:51.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.849+0000 7f65cfe45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4f910b450 con 0x55e4f9221b30 2024-09-10T10:26:51.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.849+0000 7f65cfe45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55e4f9182520 con 0x55e4f9221750 2024-09-10T10:26:51.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.849+0000 7f65c6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f921ef80 0x55e4f92503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.849+0000 7f65c6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f921ef80 0x55e4f92503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39974/0 (socket says 172.21.15.120:39974) 2024-09-10T10:26:51.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.849+0000 7f65c6d9d640 1 -- 172.21.15.120:0/750964345 learned_addr learned my addr 172.21.15.120:0/750964345 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:51.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65c759e640 1 --2- 172.21.15.120:0/750964345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221b30 0x55e4f9263a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65c659c640 1 --2- 172.21.15.120:0/750964345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 0x55e4f9250900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65c6d9d640 1 -- 172.21.15.120:0/750964345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 msgr2=0x55e4f9250900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:51.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/750964345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 0x55e4f9250900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65c6d9d640 1 -- 172.21.15.120:0/750964345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221b30 msgr2=0x55e4f9263a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:51.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/750964345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221b30 0x55e4f9263a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65c6d9d640 1 -- 172.21.15.120:0/750964345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4f921bba0 con 0x55e4f921ef80 2024-09-10T10:26:51.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65c659c640 1 --2- 172.21.15.120:0/750964345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 0x55e4f9250900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:26:51.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/750964345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f921ef80 0x55e4f92503c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65c00099d0 tx=0x7f65c0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fb8cc9fbc4dc5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:51.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65a7fff640 1 -- 172.21.15.120:0/750964345 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c001d040 con 0x55e4f921ef80 2024-09-10T10:26:51.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65a7fff640 1 -- 172.21.15.120:0/750964345 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c001e040 con 0x55e4f921ef80 2024-09-10T10:26:51.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.850+0000 7f65a7fff640 1 -- 172.21.15.120:0/750964345 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0026b90 con 0x55e4f921ef80 2024-09-10T10:26:51.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.851+0000 7f65cfe45740 1 -- 172.21.15.120:0/750964345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f921ef80 msgr2=0x55e4f92503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:51.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.851+0000 7f65cfe45740 1 --2- 172.21.15.120:0/750964345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f921ef80 0x55e4f92503c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65c00099d0 tx=0x7f65c0014000 comp rx=0 tx=0).stop 2024-09-10T10:26:51.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.851+0000 7f65cfe45740 1 -- 172.21.15.120:0/750964345 shutdown_connections 2024-09-10T10:26:51.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.851+0000 7f65cfe45740 1 --2- 172.21.15.120:0/750964345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221b30 0x55e4f9263a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.851+0000 7f65cfe45740 1 --2- 172.21.15.120:0/750964345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221750 0x55e4f9250900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.851+0000 7f65cfe45740 1 --2- 172.21.15.120:0/750964345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f921ef80 0x55e4f92503c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.851+0000 7f65cfe45740 1 -- 172.21.15.120:0/750964345 >> 172.21.15.120:0/750964345 conn(0x55e4f921fe70 msgr2=0x55e4f9220250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.851+0000 7f65cfe45740 1 -- 172.21.15.120:0/750964345 shutdown_connections 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.851+0000 7f65cfe45740 1 -- 172.21.15.120:0/750964345 wait complete. 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.852+0000 7f65cfe45740 1 -- start start 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.852+0000 7f65cfe45740 1 Processor -- start 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221750 0x55e4f92e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221b30 0x55e4f92544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65cfe45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f926da10 0x55e4f9254a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65cfe45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4f92455e0 con 0x55e4f926da10 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65cfe45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4f9274ee0 con 0x55e4f9221750 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65cfe45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55e4f924b5c0 con 0x55e4f9221b30 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65cfe45740 10 client.0 _ll_register_callbacks cb 0x7f65b8090070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f926da10 0x55e4f9254a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f926da10 0x55e4f9254a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39984/0 (socket says 172.21.15.120:39984) 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c759e640 1 -- 172.21.15.120:0/2849872204 learned_addr learned my addr 172.21.15.120:0/2849872204 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221750 0x55e4f92e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c659c640 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221b30 0x55e4f92544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c759e640 1 -- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221b30 msgr2=0x55e4f92544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c759e640 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221b30 0x55e4f92544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c759e640 1 -- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221750 msgr2=0x55e4f92e1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c759e640 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221750 0x55e4f92e1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c759e640 1 -- 172.21.15.120:0/2849872204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4f9242fe0 con 0x55e4f926da10 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c659c640 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221b30 0x55e4f92544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65c759e640 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f926da10 0x55e4f9254a00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f65b8079400 tx=0x7f65b807b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251afc7708fe88c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65a5ffb640 1 -- 172.21.15.120:0/2849872204 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b80823c0 con 0x55e4f926da10 2024-09-10T10:26:51.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65a5ffb640 1 -- 172.21.15.120:0/2849872204 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f65b8078d50 con 0x55e4f926da10 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.853+0000 7f65cfe45740 1 -- 172.21.15.120:0/2849872204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c001e040 con 0x55e4f926da10 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65a5ffb640 1 -- 172.21.15.120:0/2849872204 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f65b8077a00 con 0x55e4f926da10 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65cfe45740 10 client.4416 fetch_fsmap learned FSMap version 6 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65cfe45740 1 -- 172.21.15.120:0/2849872204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55e4f9242fe0 con 0x55e4f926da10 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65a5ffb640 1 -- 172.21.15.120:0/2849872204 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c001e040 con 0x55e4f926da10 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65a5ffb640 1 -- 172.21.15.120:0/2849872204 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 6) ==== 4720+0+0 (secure 0 0 0) 0x7f65b80a0070 con 0x55e4f926da10 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65cfe45740 10 client.4416 fetch_fsmap finished waiting for FSMap version 6 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65cfe45740 10 client.4416 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65cfe45740 20 client.4416 populate_metadata read hostname 'smithi120' 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65cfe45740 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x55e4f92e63c0 0x55e4f92eef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65cfe45740 4 client.4416 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65cfe45740 1 -- 172.21.15.120:0/2849872204 --> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] -- command(tid 0: {"prefix": "status"}) -- 0x55e4f9189fa0 con 0x55e4f92e63c0 2024-09-10T10:26:51.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.854+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x55e4f92e63c0 0x55e4f92eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65c6d9d640 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x55e4f92e63c0 0x55e4f92eef20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ffa6967225439f0 server_cookie=399f34821fe4b3a1 in_seq=0 out_seq=0 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65a5ffb640 10 client.4416 ms_handle_connect on v2:172.21.15.134:6833/1434442356 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65a5ffb640 1 -- 172.21.15.120:0/2849872204 <== mds.0 v2:172.21.15.134:6833/1434442356 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55e4f9189fa0 con 0x55e4f92e63c0 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65a5ffb640 10 client.4416 handle_command_reply: tid=0 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 client.4416 shutdown 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 -- 172.21.15.120:0/2849872204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f926da10 msgr2=0x55e4f9254a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f926da10 0x55e4f9254a00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f65b8079400 tx=0x7f65b807b000 comp rx=0 tx=0).stop 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 -- 172.21.15.120:0/2849872204 shutdown_connections 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x55e4f92e63c0 0x55e4f92eef20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4f926da10 0x55e4f9254a00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4f9221b30 0x55e4f92544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 --2- 172.21.15.120:0/2849872204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4f9221750 0x55e4f92e1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 -- 172.21.15.120:0/2849872204 >> 172.21.15.120:0/2849872204 conn(0x55e4f921fe70 msgr2=0x55e4f921f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65a5ffb640 0 client.4416 ms_handle_reset on v2:172.21.15.134:6833/1434442356 2024-09-10T10:26:51.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 -- 172.21.15.120:0/2849872204 shutdown_connections 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 1 -- 172.21.15.120:0/2849872204 wait complete. 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.855+0000 7f65cfe45740 20 client.4416 trim_cache size 0 max 16384 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 0, 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 4306, 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 10.08161956, 2024-09-10T10:26:51.863 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 5, 2024-09-10T10:26:51.864 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 26, 2024-09-10T10:26:51.864 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 25, 2024-09-10T10:26:51.864 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 16.086934111000001 2024-09-10T10:26:51.864 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:26:51.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 -- 172.21.15.120:0/3763454371 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f65a806be60 msgr2=0x7f65a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f65a806be60 0x7f65a806e320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009bd0 tx=0x7f65c000b040 comp rx=0 tx=0).stop 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 -- 172.21.15.120:0/3763454371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c8152ae0 msgr2=0x7f65c81f0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c8152ae0 0x7f65c81f0040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009870 tx=0x7f65b80046c0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 -- 172.21.15.120:0/3763454371 shutdown_connections 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f65a806be60 0x7f65a806e320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c8152ae0 0x7f65c81f0040 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c80c9580 0x7f65c80c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 --2- 172.21.15.120:0/3763454371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65c80c5140 0x7f65c8150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 -- 172.21.15.120:0/3763454371 >> 172.21.15.120:0/3763454371 conn(0x7f65c806bc10 msgr2=0x7f65c80ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 -- 172.21.15.120:0/3763454371 shutdown_connections 2024-09-10T10:26:51.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.856+0000 7f65cd0a5640 1 -- 172.21.15.120:0/3763454371 wait complete. 2024-09-10T10:26:51.871 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:26:51.872 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:26:51.872 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:26:51.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.973+0000 7f1f6e2ac640 1 Processor -- start 2024-09-10T10:26:51.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.973+0000 7f1f6e2ac640 1 -- start start 2024-09-10T10:26:51.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.974+0000 7f1f6e2ac640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1f680cc0c0 0x7f1f6815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.974+0000 7f1f6e2ac640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1f680cc4a0 0x7f1f68165430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.974+0000 7f1f6e2ac640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f6815b1d0 con 0x7f1f680cb2f0 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.974+0000 7f1f6e2ac640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1f680cbd00 con 0x7f1f680cc0c0 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.974+0000 7f1f6e2ac640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6815ba70 con 0x7f1f680cc4a0 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.974+0000 7f1f67fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1f680cc0c0 0x7f1f6815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.974+0000 7f1f677fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f1f680cb2f0 0x7f1f6815e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:46250/0 (socket says 172.21.15.120:46250) 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.974+0000 7f1f677fe640 1 -- 172.21.15.120:0/165435204 learned_addr learned my addr 172.21.15.120:0/165435204 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.974+0000 7f1f6c849640 1 --2- 172.21.15.120:0/165435204 >> v2:172.21.15.134:3300/0 conn(0x7f1f680cc4a0 0x7f1f68165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f67fff640 1 -- 172.21.15.120:0/165435204 >> v2:172.21.15.134:3300/0 conn(0x7f1f680cc4a0 msgr2=0x7f1f68165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f67fff640 1 --2- 172.21.15.120:0/165435204 >> v2:172.21.15.134:3300/0 conn(0x7f1f680cc4a0 0x7f1f68165430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f67fff640 1 -- 172.21.15.120:0/165435204 >> v1:172.21.15.120:6789/0 conn(0x7f1f680cb2f0 legacy=0x7f1f6815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f66ffd640 1 -- 172.21.15.120:0/165435204 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1854373457 0 0) 0x7f1f6815b1d0 con 0x7f1f680cb2f0 2024-09-10T10:26:51.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f67fff640 1 -- 172.21.15.120:0/165435204 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6815b4b0 con 0x7f1f680cc0c0 2024-09-10T10:26:51.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f6c849640 1 --2- 172.21.15.120:0/165435204 >> v2:172.21.15.134:3300/0 conn(0x7f1f680cc4a0 0x7f1f68165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:26:51.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f67fff640 1 --2- 172.21.15.120:0/165435204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 0x7f1f6815aab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f6806e100 tx=0x7f1f5802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdd25d30049a2fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:51.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f66ffd640 1 -- 172.21.15.120:0/165435204 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58037070 con 0x7f1f680cc0c0 2024-09-10T10:26:51.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f66ffd640 1 -- 172.21.15.120:0/165435204 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f58032440 con 0x7f1f680cc0c0 2024-09-10T10:26:51.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f66ffd640 1 -- 172.21.15.120:0/165435204 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58032810 con 0x7f1f680cc0c0 2024-09-10T10:26:51.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/165435204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 msgr2=0x7f1f6815aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:51.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f6e2ac640 1 --2- 172.21.15.120:0/165435204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 0x7f1f6815aab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f6806e100 tx=0x7f1f5802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/165435204 shutdown_connections 2024-09-10T10:26:51.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f6e2ac640 1 --2- 172.21.15.120:0/165435204 >> v2:172.21.15.134:3300/0 conn(0x7f1f680cc4a0 0x7f1f68165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f6e2ac640 1 --2- 172.21.15.120:0/165435204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 0x7f1f6815aab0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/165435204 >> 172.21.15.120:0/165435204 conn(0x7f1f6806d560 msgr2=0x7f1f6806d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:51.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/165435204 shutdown_connections 2024-09-10T10:26:51.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.976+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/165435204 wait complete. 2024-09-10T10:26:51.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.977+0000 7f1f6e2ac640 1 Processor -- start 2024-09-10T10:26:51.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.979+0000 7f1f6e2ac640 1 -- start start 2024-09-10T10:26:51.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.979+0000 7f1f6e2ac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f680cb2f0 0x7f1f680c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.979+0000 7f1f6e2ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 0x7f1f6815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.979+0000 7f1f6e2ac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f680cc4a0 0x7f1f680b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.979+0000 7f1f6e2ac640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6815e1d0 con 0x7f1f680cc0c0 2024-09-10T10:26:51.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.979+0000 7f1f6e2ac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6815e540 con 0x7f1f680cb2f0 2024-09-10T10:26:51.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.979+0000 7f1f6e2ac640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1f680cc8b0 con 0x7f1f680cc4a0 2024-09-10T10:26:51.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.979+0000 7f1f677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 0x7f1f6815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.979+0000 7f1f677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 0x7f1f6815d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40008/0 (socket says 172.21.15.120:40008) 2024-09-10T10:26:51.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.979+0000 7f1f677fe640 1 -- 172.21.15.120:0/2304317407 learned_addr learned my addr 172.21.15.120:0/2304317407 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:51.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f677fe640 1 -- 172.21.15.120:0/2304317407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f680cc4a0 msgr2=0x7f1f680b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f67fff640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f680cb2f0 0x7f1f680c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f6c849640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f680cc4a0 0x7f1f680b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f677fe640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f680cc4a0 0x7f1f680b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f677fe640 1 -- 172.21.15.120:0/2304317407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f680cb2f0 msgr2=0x7f1f680c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f677fe640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f680cb2f0 0x7f1f680c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f677fe640 1 -- 172.21.15.120:0/2304317407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f681f1a60 con 0x7f1f680cc0c0 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f677fe640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 0x7f1f6815d1d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c430 tx=0x7f1f5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f64ff9640 1 -- 172.21.15.120:0/2304317407 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0074e0 con 0x7f1f680cc0c0 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f64ff9640 1 -- 172.21.15.120:0/2304317407 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c0047e0 con 0x7f1f680cc0c0 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f64ff9640 1 -- 172.21.15.120:0/2304317407 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c004bd0 con 0x7f1f680cc0c0 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/2304317407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6815c280 con 0x7f1f680cc0c0 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.980+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/2304317407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6815c840 con 0x7f1f680cc0c0 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.981+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/2304317407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f680b4780 con 0x7f1f680cc0c0 2024-09-10T10:26:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.981+0000 7f1f64ff9640 1 -- 172.21.15.120:0/2304317407 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f5c011070 con 0x7f1f680cc0c0 2024-09-10T10:26:51.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.981+0000 7f1f64ff9640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1f5406be30 0x7f1f5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:51.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.981+0000 7f1f64ff9640 1 -- 172.21.15.120:0/2304317407 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1f5c08ae40 con 0x7f1f680cc0c0 2024-09-10T10:26:51.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.981+0000 7f1f67fff640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1f5406be30 0x7f1f5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:51.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.981+0000 7f1f67fff640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1f5406be30 0x7f1f5406e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f58034000 tx=0x7f1f5802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:51.986+0000 7f1f64ff9640 1 -- 172.21.15.120:0/2304317407 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1f680b4780 con 0x7f1f680cc0c0 2024-09-10T10:26:52.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.189+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/2304317407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1f680cbea0 con 0x7f1f680cc0c0 2024-09-10T10:26:52.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.189+0000 7f1f64ff9640 1 -- 172.21.15.120:0/2304317407 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7f1f5c0594b0 con 0x7f1f680cc0c0 2024-09-10T10:26:52.190 INFO:teuthology.orchestra.run.smithi120.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T10:26:52.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/2304317407 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1f5406be30 msgr2=0x7f1f5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:52.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1f5406be30 0x7f1f5406e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f58034000 tx=0x7f1f5802e000 comp rx=0 tx=0).stop 2024-09-10T10:26:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/2304317407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 msgr2=0x7f1f6815d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 0x7f1f6815d1d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c430 tx=0x7f1f5c00c8f0 comp rx=0 tx=0).stop 2024-09-10T10:26:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/2304317407 shutdown_connections 2024-09-10T10:26:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1f5406be30 0x7f1f5406e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f680cc4a0 0x7f1f680b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f680cc0c0 0x7f1f6815d1d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 --2- 172.21.15.120:0/2304317407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f680cb2f0 0x7f1f680c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/2304317407 >> 172.21.15.120:0/2304317407 conn(0x7f1f6806d560 msgr2=0x7f1f680ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/2304317407 shutdown_connections 2024-09-10T10:26:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52.191+0000 7f1f6e2ac640 1 -- 172.21.15.120:0/2304317407 wait complete. 2024-09-10T10:26:52.203 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:26:52.203 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:26:52.203 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi120.front.sepia.ceph.com 2024-09-10T10:26:52.203 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T10:26:52.203 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:26:52.203 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:26:52.203 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:26:52.203 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:26:52.203 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T10:26:52.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:52.204 DEBUG:teuthology.orchestra.run.smithi120:> ip addr 2024-09-10T10:26:52.219 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:26:52.219 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:26:52.219 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:26:52.219 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:26 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f0 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:27 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f1 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8e brd ff:ff:ff:ff:ff:ff 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8f brd ff:ff:ff:ff:ff:ff 2024-09-10T10:26:52.220 INFO:teuthology.orchestra.run.smithi120.stdout: inet 172.21.15.120/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T10:26:52.221 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft 42769sec preferred_lft 42769sec 2024-09-10T10:26:52.221 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::ec4:7aff:fe8f:478f/64 scope link noprefixroute 2024-09-10T10:26:52.221 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:26:52.221 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T10:26:52.221 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:26:52.221 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T10:26:52.222 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add name ceph-brx type bridge 2024-09-10T10:26:52.222 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr flush dev ceph-brx 2024-09-10T10:26:52.222 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set ceph-brx up 2024-09-10T10:26:52.222 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T10:26:52.222 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T10:26:52.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:26:52.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:26:52.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:52.428 DEBUG:teuthology.orchestra.run.smithi120:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:26:52.453 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-09-10T10:26:52.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:52.454 DEBUG:teuthology.orchestra.run.smithi120:> route 2024-09-10T10:26:52.541 INFO:teuthology.orchestra.run.smithi120.stdout:Kernel IP routing table 2024-09-10T10:26:52.541 INFO:teuthology.orchestra.run.smithi120.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:26:52.541 INFO:teuthology.orchestra.run.smithi120.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:26:52.541 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:26:52.541 INFO:teuthology.orchestra.run.smithi120.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:26:52.541 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:26:52.542 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T10:26:52.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:26:52.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:26:52.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:26:52.542 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T10:26:52.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:26:52.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:26:52.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:52.799 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-09-10T10:26:52.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:52.815 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list-id 2024-09-10T10:26:52.871 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:26:52.871 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T10:26:52.871 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:26:52.872 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T10:26:52.872 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T10:26:52.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:26:52.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:26:52.990 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T10:26:52.990 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:26:52.990 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T10:26:52.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T10:26:52.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T10:26:52.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T10:26:52.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T10:26:52.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T10:26:52.990 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T10:26:53.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:26:53.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:26:53.113 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:26:53.113 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T10:26:53.113 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set brx.0 up 2024-09-10T10:26:53.113 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T10:26:53.114 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T10:26:53.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:26:53.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:26:53.191 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T10:26:53.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:26:53.191 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:26:53.206 INFO:teuthology.orchestra.run.smithi120.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T10:26:53.206 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:26:53.206 DEBUG:teuthology.orchestra.run.smithi120:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:26:53.261 DEBUG:teuthology.orchestra.run.smithi120:> sudo modprobe fuse 2024-09-10T10:26:53.326 DEBUG:teuthology.orchestra.run.smithi120:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/proc 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/sys 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/dev 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/kernel/security 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/shm 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/pts 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/run 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/fs/cgroup 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/fs/pstore 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/fs/bpf 2024-09-10T10:26:53.382 INFO:teuthology.orchestra.run.smithi120.stdout:/ 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/fs/selinux 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/mqueue 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/hugepages 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/kernel/debug 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/kernel/tracing 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/fs/fuse/connections 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/kernel/config 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T10:26:53.383 INFO:teuthology.orchestra.run.smithi120.stdout:/run/user/1000 2024-09-10T10:26:53.384 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/ceph 2024-09-10T10:26:53.384 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T10:26:53.384 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T10:26:53.384 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T10:26:53.384 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T10:26:53.384 INFO:teuthology.orchestra.run.smithi120.stdout:/run/netns 2024-09-10T10:26:53.384 INFO:teuthology.orchestra.run.smithi120.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:26:53.384 INFO:teuthology.orchestra.run.smithi120.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:26:53.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:53.384 DEBUG:teuthology.orchestra.run.smithi120:> ls /sys/fs/fuse/connections 2024-09-10T10:26:53.437 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T10:26:53.437 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T10:26:53.479 DEBUG:teuthology.orchestra.run.smithi120:> sudo modprobe fuse 2024-09-10T10:26:53.504 DEBUG:teuthology.orchestra.run.smithi120:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T10:26:53.561 INFO:teuthology.orchestra.run.smithi120.stdout:/proc 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/sys 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/dev 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/kernel/security 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/shm 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/pts 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/run 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/fs/cgroup 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/fs/pstore 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/fs/bpf 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/ 2024-09-10T10:26:53.562 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/fs/selinux 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/mqueue 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/dev/hugepages 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/kernel/debug 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/kernel/tracing 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/fs/fuse/connections 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/sys/kernel/config 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/run/user/1000 2024-09-10T10:26:53.563 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/ceph 2024-09-10T10:26:53.564 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T10:26:53.564 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T10:26:53.564 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T10:26:53.564 INFO:teuthology.orchestra.run.smithi120.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T10:26:53.564 INFO:teuthology.orchestra.run.smithi120.stdout:/run/netns 2024-09-10T10:26:53.564 INFO:teuthology.orchestra.run.smithi120.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:26:53.564 INFO:teuthology.orchestra.run.smithi120.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:26:53.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:53.564 DEBUG:teuthology.orchestra.run.smithi120:> ls /sys/fs/fuse/connections 2024-09-10T10:26:53.570 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi120.stderr:2024-09-10T10:26:53.569+0000 7f4613952480 -1 init, newargv = 0x555adaf61620 newargc=15 2024-09-10T10:26:53.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi120.stderr:ceph-fuse[41549]: starting ceph client 2024-09-10T10:26:53.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi120.stderr:ceph-fuse[41549]: starting fuse 2024-09-10T10:26:53.586 INFO:teuthology.orchestra.run.smithi120.stdout:41 2024-09-10T10:26:53.587 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T10:26:53.587 DEBUG:teuthology.orchestra.run.smithi120:> sudo stdin-killer -- python3 -c ' 2024-09-10T10:26:53.587 DEBUG:teuthology.orchestra.run.smithi120:> import glob 2024-09-10T10:26:53.587 DEBUG:teuthology.orchestra.run.smithi120:> import re 2024-09-10T10:26:53.587 DEBUG:teuthology.orchestra.run.smithi120:> import os 2024-09-10T10:26:53.587 DEBUG:teuthology.orchestra.run.smithi120:> import subprocess 2024-09-10T10:26:53.587 DEBUG:teuthology.orchestra.run.smithi120:> 2024-09-10T10:26:53.587 DEBUG:teuthology.orchestra.run.smithi120:> def _find_admin_socket(client_name): 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> files = glob.glob(asok_path) 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> # Given a non-glob path, it better be there 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> if "*" not in asok_path: 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> assert(len(files) == 1) 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> return files[0] 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> for f in files: 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T10:26:53.588 DEBUG:teuthology.orchestra.run.smithi120:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T10:26:53.589 DEBUG:teuthology.orchestra.run.smithi120:> contents = proc_f.read() 2024-09-10T10:26:53.589 DEBUG:teuthology.orchestra.run.smithi120:> if mountpoint in contents: 2024-09-10T10:26:53.589 DEBUG:teuthology.orchestra.run.smithi120:> return f 2024-09-10T10:26:53.589 DEBUG:teuthology.orchestra.run.smithi120:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T10:26:53.589 DEBUG:teuthology.orchestra.run.smithi120:> 2024-09-10T10:26:53.589 DEBUG:teuthology.orchestra.run.smithi120:> print(_find_admin_socket("client.admin")) 2024-09-10T10:26:53.589 DEBUG:teuthology.orchestra.run.smithi120:> ' 2024-09-10T10:26:53.706 INFO:teuthology.orchestra.run.smithi120.stdout:/var/run/ceph/ceph-client.admin.41549.asok 2024-09-10T10:26:53.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:26:53.718 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41549.asok 2024-09-10T10:26:53.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:53.718 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41549.asok status 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout: "metadata": { 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout: "entity_id": "admin", 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout: "hostname": "smithi120", 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout: "pid": "41549", 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout: "root": "/" 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:26:53.854 INFO:teuthology.orchestra.run.smithi120.stdout: "dentry_count": 0, 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "dentry_pinned_count": 0, 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 4431, 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "inst": { 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "name": { 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "client", 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "num": 4431 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": { 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "192.168.0.1:0", 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 858982493 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": { 2024-09-10T10:26:53.855 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "192.168.0.1:0", 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 858982493 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: "inst_str": "client.4431 192.168.0.1:0/858982493", 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: "addr_str": "192.168.0.1:0/858982493", 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: "inode_count": 1, 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_epoch": 5, 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_epoch": 26, 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_epoch_barrier": 0, 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklisted": false, 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs" 2024-09-10T10:26:53.856 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:26:53.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:53.862 DEBUG:teuthology.orchestra.run.smithi120:> stat --file-system '--printf=%T 2024-09-10T10:26:53.862 DEBUG:teuthology.orchestra.run.smithi120:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:26:53.878 INFO:teuthology.orchestra.run.smithi120.stdout:fuseblk 2024-09-10T10:26:53.878 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:26:53.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:53.878 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:26:53.944 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T10:26:54.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:26:54.054 INFO:teuthology.orchestra.run.smithi120.stdout:# file: . 2024-09-10T10:26:54.054 INFO:teuthology.orchestra.run.smithi120.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T10:26:54.054 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:26:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:26:54.060 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:54.060 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:26:54.076 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:26:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.214+0000 7fe383b4b640 1 Processor -- start 2024-09-10T10:26:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.214+0000 7fe383b4b640 1 -- start start 2024-09-10T10:26:54.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe383b4b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c154d20 0x7fe37c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe383b4b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c0c99f0 con 0x7fe37c0c94d0 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe383b4b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c0c9c90 con 0x7fe37c15eb60 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe383b4b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c155100 con 0x7fe37c154d20 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3810e6640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fe37c0c94d0 0x7fe37c0715a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:46264/0 (socket says 172.21.15.120:46264) 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3810e6640 1 -- 172.21.15.120:0/1629249293 learned_addr learned my addr 172.21.15.120:0/1629249293 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3818e7640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c154d20 0x7fe37c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374354808 0 0) 0x7fe37c0c99f0 con 0x7fe37c0c94d0 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe364003520 con 0x7fe37c0c94d0 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2776318988 0 0) 0x7fe37c0c9c90 con 0x7fe37c15eb60 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe37c0c99f0 con 0x7fe37c15eb60 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2249479758 0 0) 0x7fe364003520 con 0x7fe37c0c94d0 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c154d20 msgr2=0x7fe37c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:54.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c154d20 0x7fe37c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 >> v1:172.21.15.134:6789/0 conn(0x7fe37c15eb60 legacy=0x7fe37c1652b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c15e0c0 con 0x7fe37c0c94d0 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7fe370001670 con 0x7fe37c0c94d0 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 >> v1:172.21.15.120:6789/0 conn(0x7fe37c0c94d0 legacy=0x7fe37c0715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe364003b70 0x7fe364003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c154d20 0x7fe364004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe364004d40 0x7fe3640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c155100 con 0x7fe364004d40 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe36400ba70 con 0x7fe364003b70 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe36400bc20 con 0x7fe37c154d20 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3820e8640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe364004d40 0x7fe3640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.215+0000 7fe3818e7640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe364003b70 0x7fe364003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.216+0000 7fe3810e6640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c154d20 0x7fe364004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.216+0000 7fe3820e8640 1 -- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c154d20 msgr2=0x7fe364004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.216+0000 7fe3820e8640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c154d20 0x7fe364004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.216+0000 7fe3820e8640 1 -- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe364003b70 msgr2=0x7fe364003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.216+0000 7fe3820e8640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe364003b70 0x7fe364003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.216+0000 7fe3820e8640 1 -- 172.21.15.120:0/1629249293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe364003520 con 0x7fe364004d40 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.216+0000 7fe3818e7640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe364003b70 0x7fe364003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.216+0000 7fe3820e8640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe364004d40 0x7fe3640051c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe36c001fd0 tx=0x7fe364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb53c110023e43d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.217+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c001bf0 con 0x7fe364004d40 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.217+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c003b70 con 0x7fe364004d40 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.217+0000 7fe3808e5640 1 -- 172.21.15.120:0/1629249293 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0050d0 con 0x7fe364004d40 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.217+0000 7fe383b4b640 1 -- 172.21.15.120:0/1629249293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe364004d40 msgr2=0x7fe3640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.217+0000 7fe383b4b640 1 --2- 172.21.15.120:0/1629249293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe364004d40 0x7fe3640051c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe36c001fd0 tx=0x7fe364005830 comp rx=0 tx=0).stop 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.217+0000 7fe3820e8640 1 -- 172.21.15.120:0/1629249293 reap_dead start 2024-09-10T10:26:54.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.217+0000 7fe383b4b640 1 -- 172.21.15.120:0/1629249293 shutdown_connections 2024-09-10T10:26:54.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.217+0000 7fe383b4b640 1 -- 172.21.15.120:0/1629249293 >> 172.21.15.120:0/1629249293 conn(0x7fe37c06bc10 msgr2=0x7fe37c06fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:54.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.217+0000 7fe383b4b640 1 -- 172.21.15.120:0/1629249293 shutdown_connections 2024-09-10T10:26:54.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.217+0000 7fe383b4b640 1 -- 172.21.15.120:0/1629249293 wait complete. 2024-09-10T10:26:54.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.218+0000 7fe383b4b640 1 Processor -- start 2024-09-10T10:26:54.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.218+0000 7fe383b4b640 1 -- start start 2024-09-10T10:26:54.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe383b4b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c1535f0 0x7fe37c1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe383b4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c1539d0 0x7fe37c1f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe383b4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c1f15c0 0x7fe37c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe383b4b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0c5460 con 0x7fe37c1f15c0 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe383b4b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0ca1f0 con 0x7fe37c1539d0 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe383b4b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c15e6d0 con 0x7fe37c1535f0 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3820e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c1f15c0 0x7fe37c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3820e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c1f15c0 0x7fe37c15c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40030/0 (socket says 172.21.15.120:40030) 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3820e8640 1 -- 172.21.15.120:0/2995354395 learned_addr learned my addr 172.21.15.120:0/2995354395 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3818e7640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c1535f0 0x7fe37c1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3810e6640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c1539d0 0x7fe37c1f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3820e8640 1 -- 172.21.15.120:0/2995354395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c1535f0 msgr2=0x7fe37c1587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3820e8640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c1535f0 0x7fe37c1587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3820e8640 1 -- 172.21.15.120:0/2995354395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c1539d0 msgr2=0x7fe37c1f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3820e8640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c1539d0 0x7fe37c1f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3820e8640 1 -- 172.21.15.120:0/2995354395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c1e5b10 con 0x7fe37c1f15c0 2024-09-10T10:26:54.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.219+0000 7fe3810e6640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c1539d0 0x7fe37c1f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.220+0000 7fe3820e8640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c1f15c0 0x7fe37c15c1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0020f0 tx=0x7fe36c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.220+0000 7fe37a7fc640 1 -- 172.21.15.120:0/2995354395 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374009690 con 0x7fe37c1f15c0 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.220+0000 7fe37a7fc640 1 -- 172.21.15.120:0/2995354395 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c003970 con 0x7fe37c1f15c0 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.220+0000 7fe383b4b640 1 -- 172.21.15.120:0/2995354395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c1f48c0 con 0x7fe37c1f15c0 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.220+0000 7fe37a7fc640 1 -- 172.21.15.120:0/2995354395 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c001bf0 con 0x7fe37c1f15c0 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.220+0000 7fe383b4b640 1 -- 172.21.15.120:0/2995354395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1f08a0 con 0x7fe37c1f15c0 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.221+0000 7fe383b4b640 1 -- 172.21.15.120:0/2995354395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c15e360 con 0x7fe37c1f15c0 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.221+0000 7fe37a7fc640 1 -- 172.21.15.120:0/2995354395 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe36c045020 con 0x7fe37c1f15c0 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.222+0000 7fe37a7fc640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe36806be60 0x7fe36806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.222+0000 7fe37a7fc640 1 -- 172.21.15.120:0/2995354395 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe36c0aaf30 con 0x7fe37c1f15c0 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.222+0000 7fe3818e7640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe36806be60 0x7fe36806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.222+0000 7fe3818e7640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe36806be60 0x7fe36806e320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe374009d60 tx=0x7fe374009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:54.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.226+0000 7fe37a7fc640 1 -- 172.21.15.120:0/2995354395 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe36c079780 con 0x7fe37c1f15c0 2024-09-10T10:26:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.401+0000 7fe383b4b640 1 -- 172.21.15.120:0/2995354395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe37c1e6670 con 0x7fe37c1f15c0 2024-09-10T10:26:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.402+0000 7fe37a7fc640 1 -- 172.21.15.120:0/2995354395 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fe36c003b40 con 0x7fe37c1f15c0 2024-09-10T10:26:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:listed 0 entries 2024-09-10T10:26:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 -- 172.21.15.120:0/2995354395 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe36806be60 msgr2=0x7fe36806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe36806be60 0x7fe36806e320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe374009d60 tx=0x7fe374009190 comp rx=0 tx=0).stop 2024-09-10T10:26:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 -- 172.21.15.120:0/2995354395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c1f15c0 msgr2=0x7fe37c15c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c1f15c0 0x7fe37c15c1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0020f0 tx=0x7fe36c001650 comp rx=0 tx=0).stop 2024-09-10T10:26:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 -- 172.21.15.120:0/2995354395 shutdown_connections 2024-09-10T10:26:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe36806be60 0x7fe36806e320 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe37c1f15c0 0x7fe37c15c1e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c1539d0 0x7fe37c1f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 --2- 172.21.15.120:0/2995354395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe37c1535f0 0x7fe37c1587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 -- 172.21.15.120:0/2995354395 >> 172.21.15.120:0/2995354395 conn(0x7fe37c06bc10 msgr2=0x7fe37c0cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:54.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 -- 172.21.15.120:0/2995354395 shutdown_connections 2024-09-10T10:26:54.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.404+0000 7fe383b4b640 1 -- 172.21.15.120:0/2995354395 wait complete. 2024-09-10T10:26:54.415 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:26:54.415 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:26:54.431 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:26:54.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.568+0000 7f9268dc4640 1 Processor -- start 2024-09-10T10:26:54.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.568+0000 7f9268dc4640 1 -- start start 2024-09-10T10:26:54.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.569+0000 7f9268dc4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9264152920 0x7f926406d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.569+0000 7f9268dc4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92640c7a60 con 0x7f9264152d00 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.569+0000 7f9268dc4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92640c7d00 con 0x7f92640c8c20 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.569+0000 7f9268dc4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f92640cd230 con 0x7f9264152920 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.569+0000 7f9261d9b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f9264152d00 0x7f926406bdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:46272/0 (socket says 172.21.15.120:46272) 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.569+0000 7f9261d9b640 1 -- 172.21.15.120:0/3262791462 learned_addr learned my addr 172.21.15.120:0/3262791462 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926259c640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9264152920 0x7f926406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2043690166 0 0) 0x7f92640c7a60 con 0x7f9264152d00 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f924c003520 con 0x7f9264152d00 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 108322420 0 0) 0x7f924c003520 con 0x7f9264152d00 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9264152920 msgr2=0x7f926406d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9264152920 0x7f926406d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 >> v1:172.21.15.134:6789/0 conn(0x7f92640c8c20 legacy=0x7f926406ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92640cdfd0 con 0x7f9264152d00 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f9258001670 con 0x7f9264152d00 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 >> v1:172.21.15.120:6789/0 conn(0x7f9264152d00 legacy=0x7f926406bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f924c0039e0 0x7f924c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f924c004390 0x7f924c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9264152920 0x7f924c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92640cd230 con 0x7f924c004390 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f924c00b9c0 con 0x7f924c0039e0 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f924c00bbc0 con 0x7f9264152920 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f9261d9b640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f924c004390 0x7f924c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f926259c640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f924c0039e0 0x7f924c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f9262d9d640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9264152920 0x7f924c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f9261d9b640 1 -- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9264152920 msgr2=0x7f924c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:54.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f9261d9b640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9264152920 0x7f924c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f9261d9b640 1 -- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f924c0039e0 msgr2=0x7f924c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:26:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f9261d9b640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f924c0039e0 0x7f924c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.570+0000 7f9261d9b640 1 -- 172.21.15.120:0/3262791462 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c003520 con 0x7f924c004390 2024-09-10T10:26:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.571+0000 7f9261d9b640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f924c004390 0x7f924c004860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9264071a20 tx=0x7f924c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a6a7563fcdf957 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.571+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258007ce0 con 0x7f924c004390 2024-09-10T10:26:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.571+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9258005af0 con 0x7f924c004390 2024-09-10T10:26:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.571+0000 7f926159a640 1 -- 172.21.15.120:0/3262791462 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258007070 con 0x7f924c004390 2024-09-10T10:26:54.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.572+0000 7f9268dc4640 1 -- 172.21.15.120:0/3262791462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f924c004390 msgr2=0x7f924c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.572+0000 7f9268dc4640 1 --2- 172.21.15.120:0/3262791462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f924c004390 0x7f924c004860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9264071a20 tx=0x7f924c005830 comp rx=0 tx=0).stop 2024-09-10T10:26:54.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.572+0000 7f9262d9d640 1 -- 172.21.15.120:0/3262791462 reap_dead start 2024-09-10T10:26:54.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.572+0000 7f9268dc4640 1 -- 172.21.15.120:0/3262791462 shutdown_connections 2024-09-10T10:26:54.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.572+0000 7f9268dc4640 1 -- 172.21.15.120:0/3262791462 >> 172.21.15.120:0/3262791462 conn(0x7f9264067950 msgr2=0x7f9264067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:54.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.572+0000 7f9268dc4640 1 -- 172.21.15.120:0/3262791462 shutdown_connections 2024-09-10T10:26:54.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.572+0000 7f9268dc4640 1 -- 172.21.15.120:0/3262791462 wait complete. 2024-09-10T10:26:54.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.573+0000 7f9268dc4640 1 Processor -- start 2024-09-10T10:26:54.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.573+0000 7f9268dc4640 1 -- start start 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9268dc4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f924c004390 0x7f926414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9268dc4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641564e0 0x7f92641e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9268dc4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264156940 0x7f92641e42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9268dc4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92640cdd40 con 0x7f9264156940 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9268dc4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92640cdbc0 con 0x7f92641564e0 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9268dc4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f926406f170 con 0x7f924c004390 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9262d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264156940 0x7f92641e42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9262d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264156940 0x7f92641e42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40050/0 (socket says 172.21.15.120:40050) 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9262d9d640 1 -- 172.21.15.120:0/1739522653 learned_addr learned my addr 172.21.15.120:0/1739522653 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9261d9b640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641564e0 0x7f92641e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f926259c640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f924c004390 0x7f926414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9262d9d640 1 -- 172.21.15.120:0/1739522653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f924c004390 msgr2=0x7f926414b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9262d9d640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f924c004390 0x7f926414b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9262d9d640 1 -- 172.21.15.120:0/1739522653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641564e0 msgr2=0x7f92641e3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9262d9d640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641564e0 0x7f92641e3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f9262d9d640 1 -- 172.21.15.120:0/1739522653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92641f15d0 con 0x7f9264156940 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.574+0000 7f926259c640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f924c004390 0x7f926414b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.575+0000 7f9262d9d640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264156940 0x7f92641e42e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92640c9a80 tx=0x7f925c00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.575+0000 7f9252ffd640 1 -- 172.21.15.120:0/1739522653 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c019070 con 0x7f9264156940 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.575+0000 7f9252ffd640 1 -- 172.21.15.120:0/1739522653 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f925c006e20 con 0x7f9264156940 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.575+0000 7f9252ffd640 1 -- 172.21.15.120:0/1739522653 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c003b90 con 0x7f9264156940 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.575+0000 7f9268dc4640 1 -- 172.21.15.120:0/1739522653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92641ee650 con 0x7f9264156940 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.575+0000 7f9268dc4640 1 -- 172.21.15.120:0/1739522653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92641e2fe0 con 0x7f9264156940 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.576+0000 7f9268dc4640 1 -- 172.21.15.120:0/1739522653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9264151d70 con 0x7f9264156940 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.576+0000 7f9252ffd640 1 -- 172.21.15.120:0/1739522653 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f925c01c070 con 0x7f9264156940 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.576+0000 7f9252ffd640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f924406be60 0x7f924406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:26:54.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.577+0000 7f9252ffd640 1 -- 172.21.15.120:0/1739522653 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f925c08a480 con 0x7f9264156940 2024-09-10T10:26:54.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.577+0000 7f926259c640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f924406be60 0x7f924406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:26:54.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.577+0000 7f926259c640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f924406be60 0x7f924406e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9254009bc0 tx=0x7f9254008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:26:54.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.581+0000 7f9252ffd640 1 -- 172.21.15.120:0/1739522653 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9264151d70 con 0x7f9264156940 2024-09-10T10:26:54.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.758+0000 7f9268dc4640 1 -- 172.21.15.120:0/1739522653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f92640c90d0 con 0x7f9264156940 2024-09-10T10:26:54.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.759+0000 7f9252ffd640 1 -- 172.21.15.120:0/1739522653 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f925c058410 con 0x7f9264156940 2024-09-10T10:26:54.760 INFO:teuthology.orchestra.run.smithi120.stderr:listed 0 entries 2024-09-10T10:26:54.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 -- 172.21.15.120:0/1739522653 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f924406be60 msgr2=0x7f924406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f924406be60 0x7f924406e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9254009bc0 tx=0x7f9254008040 comp rx=0 tx=0).stop 2024-09-10T10:26:54.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 -- 172.21.15.120:0/1739522653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264156940 msgr2=0x7f92641e42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:26:54.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264156940 0x7f92641e42e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92640c9a80 tx=0x7f925c00ff70 comp rx=0 tx=0).stop 2024-09-10T10:26:54.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 -- 172.21.15.120:0/1739522653 shutdown_connections 2024-09-10T10:26:54.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f924406be60 0x7f924406e320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264156940 0x7f92641e42e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92641564e0 0x7f92641e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 --2- 172.21.15.120:0/1739522653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f924c004390 0x7f926414b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:26:54.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 -- 172.21.15.120:0/1739522653 >> 172.21.15.120:0/1739522653 conn(0x7f9264067950 msgr2=0x7f9264070240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:26:54.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 -- 172.21.15.120:0/1739522653 shutdown_connections 2024-09-10T10:26:54.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:26:54.761+0000 7f9268dc4640 1 -- 172.21.15.120:0/1739522653 wait complete. 2024-09-10T10:26:54.773 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi120.front.sepia.ceph.com... 2024-09-10T10:26:54.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:26:54.773 DEBUG:teuthology.orchestra.run.smithi120:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:26:54.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi120.stderr:ceph-fuse[41549]: fuse finished with error 0 and tester_r 0 2024-09-10T10:26:54.798 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:26:56.765 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-09-10T10:26:56.763+0000 7f890ecec640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:27:00.900 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:27:00.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:00.901 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T10:27:00.931 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:27:01.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.080+0000 7f95d8c3c640 1 Processor -- start 2024-09-10T10:27:01.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.080+0000 7f95d8c3c640 1 -- start start 2024-09-10T10:27:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.081+0000 7f95d8c3c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d415eb60 0x7f95d415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.081+0000 7f95d8c3c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d40c99f0 con 0x7f95d40c94d0 2024-09-10T10:27:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.081+0000 7f95d8c3c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d40c9c90 con 0x7f95d4154d20 2024-09-10T10:27:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.081+0000 7f95d8c3c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f95d4155100 con 0x7f95d415eb60 2024-09-10T10:27:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.081+0000 7f95d1d9b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f95d40c94d0 0x7f95d41632a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:48454/0 (socket says 172.21.15.120:48454) 2024-09-10T10:27:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.081+0000 7f95d1d9b640 1 -- 172.21.15.120:0/1949057864 learned_addr learned my addr 172.21.15.120:0/1949057864 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.081+0000 7f95d2d9d640 1 --2- 172.21.15.120:0/1949057864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d415eb60 0x7f95d415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.081+0000 7f95d159a640 1 -- 172.21.15.120:0/1949057864 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1169825885 0 0) 0x7f95d40c99f0 con 0x7f95d40c94d0 2024-09-10T10:27:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.081+0000 7f95d159a640 1 -- 172.21.15.120:0/1949057864 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95bc003520 con 0x7f95d40c94d0 2024-09-10T10:27:01.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.082+0000 7f95d159a640 1 -- 172.21.15.120:0/1949057864 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2981174751 0 0) 0x7f95d40c9c90 con 0x7f95d4154d20 2024-09-10T10:27:01.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.082+0000 7f95d159a640 1 -- 172.21.15.120:0/1949057864 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95d40c99f0 con 0x7f95d4154d20 2024-09-10T10:27:01.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.082+0000 7f95d2d9d640 1 -- 172.21.15.120:0/1949057864 >> v1:172.21.15.134:6789/0 conn(0x7f95d4154d20 legacy=0x7f95d40715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.082+0000 7f95d2d9d640 1 -- 172.21.15.120:0/1949057864 >> v1:172.21.15.120:6789/0 conn(0x7f95d40c94d0 legacy=0x7f95d41632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.082+0000 7f95d2d9d640 1 -- 172.21.15.120:0/1949057864 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d415e0c0 con 0x7f95d415eb60 2024-09-10T10:27:01.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.082+0000 7f95d2d9d640 1 --2- 172.21.15.120:0/1949057864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d415eb60 0x7f95d415a930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009dc0 tx=0x7f95cc02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=497fddf72d8ce063 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:01.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.083+0000 7f95d159a640 1 -- 172.21.15.120:0/1949057864 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc007740 con 0x7f95d415eb60 2024-09-10T10:27:01.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.083+0000 7f95d159a640 1 -- 172.21.15.120:0/1949057864 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc0078e0 con 0x7f95d415eb60 2024-09-10T10:27:01.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.083+0000 7f95d159a640 1 -- 172.21.15.120:0/1949057864 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc007c90 con 0x7f95d415eb60 2024-09-10T10:27:01.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.083+0000 7f95d8c3c640 1 -- 172.21.15.120:0/1949057864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d415eb60 msgr2=0x7f95d415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.083+0000 7f95d8c3c640 1 --2- 172.21.15.120:0/1949057864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d415eb60 0x7f95d415a930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009dc0 tx=0x7f95cc02a6d0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.083+0000 7f95d8c3c640 1 -- 172.21.15.120:0/1949057864 shutdown_connections 2024-09-10T10:27:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.083+0000 7f95d8c3c640 1 --2- 172.21.15.120:0/1949057864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d415eb60 0x7f95d415a930 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.083+0000 7f95d8c3c640 1 -- 172.21.15.120:0/1949057864 >> 172.21.15.120:0/1949057864 conn(0x7f95d406bc10 msgr2=0x7f95d406fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.083+0000 7f95d8c3c640 1 -- 172.21.15.120:0/1949057864 shutdown_connections 2024-09-10T10:27:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.083+0000 7f95d8c3c640 1 -- 172.21.15.120:0/1949057864 wait complete. 2024-09-10T10:27:01.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.085+0000 7f95d8c3c640 1 Processor -- start 2024-09-10T10:27:01.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.085+0000 7f95d8c3c640 1 -- start start 2024-09-10T10:27:01.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.085+0000 7f95d8c3c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d40c94d0 0x7f95d414c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.085+0000 7f95d8c3c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d4154d20 0x7f95d41595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.085+0000 7f95d8c3c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d415eb60 0x7f95d4159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.085+0000 7f95d8c3c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f95d40ca1f0 con 0x7f95d415eb60 2024-09-10T10:27:01.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.085+0000 7f95d8c3c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95d40c5460 con 0x7f95d4154d20 2024-09-10T10:27:01.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.085+0000 7f95d8c3c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f95d414dbe0 con 0x7f95d40c94d0 2024-09-10T10:27:01.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d415eb60 0x7f95d4159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d415eb60 0x7f95d4159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34848/0 (socket says 172.21.15.120:34848) 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d2d9d640 1 -- 172.21.15.120:0/3851023613 learned_addr learned my addr 172.21.15.120:0/3851023613 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d1d9b640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d4154d20 0x7f95d41595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d259c640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d40c94d0 0x7f95d414c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d2d9d640 1 -- 172.21.15.120:0/3851023613 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d40c94d0 msgr2=0x7f95d414c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d2d9d640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d40c94d0 0x7f95d414c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d2d9d640 1 -- 172.21.15.120:0/3851023613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d4154d20 msgr2=0x7f95d41595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d2d9d640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d4154d20 0x7f95d41595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d2d9d640 1 -- 172.21.15.120:0/3851023613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d414c790 con 0x7f95d415eb60 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d2d9d640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d415eb60 0x7f95d4159b10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95cc02acc0 tx=0x7f95cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95c2ffd640 1 -- 172.21.15.120:0/3851023613 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc03f070 con 0x7f95d415eb60 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95c2ffd640 1 -- 172.21.15.120:0/3851023613 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc031c40 con 0x7f95d415eb60 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d8c3c640 1 -- 172.21.15.120:0/3851023613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d414de10 con 0x7f95d415eb60 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95c2ffd640 1 -- 172.21.15.120:0/3851023613 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc009450 con 0x7f95d415eb60 2024-09-10T10:27:01.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.086+0000 7f95d8c3c640 1 -- 172.21.15.120:0/3851023613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d41f4a30 con 0x7f95d415eb60 2024-09-10T10:27:01.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.087+0000 7f95d8c3c640 1 -- 172.21.15.120:0/3851023613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d40c4e70 con 0x7f95d415eb60 2024-09-10T10:27:01.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.088+0000 7f95c2ffd640 1 -- 172.21.15.120:0/3851023613 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95cc007790 con 0x7f95d415eb60 2024-09-10T10:27:01.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.088+0000 7f95c2ffd640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95b406be60 0x7f95b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.088+0000 7f95d259c640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95b406be60 0x7f95b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.089+0000 7f95c2ffd640 1 -- 172.21.15.120:0/3851023613 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f95cc0ac630 con 0x7f95d415eb60 2024-09-10T10:27:01.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.089+0000 7f95d259c640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95b406be60 0x7f95b406e320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f95c4007fd0 tx=0x7f95c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:01.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.093+0000 7f95c2ffd640 1 -- 172.21.15.120:0/3851023613 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f95d40c4e70 con 0x7f95d415eb60 2024-09-10T10:27:01.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.259+0000 7f95d8c3c640 1 -- 172.21.15.120:0/3851023613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95d415e360 con 0x7f95d415eb60 2024-09-10T10:27:01.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.260+0000 7f95c2ffd640 1 -- 172.21.15.120:0/3851023613 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f95cc07a6e0 con 0x7f95d415eb60 2024-09-10T10:27:01.262 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:01.262 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:47.823215+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:27:01.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.262+0000 7f95d8c3c640 1 -- 172.21.15.120:0/3851023613 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95b406be60 msgr2=0x7f95b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.262+0000 7f95d8c3c640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95b406be60 0x7f95b406e320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f95c4007fd0 tx=0x7f95c4009040 comp rx=0 tx=0).stop 2024-09-10T10:27:01.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.262+0000 7f95d8c3c640 1 -- 172.21.15.120:0/3851023613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d415eb60 msgr2=0x7f95d4159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.262+0000 7f95d8c3c640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d415eb60 0x7f95d4159b10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95cc02acc0 tx=0x7f95cc004a40 comp rx=0 tx=0).stop 2024-09-10T10:27:01.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.262+0000 7f95d8c3c640 1 -- 172.21.15.120:0/3851023613 shutdown_connections 2024-09-10T10:27:01.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.262+0000 7f95d8c3c640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95b406be60 0x7f95b406e320 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.262+0000 7f95d8c3c640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d415eb60 0x7f95d4159b10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.262+0000 7f95d8c3c640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d4154d20 0x7f95d41595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.262+0000 7f95d8c3c640 1 --2- 172.21.15.120:0/3851023613 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d40c94d0 0x7f95d414c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.262+0000 7f95d8c3c640 1 -- 172.21.15.120:0/3851023613 >> 172.21.15.120:0/3851023613 conn(0x7f95d406bc10 msgr2=0x7f95d415d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:01.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.263+0000 7f95d8c3c640 1 -- 172.21.15.120:0/3851023613 shutdown_connections 2024-09-10T10:27:01.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.263+0000 7f95d8c3c640 1 -- 172.21.15.120:0/3851023613 wait complete. 2024-09-10T10:27:01.278 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:27:01.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.374+0000 7f56a79ed640 1 Processor -- start 2024-09-10T10:27:01.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.374+0000 7f56a79ed640 1 -- start start 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.374+0000 7f56a79ed640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f56a00cb410 0x7f56a00cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.374+0000 7f56a79ed640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a015b1a0 con 0x7f56a00c7770 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.375+0000 7f56a79ed640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a015b420 con 0x7f56a00c6980 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.375+0000 7f56a79ed640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f56a015b9e0 con 0x7f56a00cb410 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.375+0000 7f56a5f8a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f56a00cb410 0x7f56a00cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.375+0000 7f56a4f88640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f56a00c7770 0x7f56a00cbfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:48462/0 (socket says 172.21.15.120:48462) 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.375+0000 7f56a5f8a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f56a00cb410 0x7f56a00cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34850/0 (socket says 172.21.15.120:34850) 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.375+0000 7f56a4f88640 1 -- 172.21.15.120:0/2291257147 learned_addr learned my addr 172.21.15.120:0/2291257147 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.375+0000 7f56a5f8a640 1 -- 172.21.15.120:0/2291257147 >> v1:172.21.15.134:6789/0 conn(0x7f56a00c6980 legacy=0x7f56a015aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.375+0000 7f56a5f8a640 1 -- 172.21.15.120:0/2291257147 >> v1:172.21.15.120:6789/0 conn(0x7f56a00c7770 legacy=0x7f56a00cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.375+0000 7f56a5f8a640 1 -- 172.21.15.120:0/2291257147 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a015b700 con 0x7f56a00cb410 2024-09-10T10:27:01.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.375+0000 7f5693fff640 1 -- 172.21.15.120:0/2291257147 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3292948422 0 0) 0x7f569c001110 con 0x7f56a00c7770 2024-09-10T10:27:01.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f56a5f8a640 1 --2- 172.21.15.120:0/2291257147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00cb410 0x7f56a00cd730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5698009d90 tx=0x7f569802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184ed6585e7999ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:01.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f5693fff640 1 -- 172.21.15.120:0/2291257147 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698007740 con 0x7f56a00cb410 2024-09-10T10:27:01.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f5693fff640 1 -- 172.21.15.120:0/2291257147 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56980078e0 con 0x7f56a00cb410 2024-09-10T10:27:01.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f5693fff640 1 -- 172.21.15.120:0/2291257147 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698007cb0 con 0x7f56a00cb410 2024-09-10T10:27:01.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f56a79ed640 1 -- 172.21.15.120:0/2291257147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00cb410 msgr2=0x7f56a00cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f56a79ed640 1 --2- 172.21.15.120:0/2291257147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00cb410 0x7f56a00cd730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5698009d90 tx=0x7f569802a660 comp rx=0 tx=0).stop 2024-09-10T10:27:01.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f56a79ed640 1 -- 172.21.15.120:0/2291257147 shutdown_connections 2024-09-10T10:27:01.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f56a79ed640 1 --2- 172.21.15.120:0/2291257147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00cb410 0x7f56a00cd730 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f56a79ed640 1 -- 172.21.15.120:0/2291257147 >> 172.21.15.120:0/2291257147 conn(0x7f56a006cb70 msgr2=0x7f56a006e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:01.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f56a79ed640 1 -- 172.21.15.120:0/2291257147 shutdown_connections 2024-09-10T10:27:01.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.376+0000 7f56a79ed640 1 -- 172.21.15.120:0/2291257147 wait complete. 2024-09-10T10:27:01.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.377+0000 7f56a79ed640 1 Processor -- start 2024-09-10T10:27:01.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.378+0000 7f56a79ed640 1 -- start start 2024-09-10T10:27:01.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.378+0000 7f56a79ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00c6980 0x7f56a0157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.378+0000 7f56a79ed640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56a00c7770 0x7f56a01e0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.378+0000 7f56a79ed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56a00cb410 0x7f56a01f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.378+0000 7f56a79ed640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56a00cd210 con 0x7f56a00c6980 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.378+0000 7f56a79ed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56a00ccf30 con 0x7f56a00cb410 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.378+0000 7f56a79ed640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f56a00d0fe0 con 0x7f56a00c7770 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.378+0000 7f56a5789640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00c6980 0x7f56a0157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.378+0000 7f56a5789640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00c6980 0x7f56a0157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34852/0 (socket says 172.21.15.120:34852) 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.378+0000 7f56a5789640 1 -- 172.21.15.120:0/1124833999 learned_addr learned my addr 172.21.15.120:0/1124833999 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f56a5789640 1 -- 172.21.15.120:0/1124833999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56a00c7770 msgr2=0x7f56a01e0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f56a5789640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56a00c7770 0x7f56a01e0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f56a5789640 1 -- 172.21.15.120:0/1124833999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56a00cb410 msgr2=0x7f56a01f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f56a5f8a640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56a00cb410 0x7f56a01f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f56a5789640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56a00cb410 0x7f56a01f3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f56a5789640 1 -- 172.21.15.120:0/1124833999 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a00c5500 con 0x7f56a00c6980 2024-09-10T10:27:01.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f56a5789640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00c6980 0x7f56a0157840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f569400c450 tx=0x7f569400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:01.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f5691ffb640 1 -- 172.21.15.120:0/1124833999 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940052d0 con 0x7f56a00c6980 2024-09-10T10:27:01.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f5691ffb640 1 -- 172.21.15.120:0/1124833999 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5694005470 con 0x7f56a00c6980 2024-09-10T10:27:01.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f56a79ed640 1 -- 172.21.15.120:0/1124833999 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a00c57a0 con 0x7f56a00c6980 2024-09-10T10:27:01.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f5691ffb640 1 -- 172.21.15.120:0/1124833999 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694012870 con 0x7f56a00c6980 2024-09-10T10:27:01.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.379+0000 7f56a79ed640 1 -- 172.21.15.120:0/1124833999 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01f0ce0 con 0x7f56a00c6980 2024-09-10T10:27:01.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.380+0000 7f56a79ed640 1 -- 172.21.15.120:0/1124833999 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a00d1220 con 0x7f56a00c6980 2024-09-10T10:27:01.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.381+0000 7f5691ffb640 1 -- 172.21.15.120:0/1124833999 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5694012a10 con 0x7f56a00c6980 2024-09-10T10:27:01.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.381+0000 7f5691ffb640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f568406be30 0x7f568406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.381+0000 7f5691ffb640 1 -- 172.21.15.120:0/1124833999 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f569408ad90 con 0x7f56a00c6980 2024-09-10T10:27:01.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.381+0000 7f56a4f88640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f568406be30 0x7f568406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.382+0000 7f56a4f88640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f568406be30 0x7f568406e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c001250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:01.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.385+0000 7f5691ffb640 1 -- 172.21.15.120:0/1124833999 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f56a00d1220 con 0x7f56a00c6980 2024-09-10T10:27:01.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.552+0000 7f56a79ed640 1 -- 172.21.15.120:0/1124833999 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56a015bbf0 con 0x7f56a00c6980 2024-09-10T10:27:01.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.553+0000 7f5691ffb640 1 -- 172.21.15.120:0/1124833999 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f56a015bbf0 con 0x7f56a00c6980 2024-09-10T10:27:01.555 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:01.555 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:47.823215+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:27:01.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 -- 172.21.15.120:0/1124833999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f568406be30 msgr2=0x7f568406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f568406be30 0x7f568406e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c001250 comp rx=0 tx=0).stop 2024-09-10T10:27:01.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 -- 172.21.15.120:0/1124833999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00c6980 msgr2=0x7f56a0157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00c6980 0x7f56a0157840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f569400c450 tx=0x7f569400c910 comp rx=0 tx=0).stop 2024-09-10T10:27:01.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 -- 172.21.15.120:0/1124833999 shutdown_connections 2024-09-10T10:27:01.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f568406be30 0x7f568406e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56a00cb410 0x7f56a01f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56a00c7770 0x7f56a01e0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 --2- 172.21.15.120:0/1124833999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56a00c6980 0x7f56a0157840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 -- 172.21.15.120:0/1124833999 >> 172.21.15.120:0/1124833999 conn(0x7f56a006cb70 msgr2=0x7f56a00c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:01.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 -- 172.21.15.120:0/1124833999 shutdown_connections 2024-09-10T10:27:01.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.555+0000 7f56a79ed640 1 -- 172.21.15.120:0/1124833999 wait complete. 2024-09-10T10:27:01.568 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:27:01.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.668+0000 7f1ba3082640 1 Processor -- start 2024-09-10T10:27:01.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.668+0000 7f1ba3082640 1 -- start start 2024-09-10T10:27:01.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.669+0000 7f1ba3082640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1b9c0cc0a0 0x7f1b9c0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.669+0000 7f1ba3082640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9c15ba20 con 0x7f1b9c0d0630 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.669+0000 7f1ba3082640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9c15aea0 con 0x7f1b9c15ed70 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.669+0000 7f1ba3082640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9c15e1d0 con 0x7f1b9c0cc0a0 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.669+0000 7f1b93fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f1b9c0d0630 0x7f1b9c15e660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:48478/0 (socket says 172.21.15.120:48478) 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.669+0000 7f1b93fff640 1 -- 172.21.15.120:0/325847090 learned_addr learned my addr 172.21.15.120:0/325847090 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.669+0000 7f1ba0e1e640 1 --2- 172.21.15.120:0/325847090 >> v2:172.21.15.134:3300/0 conn(0x7f1b9c0cc0a0 0x7f1b9c0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.669+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1503230780 0 0) 0x7f1b9c15ba20 con 0x7f1b9c0d0630 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.669+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b84003520 con 0x7f1b9c0d0630 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3234396904 0 0) 0x7f1b84003520 con 0x7f1b9c0d0630 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 >> v2:172.21.15.134:3300/0 conn(0x7f1b9c0cc0a0 msgr2=0x7f1b9c0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 --2- 172.21.15.120:0/325847090 >> v2:172.21.15.134:3300/0 conn(0x7f1b9c0cc0a0 0x7f1b9c0cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 >> v1:172.21.15.134:6789/0 conn(0x7f1b9c15ed70 legacy=0x7f1b9c165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c15b180 con 0x7f1b9c0d0630 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f1b8c001670 con 0x7f1b9c0d0630 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 >> v1:172.21.15.120:6789/0 conn(0x7f1b9c0d0630 legacy=0x7f1b9c15e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b840039b0 0x7f1b84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b84004800 0x7f1b84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b840051c0 0x7f1b8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c15e1d0 con 0x7f1b84004800 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8400c070 con 0x7f1b840051c0 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8400c270 con 0x7f1b840039b0 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b93fff640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b84004800 0x7f1b84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1ba161f640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b840051c0 0x7f1b8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1ba0e1e640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b840039b0 0x7f1b84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b93fff640 1 -- 172.21.15.120:0/325847090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b840039b0 msgr2=0x7f1b84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:01.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b93fff640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b840039b0 0x7f1b84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b93fff640 1 -- 172.21.15.120:0/325847090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b840051c0 msgr2=0x7f1b8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:01.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b93fff640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b840051c0 0x7f1b8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1ba161f640 1 -- 172.21.15.120:0/325847090 reap_dead start 2024-09-10T10:27:01.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.670+0000 7f1b93fff640 1 -- 172.21.15.120:0/325847090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b84003520 con 0x7f1b84004800 2024-09-10T10:27:01.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.671+0000 7f1b93fff640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b84004800 0x7f1b84004c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c290 tx=0x7f1b84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e809cb8c16069d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:01.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.671+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c007d30 con 0x7f1b84004800 2024-09-10T10:27:01.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.671+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c0058d0 con 0x7f1b84004800 2024-09-10T10:27:01.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.671+0000 7f1b937fe640 1 -- 172.21.15.120:0/325847090 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c005ce0 con 0x7f1b84004800 2024-09-10T10:27:01.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.672+0000 7f1ba3082640 1 -- 172.21.15.120:0/325847090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b84004800 msgr2=0x7f1b84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.672+0000 7f1ba3082640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b84004800 0x7f1b84004c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c290 tx=0x7f1b84005830 comp rx=0 tx=0).stop 2024-09-10T10:27:01.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.672+0000 7f1ba3082640 1 -- 172.21.15.120:0/325847090 shutdown_connections 2024-09-10T10:27:01.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.672+0000 7f1ba3082640 1 --2- 172.21.15.120:0/325847090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b84004800 0x7f1b84004c80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.672+0000 7f1ba3082640 1 -- 172.21.15.120:0/325847090 >> 172.21.15.120:0/325847090 conn(0x7f1b9c06d560 msgr2=0x7f1b9c073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:01.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.672+0000 7f1ba3082640 1 -- 172.21.15.120:0/325847090 shutdown_connections 2024-09-10T10:27:01.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.672+0000 7f1ba3082640 1 -- 172.21.15.120:0/325847090 wait complete. 2024-09-10T10:27:01.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.673+0000 7f1ba3082640 1 Processor -- start 2024-09-10T10:27:01.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.673+0000 7f1ba3082640 1 -- start start 2024-09-10T10:27:01.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba3082640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b84004800 0x7f1b9c15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba3082640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b9c15d1a0 0x7f1b9c1f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba3082640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c1f1150 0x7f1b9c14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba3082640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c0cbde0 con 0x7f1b9c1f1150 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba3082640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c15b740 con 0x7f1b84004800 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba3082640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9c15aaf0 con 0x7f1b9c15d1a0 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba161f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c1f1150 0x7f1b9c14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba161f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c1f1150 0x7f1b9c14afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34870/0 (socket says 172.21.15.120:34870) 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba161f640 1 -- 172.21.15.120:0/3967023959 learned_addr learned my addr 172.21.15.120:0/3967023959 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba0e1e640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b84004800 0x7f1b9c15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1b93fff640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b9c15d1a0 0x7f1b9c1f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba161f640 1 -- 172.21.15.120:0/3967023959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b9c15d1a0 msgr2=0x7f1b9c1f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba161f640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b9c15d1a0 0x7f1b9c1f0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba161f640 1 -- 172.21.15.120:0/3967023959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b84004800 msgr2=0x7f1b9c15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba161f640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b84004800 0x7f1b9c15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba161f640 1 -- 172.21.15.120:0/3967023959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c1596c0 con 0x7f1b9c1f1150 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba161f640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c1f1150 0x7f1b9c14afe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c15f1c0 tx=0x7f1b9400fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1b917fa640 1 -- 172.21.15.120:0/3967023959 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9401a070 con 0x7f1b9c1f1150 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1b917fa640 1 -- 172.21.15.120:0/3967023959 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b94006b50 con 0x7f1b9c1f1150 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba3082640 1 -- 172.21.15.120:0/3967023959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c15de80 con 0x7f1b9c1f1150 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1b917fa640 1 -- 172.21.15.120:0/3967023959 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94003970 con 0x7f1b9c1f1150 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.674+0000 7f1ba3082640 1 -- 172.21.15.120:0/3967023959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c158d20 con 0x7f1b9c1f1150 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.676+0000 7f1ba3082640 1 -- 172.21.15.120:0/3967023959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9c155140 con 0x7f1b9c1f1150 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.676+0000 7f1b917fa640 1 -- 172.21.15.120:0/3967023959 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b94003b10 con 0x7f1b9c1f1150 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.676+0000 7f1b917fa640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1b8806be60 0x7f1b8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:01.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.676+0000 7f1b917fa640 1 -- 172.21.15.120:0/3967023959 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1b9408a4c0 con 0x7f1b9c1f1150 2024-09-10T10:27:01.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.676+0000 7f1ba0e1e640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1b8806be60 0x7f1b8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:01.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.677+0000 7f1ba0e1e640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1b8806be60 0x7f1b8806e320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009c40 tx=0x7f1b98009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:01.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.681+0000 7f1b917fa640 1 -- 172.21.15.120:0/3967023959 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1b94058450 con 0x7f1b9c1f1150 2024-09-10T10:27:01.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:01.882+0000 7f1ba3082640 1 -- 172.21.15.120:0/3967023959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1b9c0cce90 con 0x7f1b9c1f1150 2024-09-10T10:27:02.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.861+0000 7f1b917fa640 1 -- 172.21.15.120:0/3967023959 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f1b9405c750 con 0x7f1b9c1f1150 2024-09-10T10:27:02.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 -- 172.21.15.120:0/3967023959 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1b8806be60 msgr2=0x7f1b8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1b8806be60 0x7f1b8806e320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009c40 tx=0x7f1b98009190 comp rx=0 tx=0).stop 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 -- 172.21.15.120:0/3967023959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c1f1150 msgr2=0x7f1b9c14afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c1f1150 0x7f1b9c14afe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c15f1c0 tx=0x7f1b9400fb20 comp rx=0 tx=0).stop 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 -- 172.21.15.120:0/3967023959 shutdown_connections 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1b8806be60 0x7f1b8806e320 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c1f1150 0x7f1b9c14afe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b9c15d1a0 0x7f1b9c1f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 --2- 172.21.15.120:0/3967023959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b84004800 0x7f1b9c15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 -- 172.21.15.120:0/3967023959 >> 172.21.15.120:0/3967023959 conn(0x7f1b9c06d560 msgr2=0x7f1b9c0c6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 -- 172.21.15.120:0/3967023959 shutdown_connections 2024-09-10T10:27:02.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.863+0000 7f1ba3082640 1 -- 172.21.15.120:0/3967023959 wait complete. 2024-09-10T10:27:02.875 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300} 2024-09-10T10:27:02.876 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-09-10T10:27:02.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.977+0000 7f270c9c4640 1 Processor -- start 2024-09-10T10:27:02.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.977+0000 7f270c9c4640 1 -- start start 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.978+0000 7f270c9c4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f27080ccfa0 0x7f27080cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.978+0000 7f270c9c4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080cdce0 0x7f270815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.978+0000 7f270c9c4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f270806d920 0x7f270815c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.978+0000 7f270c9c4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f27080c78e0 con 0x7f270806d920 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.978+0000 7f270c9c4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f270806d520 con 0x7f27080ccfa0 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.978+0000 7f270c9c4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27080c7e20 con 0x7f27080cdce0 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.978+0000 7f2706d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f270806d920 0x7f270815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.978+0000 7f2706d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f270806d920 0x7f270815c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34880/0 (socket says 172.21.15.120:34880) 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.978+0000 7f2706d9d640 1 -- 172.21.15.120:0/2296963161 learned_addr learned my addr 172.21.15.120:0/2296963161 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.979+0000 7f270659c640 1 --2- 172.21.15.120:0/2296963161 >> v2:172.21.15.134:3300/0 conn(0x7f27080ccfa0 0x7f27080cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:02.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.979+0000 7f2705d9b640 1 --2- 172.21.15.120:0/2296963161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080cdce0 0x7f270815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.979+0000 7f2706d9d640 1 -- 172.21.15.120:0/2296963161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080cdce0 msgr2=0x7f270815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.979+0000 7f2706d9d640 1 --2- 172.21.15.120:0/2296963161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080cdce0 0x7f270815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.979+0000 7f2706d9d640 1 -- 172.21.15.120:0/2296963161 >> v2:172.21.15.134:3300/0 conn(0x7f27080ccfa0 msgr2=0x7f27080cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.979+0000 7f2706d9d640 1 --2- 172.21.15.120:0/2296963161 >> v2:172.21.15.134:3300/0 conn(0x7f27080ccfa0 0x7f27080cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.979+0000 7f2706d9d640 1 -- 172.21.15.120:0/2296963161 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27080c7b40 con 0x7f270806d920 2024-09-10T10:27:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.979+0000 7f2706d9d640 1 --2- 172.21.15.120:0/2296963161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f270806d920 0x7f270815c500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27080c0e60 tx=0x7f270002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ed7382acc13021 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270559a640 1 -- 172.21.15.120:0/2296963161 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700037070 con 0x7f270806d920 2024-09-10T10:27:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270559a640 1 -- 172.21.15.120:0/2296963161 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2700032440 con 0x7f270806d920 2024-09-10T10:27:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270559a640 1 -- 172.21.15.120:0/2296963161 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700032810 con 0x7f270806d920 2024-09-10T10:27:02.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270c9c4640 1 -- 172.21.15.120:0/2296963161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f270806d920 msgr2=0x7f270815c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:02.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270c9c4640 1 --2- 172.21.15.120:0/2296963161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f270806d920 0x7f270815c500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27080c0e60 tx=0x7f270002a270 comp rx=0 tx=0).stop 2024-09-10T10:27:02.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270c9c4640 1 -- 172.21.15.120:0/2296963161 shutdown_connections 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270c9c4640 1 --2- 172.21.15.120:0/2296963161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f270806d920 0x7f270815c500 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270c9c4640 1 --2- 172.21.15.120:0/2296963161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080cdce0 0x7f270815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270c9c4640 1 --2- 172.21.15.120:0/2296963161 >> v2:172.21.15.134:3300/0 conn(0x7f27080ccfa0 0x7f27080cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270c9c4640 1 -- 172.21.15.120:0/2296963161 >> 172.21.15.120:0/2296963161 conn(0x7f27080728b0 msgr2=0x7f2708072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270c9c4640 1 -- 172.21.15.120:0/2296963161 shutdown_connections 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.980+0000 7f270c9c4640 1 -- 172.21.15.120:0/2296963161 wait complete. 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.981+0000 7f270c9c4640 1 Processor -- start 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.982+0000 7f270c9c4640 1 -- start start 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.982+0000 7f270c9c4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f270806d920 0x7f27081eee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.982+0000 7f270c9c4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080ccfa0 0x7f27081501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:02.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.982+0000 7f270c9c4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27080cdce0 0x7f27081f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.982+0000 7f270c9c4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f270806c270 con 0x7f27080cdce0 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.982+0000 7f270c9c4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f270806bf90 con 0x7f270806d920 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.982+0000 7f270c9c4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27080735e0 con 0x7f27080ccfa0 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.982+0000 7f2706d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27080cdce0 0x7f27081f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.982+0000 7f2706d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27080cdce0 0x7f27081f1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34892/0 (socket says 172.21.15.120:34892) 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.982+0000 7f2706d9d640 1 -- 172.21.15.120:0/1678202123 learned_addr learned my addr 172.21.15.120:0/1678202123 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f270659c640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f270806d920 0x7f27081eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f2705d9b640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080ccfa0 0x7f27081501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f2706d9d640 1 -- 172.21.15.120:0/1678202123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080ccfa0 msgr2=0x7f27081501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f2706d9d640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080ccfa0 0x7f27081501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f2706d9d640 1 -- 172.21.15.120:0/1678202123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f270806d920 msgr2=0x7f27081eee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:02.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f2706d9d640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f270806d920 0x7f27081eee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f2706d9d640 1 -- 172.21.15.120:0/1678202123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27081f1a10 con 0x7f27080cdce0 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f270659c640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f270806d920 0x7f27081eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f2705d9b640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080ccfa0 0x7f27081501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f2706d9d640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27080cdce0 0x7f27081f1ef0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f270002a2b0 tx=0x7f27000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f26f6ffd640 1 -- 172.21.15.120:0/1678202123 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700041070 con 0x7f27080cdce0 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f26f6ffd640 1 -- 172.21.15.120:0/1678202123 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2700032e50 con 0x7f27080cdce0 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f270c9c4640 1 -- 172.21.15.120:0/1678202123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27081ef390 con 0x7f27080cdce0 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f26f6ffd640 1 -- 172.21.15.120:0/1678202123 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700037040 con 0x7f27080cdce0 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.983+0000 7f270c9c4640 1 -- 172.21.15.120:0/1678202123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27080c2970 con 0x7f27080cdce0 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.984+0000 7f270c9c4640 1 -- 172.21.15.120:0/1678202123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27081f1be0 con 0x7f27080cdce0 2024-09-10T10:27:02.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.984+0000 7f26f6ffd640 1 -- 172.21.15.120:0/1678202123 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2700044070 con 0x7f27080cdce0 2024-09-10T10:27:02.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.985+0000 7f26f6ffd640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26e806be60 0x7f26e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:02.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.985+0000 7f26f6ffd640 1 -- 172.21.15.120:0/1678202123 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f27000aba80 con 0x7f27080cdce0 2024-09-10T10:27:02.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.985+0000 7f270659c640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26e806be60 0x7f26e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:02.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.985+0000 7f270659c640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26e806be60 0x7f26e806e320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f26f8007520 tx=0x7f26f800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:02.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:02.989+0000 7f26f6ffd640 1 -- 172.21.15.120:0/1678202123 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f270007a2a0 con 0x7f27080cdce0 2024-09-10T10:27:03.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.188+0000 7f270c9c4640 1 -- 172.21.15.120:0/1678202123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f27081561b0 con 0x7f27080cdce0 2024-09-10T10:27:03.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.862+0000 7f26f6ffd640 1 -- 172.21.15.120:0/1678202123 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v8) ==== 105+0+0 (secure 0 0 0) 0x7f2700079ba0 con 0x7f27080cdce0 2024-09-10T10:27:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 -- 172.21.15.120:0/1678202123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26e806be60 msgr2=0x7f26e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26e806be60 0x7f26e806e320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f26f8007520 tx=0x7f26f800b040 comp rx=0 tx=0).stop 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 -- 172.21.15.120:0/1678202123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27080cdce0 msgr2=0x7f27081f1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27080cdce0 0x7f27081f1ef0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f270002a2b0 tx=0x7f27000046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 -- 172.21.15.120:0/1678202123 shutdown_connections 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f26e806be60 0x7f26e806e320 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27080cdce0 0x7f27081f1ef0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27080ccfa0 0x7f27081501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 --2- 172.21.15.120:0/1678202123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f270806d920 0x7f27081eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 -- 172.21.15.120:0/1678202123 >> 172.21.15.120:0/1678202123 conn(0x7f27080728b0 msgr2=0x7f2708072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 -- 172.21.15.120:0/1678202123 shutdown_connections 2024-09-10T10:27:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.864+0000 7f270c9c4640 1 -- 172.21.15.120:0/1678202123 wait complete. 2024-09-10T10:27:03.877 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:27:03.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.979+0000 7ff34d5d6640 1 Processor -- start 2024-09-10T10:27:03.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.979+0000 7ff34d5d6640 1 -- start start 2024-09-10T10:27:03.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff34d5d6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff3480cc0a0 0x7ff3480cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:03.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff34d5d6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff3480d0630 0x7ff3480d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:03.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff34d5d6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34815ba20 con 0x7ff34815e660 2024-09-10T10:27:03.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff34d5d6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff34815e1d0 con 0x7ff3480d0630 2024-09-10T10:27:03.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff34d5d6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff34815aea0 con 0x7ff3480cc0a0 2024-09-10T10:27:03.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff33e7fc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff3480d0630 0x7ff3480d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:03.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff33e7fc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff3480d0630 0x7ff3480d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34902/0 (socket says 172.21.15.120:34902) 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff33e7fc640 1 -- 172.21.15.120:0/2526141411 learned_addr learned my addr 172.21.15.120:0/2526141411 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff3477fe640 1 --1- 172.21.15.120:0/2526141411 >> v1:172.21.15.120:6789/0 conn(0x7ff34815e660 0x7ff34815ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:48490/0 (socket says 172.21.15.120:48490) 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff33e7fc640 1 -- 172.21.15.120:0/2526141411 >> v2:172.21.15.134:3300/0 conn(0x7ff3480cc0a0 msgr2=0x7ff3480cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff346ffd640 1 --2- 172.21.15.120:0/2526141411 >> v2:172.21.15.134:3300/0 conn(0x7ff3480cc0a0 0x7ff3480cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff33e7fc640 1 --2- 172.21.15.120:0/2526141411 >> v2:172.21.15.134:3300/0 conn(0x7ff3480cc0a0 0x7ff3480cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff33e7fc640 1 -- 172.21.15.120:0/2526141411 >> v1:172.21.15.120:6789/0 conn(0x7ff34815e660 legacy=0x7ff34815ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff3467fc640 1 -- 172.21.15.120:0/2526141411 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1302083566 0 0) 0x7ff34815ba20 con 0x7ff34815e660 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff33e7fc640 1 -- 172.21.15.120:0/2526141411 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34815b180 con 0x7ff3480d0630 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.980+0000 7ff346ffd640 1 --2- 172.21.15.120:0/2526141411 >> v2:172.21.15.134:3300/0 conn(0x7ff3480cc0a0 0x7ff3480cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.981+0000 7ff33e7fc640 1 --2- 172.21.15.120:0/2526141411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480d0630 0x7ff3480d0a50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff3480caca0 tx=0x7ff33002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fd4d34b1bd8626 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.981+0000 7ff3467fc640 1 -- 172.21.15.120:0/2526141411 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330037070 con 0x7ff3480d0630 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.981+0000 7ff3467fc640 1 -- 172.21.15.120:0/2526141411 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33002ae50 con 0x7ff3480d0630 2024-09-10T10:27:03.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.981+0000 7ff3467fc640 1 -- 172.21.15.120:0/2526141411 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330032860 con 0x7ff3480d0630 2024-09-10T10:27:03.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.981+0000 7ff34d5d6640 1 -- 172.21.15.120:0/2526141411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480d0630 msgr2=0x7ff3480d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:03.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.981+0000 7ff34d5d6640 1 --2- 172.21.15.120:0/2526141411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480d0630 0x7ff3480d0a50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff3480caca0 tx=0x7ff33002a300 comp rx=0 tx=0).stop 2024-09-10T10:27:03.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.982+0000 7ff34d5d6640 1 -- 172.21.15.120:0/2526141411 shutdown_connections 2024-09-10T10:27:03.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.982+0000 7ff34d5d6640 1 --2- 172.21.15.120:0/2526141411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480d0630 0x7ff3480d0a50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:03.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.982+0000 7ff34d5d6640 1 --2- 172.21.15.120:0/2526141411 >> v2:172.21.15.134:3300/0 conn(0x7ff3480cc0a0 0x7ff3480cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.982+0000 7ff34d5d6640 1 -- 172.21.15.120:0/2526141411 >> 172.21.15.120:0/2526141411 conn(0x7ff34806d560 msgr2=0x7ff348073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.982+0000 7ff34d5d6640 1 -- 172.21.15.120:0/2526141411 shutdown_connections 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.982+0000 7ff34d5d6640 1 -- 172.21.15.120:0/2526141411 wait complete. 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.983+0000 7ff34d5d6640 1 Processor -- start 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.983+0000 7ff34d5d6640 1 -- start start 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.983+0000 7ff34d5d6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480cc0a0 0x7ff348155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.983+0000 7ff34d5d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480d0630 0x7ff3480c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff34d5d6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34815e660 0x7ff34815c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff34d5d6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3480cbde0 con 0x7ff3480cc0a0 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff34d5d6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff34815b740 con 0x7ff3480d0630 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff34d5d6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff34815aaf0 con 0x7ff34815e660 2024-09-10T10:27:03.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff346ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480cc0a0 0x7ff348155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff346ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480cc0a0 0x7ff348155440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34918/0 (socket says 172.21.15.120:34918) 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff346ffd640 1 -- 172.21.15.120:0/3177488960 learned_addr learned my addr 172.21.15.120:0/3177488960 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff33e7fc640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480d0630 0x7ff3480c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff3477fe640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34815e660 0x7ff34815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff346ffd640 1 -- 172.21.15.120:0/3177488960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34815e660 msgr2=0x7ff34815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff346ffd640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34815e660 0x7ff34815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff346ffd640 1 -- 172.21.15.120:0/3177488960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480d0630 msgr2=0x7ff3480c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff346ffd640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480d0630 0x7ff3480c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff346ffd640 1 -- 172.21.15.120:0/3177488960 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3481f0ec0 con 0x7ff3480cc0a0 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff3477fe640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34815e660 0x7ff34815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff346ffd640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480cc0a0 0x7ff348155440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff33800c850 tx=0x7ff33800cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff33ffff640 1 -- 172.21.15.120:0/3177488960 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338015070 con 0x7ff3480cc0a0 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff33ffff640 1 -- 172.21.15.120:0/3177488960 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3380077d0 con 0x7ff3480cc0a0 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff34d5d6640 1 -- 172.21.15.120:0/3177488960 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3480ca630 con 0x7ff3480cc0a0 2024-09-10T10:27:03.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff33ffff640 1 -- 172.21.15.120:0/3177488960 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338007b80 con 0x7ff3480cc0a0 2024-09-10T10:27:03.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.984+0000 7ff34d5d6640 1 -- 172.21.15.120:0/3177488960 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3480cabb0 con 0x7ff3480cc0a0 2024-09-10T10:27:03.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.985+0000 7ff34d5d6640 1 -- 172.21.15.120:0/3177488960 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff348155140 con 0x7ff3480cc0a0 2024-09-10T10:27:03.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.986+0000 7ff33ffff640 1 -- 172.21.15.120:0/3177488960 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff338018070 con 0x7ff3480cc0a0 2024-09-10T10:27:03.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.986+0000 7ff33ffff640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32806be60 0x7ff32806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:03.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.986+0000 7ff33ffff640 1 -- 172.21.15.120:0/3177488960 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff33808af70 con 0x7ff3480cc0a0 2024-09-10T10:27:03.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.986+0000 7ff33e7fc640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32806be60 0x7ff32806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:03.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.987+0000 7ff33e7fc640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32806be60 0x7ff32806e320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff330034000 tx=0x7ff33002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:03.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:03.990+0000 7ff33ffff640 1 -- 172.21.15.120:0/3177488960 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff3380595e0 con 0x7ff3480cc0a0 2024-09-10T10:27:04.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.190+0000 7ff34d5d6640 1 -- 172.21.15.120:0/3177488960 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff3480d0b30 con 0x7ff3480cc0a0 2024-09-10T10:27:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.871+0000 7ff33ffff640 1 -- 172.21.15.120:0/3177488960 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7ff338058ee0 con 0x7ff3480cc0a0 2024-09-10T10:27:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 -- 172.21.15.120:0/3177488960 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32806be60 msgr2=0x7ff32806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32806be60 0x7ff32806e320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff330034000 tx=0x7ff33002e000 comp rx=0 tx=0).stop 2024-09-10T10:27:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 -- 172.21.15.120:0/3177488960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480cc0a0 msgr2=0x7ff348155440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480cc0a0 0x7ff348155440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff33800c850 tx=0x7ff33800cd10 comp rx=0 tx=0).stop 2024-09-10T10:27:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 -- 172.21.15.120:0/3177488960 shutdown_connections 2024-09-10T10:27:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32806be60 0x7ff32806e320 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:04.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34815e660 0x7ff34815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:04.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480d0630 0x7ff3480c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:04.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 --2- 172.21.15.120:0/3177488960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3480cc0a0 0x7ff348155440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:04.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 -- 172.21.15.120:0/3177488960 >> 172.21.15.120:0/3177488960 conn(0x7ff34806d560 msgr2=0x7ff3480cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:04.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 -- 172.21.15.120:0/3177488960 shutdown_connections 2024-09-10T10:27:04.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.873+0000 7ff34d5d6640 1 -- 172.21.15.120:0/3177488960 wait complete. 2024-09-10T10:27:04.886 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-10T10:27:04.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.986+0000 7ff351113640 1 Processor -- start 2024-09-10T10:27:04.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.986+0000 7ff351113640 1 -- start start 2024-09-10T10:27:04.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff351113640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34c154b10 0x7ff34c15b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:04.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff351113640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff34c06ec00 0x7ff34c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:04.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff351113640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c06fb10 con 0x7ff34c15dee0 2024-09-10T10:27:04.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff351113640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff34c0cc0f0 con 0x7ff34c06ec00 2024-09-10T10:27:04.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff351113640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c0cc390 con 0x7ff34c154b10 2024-09-10T10:27:04.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff343fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff34c06ec00 0x7ff34c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:04.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff34b59e640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff34c15dee0 0x7ff34c0ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:48494/0 (socket says 172.21.15.120:48494) 2024-09-10T10:27:04.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff34b59e640 1 -- 172.21.15.120:0/1247546152 learned_addr learned my addr 172.21.15.120:0/1247546152 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:04.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff343fff640 1 --2- 172.21.15.120:0/1247546152 >> v2:172.21.15.120:3300/0 conn(0x7ff34c06ec00 0x7ff34c06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34920/0 (socket says 172.21.15.120:34920) 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff34ad9d640 1 --2- 172.21.15.120:0/1247546152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34c154b10 0x7ff34c15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff343fff640 1 -- 172.21.15.120:0/1247546152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34c154b10 msgr2=0x7ff34c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff343fff640 1 --2- 172.21.15.120:0/1247546152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34c154b10 0x7ff34c15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff343fff640 1 -- 172.21.15.120:0/1247546152 >> v1:172.21.15.120:6789/0 conn(0x7ff34c15dee0 legacy=0x7ff34c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff343fff640 1 -- 172.21.15.120:0/1247546152 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c0cc670 con 0x7ff34c06ec00 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.987+0000 7ff34a59c640 1 -- 172.21.15.120:0/1247546152 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4034831508 0 0) 0x7ff34c06fb10 con 0x7ff34c15dee0 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.988+0000 7ff343fff640 1 --2- 172.21.15.120:0/1247546152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 0x7ff34c06cd20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff3340097a0 tx=0x7ff33402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c261f13652000e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.988+0000 7ff34a59c640 1 -- 172.21.15.120:0/1247546152 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334037070 con 0x7ff34c06ec00 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.988+0000 7ff34a59c640 1 -- 172.21.15.120:0/1247546152 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff334032440 con 0x7ff34c06ec00 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.988+0000 7ff34a59c640 1 -- 172.21.15.120:0/1247546152 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334032810 con 0x7ff34c06ec00 2024-09-10T10:27:04.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.988+0000 7ff351113640 1 -- 172.21.15.120:0/1247546152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 msgr2=0x7ff34c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:04.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.988+0000 7ff351113640 1 --2- 172.21.15.120:0/1247546152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 0x7ff34c06cd20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff3340097a0 tx=0x7ff33402a180 comp rx=0 tx=0).stop 2024-09-10T10:27:04.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.989+0000 7ff351113640 1 -- 172.21.15.120:0/1247546152 shutdown_connections 2024-09-10T10:27:04.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.989+0000 7ff351113640 1 --2- 172.21.15.120:0/1247546152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 0x7ff34c06cd20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:04.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.989+0000 7ff351113640 1 --2- 172.21.15.120:0/1247546152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34c154b10 0x7ff34c15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:04.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.989+0000 7ff351113640 1 -- 172.21.15.120:0/1247546152 >> 172.21.15.120:0/1247546152 conn(0x7ff34c067950 msgr2=0x7ff34c067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:04.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.989+0000 7ff351113640 1 -- 172.21.15.120:0/1247546152 shutdown_connections 2024-09-10T10:27:04.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.989+0000 7ff351113640 1 -- 172.21.15.120:0/1247546152 wait complete. 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.990+0000 7ff351113640 1 Processor -- start 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.990+0000 7ff351113640 1 -- start start 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.990+0000 7ff351113640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 0x7ff34c0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.990+0000 7ff351113640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34c15dee0 0x7ff34c0aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.990+0000 7ff351113640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c0c5cf0 0x7ff34c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.990+0000 7ff351113640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c15b7b0 con 0x7ff34c06ec00 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff351113640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c15ad50 con 0x7ff34c0c5cf0 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff351113640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c154f00 con 0x7ff34c15dee0 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff34ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 0x7ff34c0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff34ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 0x7ff34c0ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34930/0 (socket says 172.21.15.120:34930) 2024-09-10T10:27:04.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff34ad9d640 1 -- 172.21.15.120:0/1720250516 learned_addr learned my addr 172.21.15.120:0/1720250516 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:04.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff34b59e640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c0c5cf0 0x7ff34c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:04.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff343fff640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34c15dee0 0x7ff34c0aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:04.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff34ad9d640 1 -- 172.21.15.120:0/1720250516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34c15dee0 msgr2=0x7ff34c0aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:04.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff34ad9d640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34c15dee0 0x7ff34c0aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:04.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff34ad9d640 1 -- 172.21.15.120:0/1720250516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c0c5cf0 msgr2=0x7ff34c152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:04.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff34ad9d640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c0c5cf0 0x7ff34c152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:04.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff34ad9d640 1 -- 172.21.15.120:0/1720250516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c0cda80 con 0x7ff34c06ec00 2024-09-10T10:27:04.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff34ad9d640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 0x7ff34c0ceb00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00d800 tx=0x7ff33c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:04.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff3437fe640 1 -- 172.21.15.120:0/1720250516 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00b660 con 0x7ff34c06ec00 2024-09-10T10:27:04.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff3437fe640 1 -- 172.21.15.120:0/1720250516 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33c019070 con 0x7ff34c06ec00 2024-09-10T10:27:04.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff3437fe640 1 -- 172.21.15.120:0/1720250516 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00ba20 con 0x7ff34c06ec00 2024-09-10T10:27:04.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff351113640 1 -- 172.21.15.120:0/1720250516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c1f39d0 con 0x7ff34c06ec00 2024-09-10T10:27:04.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.991+0000 7ff351113640 1 -- 172.21.15.120:0/1720250516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c0ce020 con 0x7ff34c06ec00 2024-09-10T10:27:04.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.992+0000 7ff351113640 1 -- 172.21.15.120:0/1720250516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c06f7d0 con 0x7ff34c06ec00 2024-09-10T10:27:04.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.993+0000 7ff3437fe640 1 -- 172.21.15.120:0/1720250516 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff33c00bbc0 con 0x7ff34c06ec00 2024-09-10T10:27:04.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.993+0000 7ff3437fe640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32c06be30 0x7ff32c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:04.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.993+0000 7ff3437fe640 1 -- 172.21.15.120:0/1720250516 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff33c08b420 con 0x7ff34c06ec00 2024-09-10T10:27:04.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.993+0000 7ff343fff640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32c06be30 0x7ff32c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:04.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.993+0000 7ff343fff640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32c06be30 0x7ff32c06e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff334034000 tx=0x7ff33402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:04.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:04.997+0000 7ff3437fe640 1 -- 172.21.15.120:0/1720250516 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff33c0594d0 con 0x7ff34c06ec00 2024-09-10T10:27:05.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.196+0000 7ff351113640 1 -- 172.21.15.120:0/1720250516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7ff34c070330 con 0x7ff34c06ec00 2024-09-10T10:27:05.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.877+0000 7ff3437fe640 1 -- 172.21.15.120:0/1720250516 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v11) ==== 115+0+0 (secure 0 0 0) 0x7ff33c05d7d0 con 0x7ff34c06ec00 2024-09-10T10:27:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.879+0000 7ff351113640 1 -- 172.21.15.120:0/1720250516 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32c06be30 msgr2=0x7ff32c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.879+0000 7ff351113640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32c06be30 0x7ff32c06e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff334034000 tx=0x7ff33402e000 comp rx=0 tx=0).stop 2024-09-10T10:27:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.880+0000 7ff351113640 1 -- 172.21.15.120:0/1720250516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 msgr2=0x7ff34c0ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.880+0000 7ff351113640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 0x7ff34c0ceb00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00d800 tx=0x7ff33c00dcc0 comp rx=0 tx=0).stop 2024-09-10T10:27:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.880+0000 7ff351113640 1 -- 172.21.15.120:0/1720250516 shutdown_connections 2024-09-10T10:27:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.880+0000 7ff351113640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff32c06be30 0x7ff32c06e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.880+0000 7ff351113640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c0c5cf0 0x7ff34c152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.880+0000 7ff351113640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff34c15dee0 0x7ff34c0aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:05.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.880+0000 7ff351113640 1 --2- 172.21.15.120:0/1720250516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c06ec00 0x7ff34c0ceb00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:05.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.880+0000 7ff351113640 1 -- 172.21.15.120:0/1720250516 >> 172.21.15.120:0/1720250516 conn(0x7ff34c067950 msgr2=0x7ff34c1e5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:05.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.880+0000 7ff351113640 1 -- 172.21.15.120:0/1720250516 shutdown_connections 2024-09-10T10:27:05.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.880+0000 7ff351113640 1 -- 172.21.15.120:0/1720250516 wait complete. 2024-09-10T10:27:05.893 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:27:05.893 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:27:05.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.991+0000 7f372ad9f640 1 Processor -- start 2024-09-10T10:27:05.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.991+0000 7f372ad9f640 1 -- start start 2024-09-10T10:27:05.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f372ad9f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f37240c6b70 0x7f372415b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f372ad9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240cc040 0x7f37240cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f372ad9f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f372406d140 con 0x7f37240c75a0 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f372ad9f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f372415b880 con 0x7f37240c6b70 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f372ad9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f37240cab20 con 0x7f37240cc040 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f3729d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f37240c6b70 0x7f372415b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f3729d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f37240c6b70 0x7f372415b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34938/0 (socket says 172.21.15.120:34938) 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f372959c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f37240c75a0 0x7f37240cb570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:48500/0 (socket says 172.21.15.120:48500) 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f3729d9d640 1 -- 172.21.15.120:0/346562458 learned_addr learned my addr 172.21.15.120:0/346562458 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f372a59e640 1 --2- 172.21.15.120:0/346562458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240cc040 0x7f37240cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f3729d9d640 1 -- 172.21.15.120:0/346562458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240cc040 msgr2=0x7f37240cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f3729d9d640 1 --2- 172.21.15.120:0/346562458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240cc040 0x7f37240cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:05.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f3729d9d640 1 -- 172.21.15.120:0/346562458 >> v1:172.21.15.120:6789/0 conn(0x7f37240c75a0 legacy=0x7f37240cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:05.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f3729d9d640 1 -- 172.21.15.120:0/346562458 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372406d420 con 0x7f37240c6b70 2024-09-10T10:27:05.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f3728d9b640 1 -- 172.21.15.120:0/346562458 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637593662 0 0) 0x7f372406d140 con 0x7f37240c75a0 2024-09-10T10:27:05.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.992+0000 7f372a59e640 1 --2- 172.21.15.120:0/346562458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240cc040 0x7f37240cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:05.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.993+0000 7f3729d9d640 1 --2- 172.21.15.120:0/346562458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240c6b70 0x7f372415b340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3720009870 tx=0x7f372002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303c23a937a27ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:05.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.993+0000 7f3728d9b640 1 -- 172.21.15.120:0/346562458 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720037070 con 0x7f37240c6b70 2024-09-10T10:27:05.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.993+0000 7f3728d9b640 1 -- 172.21.15.120:0/346562458 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3720032440 con 0x7f37240c6b70 2024-09-10T10:27:05.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.993+0000 7f3728d9b640 1 -- 172.21.15.120:0/346562458 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720032810 con 0x7f37240c6b70 2024-09-10T10:27:05.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.994+0000 7f372ad9f640 1 -- 172.21.15.120:0/346562458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240c6b70 msgr2=0x7f372415b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:05.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.994+0000 7f372ad9f640 1 --2- 172.21.15.120:0/346562458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240c6b70 0x7f372415b340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3720009870 tx=0x7f372002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.994+0000 7f372ad9f640 1 -- 172.21.15.120:0/346562458 shutdown_connections 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.994+0000 7f372ad9f640 1 --2- 172.21.15.120:0/346562458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240cc040 0x7f37240cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.994+0000 7f372ad9f640 1 --2- 172.21.15.120:0/346562458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240c6b70 0x7f372415b340 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.994+0000 7f372ad9f640 1 -- 172.21.15.120:0/346562458 >> 172.21.15.120:0/346562458 conn(0x7f37240ccc20 msgr2=0x7f37240cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.994+0000 7f372ad9f640 1 -- 172.21.15.120:0/346562458 shutdown_connections 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.994+0000 7f372ad9f640 1 -- 172.21.15.120:0/346562458 wait complete. 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.995+0000 7f372ad9f640 1 Processor -- start 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.995+0000 7f372ad9f640 1 -- start start 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.995+0000 7f372ad9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37240c6b70 0x7f372415c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.995+0000 7f372ad9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240c75a0 0x7f37241f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.995+0000 7f372ad9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240cc040 0x7f37241f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372ad9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37240cb050 con 0x7f37240cc040 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372ad9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37240cad70 con 0x7f37240c6b70 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372ad9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f37240cbc80 con 0x7f37240c75a0 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372a59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240cc040 0x7f37241f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372a59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240cc040 0x7f37241f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34954/0 (socket says 172.21.15.120:34954) 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372a59e640 1 -- 172.21.15.120:0/2585915500 learned_addr learned my addr 172.21.15.120:0/2585915500 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f3729d9d640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37240c6b70 0x7f372415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372959c640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240c75a0 0x7f37241f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372a59e640 1 -- 172.21.15.120:0/2585915500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240c75a0 msgr2=0x7f37241f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372a59e640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240c75a0 0x7f37241f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372a59e640 1 -- 172.21.15.120:0/2585915500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37240c6b70 msgr2=0x7f372415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372a59e640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37240c6b70 0x7f372415c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372a59e640 1 -- 172.21.15.120:0/2585915500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372415bc30 con 0x7f37240cc040 2024-09-10T10:27:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372959c640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240c75a0 0x7f37241f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:06.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372a59e640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240cc040 0x7f37241f0c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f371c00ced0 tx=0x7f371c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f371a7fc640 1 -- 172.21.15.120:0/2585915500 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c003b90 con 0x7f37240cc040 2024-09-10T10:27:06.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f371a7fc640 1 -- 172.21.15.120:0/2585915500 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371c003d50 con 0x7f37240cc040 2024-09-10T10:27:06.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f371a7fc640 1 -- 172.21.15.120:0/2585915500 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c004260 con 0x7f37240cc040 2024-09-10T10:27:06.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372ad9f640 1 -- 172.21.15.120:0/2585915500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372415bf10 con 0x7f37240cc040 2024-09-10T10:27:06.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.996+0000 7f372ad9f640 1 -- 172.21.15.120:0/2585915500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372415a600 con 0x7f37240cc040 2024-09-10T10:27:06.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.997+0000 7f372ad9f640 1 -- 172.21.15.120:0/2585915500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372406cd20 con 0x7f37240cc040 2024-09-10T10:27:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.998+0000 7f371a7fc640 1 -- 172.21.15.120:0/2585915500 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f371c004400 con 0x7f37240cc040 2024-09-10T10:27:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.998+0000 7f371a7fc640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f370806be60 0x7f370806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.998+0000 7f371a7fc640 1 -- 172.21.15.120:0/2585915500 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f371c08b9f0 con 0x7f37240cc040 2024-09-10T10:27:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.998+0000 7f3729d9d640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f370806be60 0x7f370806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:05.998+0000 7f3729d9d640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f370806be60 0x7f370806e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3720034000 tx=0x7f372002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.002+0000 7f371a7fc640 1 -- 172.21.15.120:0/2585915500 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f371c059b60 con 0x7f37240cc040 2024-09-10T10:27:06.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.200+0000 7f372ad9f640 1 -- 172.21.15.120:0/2585915500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f372415c0e0 con 0x7f37240cc040 2024-09-10T10:27:06.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.200+0000 7f371a7fc640 1 -- 172.21.15.120:0/2585915500 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6895 (secure 0 0 0) 0x7f371c05de60 con 0x7f37240cc040 2024-09-10T10:27:06.202 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:06.202 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":11,"btime":"2024-09-10T10:27:05:872448+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:26:41.113510+0000","modified":"2024-09-10T10:27:05.197809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1],"up":{"mds_0":4306,"mds_1":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":1,"incarnation":9,"state":"up:creating","state_seq":1,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-10T10:27:06.202 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 11 2024-09-10T10:27:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 -- 172.21.15.120:0/2585915500 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f370806be60 msgr2=0x7f370806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f370806be60 0x7f370806e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3720034000 tx=0x7f372002e000 comp rx=0 tx=0).stop 2024-09-10T10:27:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 -- 172.21.15.120:0/2585915500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240cc040 msgr2=0x7f37241f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240cc040 0x7f37241f0c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f371c00ced0 tx=0x7f371c00a710 comp rx=0 tx=0).stop 2024-09-10T10:27:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 -- 172.21.15.120:0/2585915500 shutdown_connections 2024-09-10T10:27:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f370806be60 0x7f370806e320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37240cc040 0x7f37241f0c00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37240c75a0 0x7f37241f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 --2- 172.21.15.120:0/2585915500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37240c6b70 0x7f372415c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 -- 172.21.15.120:0/2585915500 >> 172.21.15.120:0/2585915500 conn(0x7f37240ccc20 msgr2=0x7f37240cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:06.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 -- 172.21.15.120:0/2585915500 shutdown_connections 2024-09-10T10:27:06.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.202+0000 7f372ad9f640 1 -- 172.21.15.120:0/2585915500 wait complete. 2024-09-10T10:27:06.215 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:27:06.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.312+0000 7f0cc340e640 1 Processor -- start 2024-09-10T10:27:06.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.312+0000 7f0cc340e640 1 -- start start 2024-09-10T10:27:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc340e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c6980 0x7f0cbc0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc340e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0cbc0c7770 0x7f0cbc15aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc340e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cbc15b420 con 0x7f0cbc0cb410 2024-09-10T10:27:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc340e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f0cbc15b9e0 con 0x7f0cbc0c7770 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc340e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0cbc15b700 con 0x7f0cbc0c6980 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc09a9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0cbc0c7770 0x7f0cbc15aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc19ab640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f0cbc0cb410 0x7f0cbc0cbfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:48510/0 (socket says 172.21.15.120:48510) 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc19ab640 1 -- 172.21.15.120:0/1870348714 learned_addr learned my addr 172.21.15.120:0/1870348714 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc09a9640 1 -- 172.21.15.120:0/1870348714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c6980 msgr2=0x7f0cbc0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc11aa640 1 --2- 172.21.15.120:0/1870348714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c6980 0x7f0cbc0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc09a9640 1 --2- 172.21.15.120:0/1870348714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c6980 0x7f0cbc0cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc09a9640 1 -- 172.21.15.120:0/1870348714 >> v1:172.21.15.120:6789/0 conn(0x7f0cbc0cb410 legacy=0x7f0cbc0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cb3fff640 1 -- 172.21.15.120:0/1870348714 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 956716554 0 0) 0x7f0cbc15b420 con 0x7f0cbc0cb410 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc09a9640 1 -- 172.21.15.120:0/1870348714 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc0ccc50 con 0x7f0cbc0c7770 2024-09-10T10:27:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.313+0000 7f0cc11aa640 1 --2- 172.21.15.120:0/1870348714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c6980 0x7f0cbc0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.314+0000 7f0cc09a9640 1 --2- 172.21.15.120:0/1870348714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0c7770 0x7f0cbc15aa90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0097a0 tx=0x7f0cac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663910f6402e7a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.314+0000 7f0cb3fff640 1 -- 172.21.15.120:0/1870348714 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac037070 con 0x7f0cbc0c7770 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.314+0000 7f0cb3fff640 1 -- 172.21.15.120:0/1870348714 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cac032440 con 0x7f0cbc0c7770 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.314+0000 7f0cb3fff640 1 -- 172.21.15.120:0/1870348714 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac032810 con 0x7f0cbc0c7770 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.314+0000 7f0cc340e640 1 -- 172.21.15.120:0/1870348714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0c7770 msgr2=0x7f0cbc15aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.314+0000 7f0cc340e640 1 --2- 172.21.15.120:0/1870348714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0c7770 0x7f0cbc15aa90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0097a0 tx=0x7f0cac02a180 comp rx=0 tx=0).stop 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.314+0000 7f0cc340e640 1 -- 172.21.15.120:0/1870348714 shutdown_connections 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.314+0000 7f0cc340e640 1 --2- 172.21.15.120:0/1870348714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0c7770 0x7f0cbc15aa90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.314+0000 7f0cc340e640 1 --2- 172.21.15.120:0/1870348714 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c6980 0x7f0cbc0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.314+0000 7f0cc340e640 1 -- 172.21.15.120:0/1870348714 >> 172.21.15.120:0/1870348714 conn(0x7f0cbc06cb70 msgr2=0x7f0cbc06e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.315+0000 7f0cc340e640 1 -- 172.21.15.120:0/1870348714 shutdown_connections 2024-09-10T10:27:06.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.315+0000 7f0cc340e640 1 -- 172.21.15.120:0/1870348714 wait complete. 2024-09-10T10:27:06.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.316+0000 7f0cc340e640 1 Processor -- start 2024-09-10T10:27:06.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.316+0000 7f0cc340e640 1 -- start start 2024-09-10T10:27:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.316+0000 7f0cc340e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cbc0c6980 0x7f0cbc1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.316+0000 7f0cc340e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c7770 0x7f0cbc1efe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.316+0000 7f0cc340e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0cb410 0x7f0cbc14ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.316+0000 7f0cc340e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc0d0fe0 con 0x7f0cbc0cb410 2024-09-10T10:27:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.316+0000 7f0cc340e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc0cd210 con 0x7f0cbc0c6980 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.316+0000 7f0cc340e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0cbc0cc6f0 con 0x7f0cbc0c7770 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.316+0000 7f0cc19ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0cb410 0x7f0cbc14ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.316+0000 7f0cc19ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0cb410 0x7f0cbc14ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34960/0 (socket says 172.21.15.120:34960) 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc19ab640 1 -- 172.21.15.120:0/2416786957 learned_addr learned my addr 172.21.15.120:0/2416786957 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc11aa640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cbc0c6980 0x7f0cbc1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc09a9640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c7770 0x7f0cbc1efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc19ab640 1 -- 172.21.15.120:0/2416786957 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c7770 msgr2=0x7f0cbc1efe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc19ab640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c7770 0x7f0cbc1efe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc19ab640 1 -- 172.21.15.120:0/2416786957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cbc0c6980 msgr2=0x7f0cbc1510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc19ab640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cbc0c6980 0x7f0cbc1510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc19ab640 1 -- 172.21.15.120:0/2416786957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc15a320 con 0x7f0cbc0cb410 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc09a9640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c7770 0x7f0cbc1efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc11aa640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cbc0c6980 0x7f0cbc1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc19ab640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0cb410 0x7f0cbc14ea00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400c7a0 tx=0x7f0cb400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/2416786957 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4018070 con 0x7f0cbc0cb410 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/2416786957 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb400a640 con 0x7f0cbc0cb410 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc340e640 1 -- 172.21.15.120:0/2416786957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cbc1e0bb0 con 0x7f0cbc0cb410 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/2416786957 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb400aa70 con 0x7f0cbc0cb410 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.317+0000 7f0cc340e640 1 -- 172.21.15.120:0/2416786957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cbc1f4280 con 0x7f0cbc0cb410 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.318+0000 7f0cc340e640 1 -- 172.21.15.120:0/2416786957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cbc1e0d80 con 0x7f0cbc0cb410 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.319+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/2416786957 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cb401b070 con 0x7f0cbc0cb410 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.319+0000 7f0cb1ffb640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0ca006be30 0x7f0ca006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.319+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/2416786957 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0cb408a800 con 0x7f0cbc0cb410 2024-09-10T10:27:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.319+0000 7f0cc11aa640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0ca006be30 0x7f0ca006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.320+0000 7f0cc11aa640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0ca006be30 0x7f0ca006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8006fd0 tx=0x7f0cb8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.323+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/2416786957 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0cb4058de0 con 0x7f0cbc0cb410 2024-09-10T10:27:06.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.490+0000 7f0cc340e640 1 -- 172.21.15.120:0/2416786957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cbc15b420 con 0x7f0cbc0cb410 2024-09-10T10:27:06.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.491+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/2416786957 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f0cb40586e0 con 0x7f0cbc0cb410 2024-09-10T10:27:06.493 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:06.493 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:47.823215+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.493+0000 7f0cc340e640 1 -- 172.21.15.120:0/2416786957 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0ca006be30 msgr2=0x7f0ca006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.493+0000 7f0cc340e640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0ca006be30 0x7f0ca006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8006fd0 tx=0x7f0cb8008040 comp rx=0 tx=0).stop 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.493+0000 7f0cc340e640 1 -- 172.21.15.120:0/2416786957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0cb410 msgr2=0x7f0cbc14ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.493+0000 7f0cc340e640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0cb410 0x7f0cbc14ea00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400c7a0 tx=0x7f0cb400cc60 comp rx=0 tx=0).stop 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.493+0000 7f0cc340e640 1 -- 172.21.15.120:0/2416786957 shutdown_connections 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.493+0000 7f0cc340e640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0ca006be30 0x7f0ca006e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.493+0000 7f0cc340e640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cbc0cb410 0x7f0cbc14ea00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.493+0000 7f0cc340e640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cbc0c7770 0x7f0cbc1efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.493+0000 7f0cc340e640 1 --2- 172.21.15.120:0/2416786957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cbc0c6980 0x7f0cbc1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.493+0000 7f0cc340e640 1 -- 172.21.15.120:0/2416786957 >> 172.21.15.120:0/2416786957 conn(0x7f0cbc06cb70 msgr2=0x7f0cbc0cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.494+0000 7f0cc340e640 1 -- 172.21.15.120:0/2416786957 shutdown_connections 2024-09-10T10:27:06.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.494+0000 7f0cc340e640 1 -- 172.21.15.120:0/2416786957 wait complete. 2024-09-10T10:27:06.506 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:27:06.506 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:27:06.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.602+0000 7f0157559640 1 Processor -- start 2024-09-10T10:27:06.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.602+0000 7f0157559640 1 -- start start 2024-09-10T10:27:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.603+0000 7f0157559640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0150158140 0x7f015006cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.603+0000 7f0157559640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f015015aa00 0x7f01500ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.603+0000 7f0157559640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015006d4a0 con 0x7f015006eaa0 2024-09-10T10:27:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.603+0000 7f0157559640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f01500c8dc0 con 0x7f0150158140 2024-09-10T10:27:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.603+0000 7f0157559640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0150158520 con 0x7f015015aa00 2024-09-10T10:27:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.603+0000 7f01552f5640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0150158140 0x7f015006cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.603+0000 7f01552f5640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0150158140 0x7f015006cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34970/0 (socket says 172.21.15.120:34970) 2024-09-10T10:27:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.603+0000 7f01552f5640 1 -- 172.21.15.120:0/905475347 learned_addr learned my addr 172.21.15.120:0/905475347 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.603+0000 7f0155af6640 1 --2- 172.21.15.120:0/905475347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f015015aa00 0x7f01500ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.604+0000 7f01552f5640 1 -- 172.21.15.120:0/905475347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f015015aa00 msgr2=0x7f01500ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.604+0000 7f01552f5640 1 --2- 172.21.15.120:0/905475347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f015015aa00 0x7f01500ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.604+0000 7f01552f5640 1 -- 172.21.15.120:0/905475347 >> v1:172.21.15.134:6789/0 conn(0x7f015006eaa0 legacy=0x7f01500ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.604+0000 7f01552f5640 1 -- 172.21.15.120:0/905475347 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01500c9440 con 0x7f0150158140 2024-09-10T10:27:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.604+0000 7f01552f5640 1 --2- 172.21.15.120:0/905475347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0150158140 0x7f015006cf60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0140009e50 tx=0x7f014002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35c7a72ab67c85c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.604+0000 7f014ffff640 1 -- 172.21.15.120:0/905475347 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140036070 con 0x7f0150158140 2024-09-10T10:27:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.604+0000 7f014ffff640 1 -- 172.21.15.120:0/905475347 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0140031600 con 0x7f0150158140 2024-09-10T10:27:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.604+0000 7f014ffff640 1 -- 172.21.15.120:0/905475347 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01400319d0 con 0x7f0150158140 2024-09-10T10:27:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.605+0000 7f0157559640 1 -- 172.21.15.120:0/905475347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0150158140 msgr2=0x7f015006cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.605+0000 7f0157559640 1 --2- 172.21.15.120:0/905475347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0150158140 0x7f015006cf60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0140009e50 tx=0x7f014002a7a0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.605+0000 7f0157559640 1 -- 172.21.15.120:0/905475347 shutdown_connections 2024-09-10T10:27:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.605+0000 7f0157559640 1 --2- 172.21.15.120:0/905475347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f015015aa00 0x7f01500ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.605+0000 7f0157559640 1 --2- 172.21.15.120:0/905475347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0150158140 0x7f015006cf60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.605+0000 7f0157559640 1 -- 172.21.15.120:0/905475347 >> 172.21.15.120:0/905475347 conn(0x7f0150067950 msgr2=0x7f0150067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.605+0000 7f0157559640 1 -- 172.21.15.120:0/905475347 shutdown_connections 2024-09-10T10:27:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.605+0000 7f0157559640 1 -- 172.21.15.120:0/905475347 wait complete. 2024-09-10T10:27:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.606+0000 7f0157559640 1 Processor -- start 2024-09-10T10:27:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.606+0000 7f0157559640 1 -- start start 2024-09-10T10:27:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f0157559640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f015006eaa0 0x7f01500ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f0157559640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0150158140 0x7f01500afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f0157559640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f015015aa00 0x7f01500b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f0157559640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01500c99c0 con 0x7f015006eaa0 2024-09-10T10:27:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f0157559640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01500c96e0 con 0x7f0150158140 2024-09-10T10:27:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f0157559640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01500b1020 con 0x7f015015aa00 2024-09-10T10:27:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f01552f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f015006eaa0 0x7f01500ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f01552f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f015006eaa0 0x7f01500ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34974/0 (socket says 172.21.15.120:34974) 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f01552f5640 1 -- 172.21.15.120:0/91866242 learned_addr learned my addr 172.21.15.120:0/91866242 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f0154af4640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0150158140 0x7f01500afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f0155af6640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f015015aa00 0x7f01500b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f01552f5640 1 -- 172.21.15.120:0/91866242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f015015aa00 msgr2=0x7f01500b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f01552f5640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f015015aa00 0x7f01500b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f01552f5640 1 -- 172.21.15.120:0/91866242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0150158140 msgr2=0x7f01500afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f01552f5640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0150158140 0x7f01500afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.607+0000 7f01552f5640 1 -- 172.21.15.120:0/91866242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01500cef00 con 0x7f015006eaa0 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.608+0000 7f01552f5640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f015006eaa0 0x7f01500ce9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0140009f70 tx=0x7f0140004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.608+0000 7f014dffb640 1 -- 172.21.15.120:0/91866242 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140036050 con 0x7f015006eaa0 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.608+0000 7f014dffb640 1 -- 172.21.15.120:0/91866242 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0140042070 con 0x7f015006eaa0 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.608+0000 7f0157559640 1 -- 172.21.15.120:0/91866242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01501f09c0 con 0x7f015006eaa0 2024-09-10T10:27:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.608+0000 7f014dffb640 1 -- 172.21.15.120:0/91866242 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140009510 con 0x7f015006eaa0 2024-09-10T10:27:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.608+0000 7f0157559640 1 -- 172.21.15.120:0/91866242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01500b1980 con 0x7f015006eaa0 2024-09-10T10:27:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.609+0000 7f0157559640 1 -- 172.21.15.120:0/91866242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015015e980 con 0x7f015006eaa0 2024-09-10T10:27:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.609+0000 7f014dffb640 1 -- 172.21.15.120:0/91866242 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0140007bf0 con 0x7f015006eaa0 2024-09-10T10:27:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.610+0000 7f014dffb640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f013c06be30 0x7f013c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.610+0000 7f014dffb640 1 -- 172.21.15.120:0/91866242 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f01400abf70 con 0x7f015006eaa0 2024-09-10T10:27:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.610+0000 7f0154af4640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f013c06be30 0x7f013c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.610+0000 7f0154af4640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f013c06be30 0x7f013c06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0150157a00 tx=0x7f0144009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.614+0000 7f014dffb640 1 -- 172.21.15.120:0/91866242 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0140079f00 con 0x7f015006eaa0 2024-09-10T10:27:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.776+0000 7f0157559640 1 -- 172.21.15.120:0/91866242 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01500b09c0 con 0x7f013c06be30 2024-09-10T10:27:06.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.784+0000 7f014dffb640 1 -- 172.21.15.120:0/91866242 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332611 (secure 0 0 0) 0x7f01500b09c0 con 0x7f013c06be30 2024-09-10T10:27:06.789 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:27:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 -- 172.21.15.120:0/91866242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f013c06be30 msgr2=0x7f013c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f013c06be30 0x7f013c06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0150157a00 tx=0x7f0144009040 comp rx=0 tx=0).stop 2024-09-10T10:27:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 -- 172.21.15.120:0/91866242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f015006eaa0 msgr2=0x7f01500ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f015006eaa0 0x7f01500ce9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0140009f70 tx=0x7f0140004a40 comp rx=0 tx=0).stop 2024-09-10T10:27:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 -- 172.21.15.120:0/91866242 shutdown_connections 2024-09-10T10:27:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f013c06be30 0x7f013c06e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f015015aa00 0x7f01500b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0150158140 0x7f01500afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 --2- 172.21.15.120:0/91866242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f015006eaa0 0x7f01500ce9c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 -- 172.21.15.120:0/91866242 >> 172.21.15.120:0/91866242 conn(0x7f0150067950 msgr2=0x7f0150156f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 -- 172.21.15.120:0/91866242 shutdown_connections 2024-09-10T10:27:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.788+0000 7f0157559640 1 -- 172.21.15.120:0/91866242 wait complete. 2024-09-10T10:27:06.800 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-10T10:27:06.095174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218616,"kb_used_data":2368,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756104,"statfs":{"total":773094113280,"available":772870250496,"internally_reserved":0,"allocated":2424832,"data_stored":2086532,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002745"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271924+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:26:48.271924+0000","last_peered":"2024-09-10T10:26:48.271924+0000","last_clean":"2024-09-10T10:26:48.271924+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:26:48.271924+0000","last_undegraded":"2024-09-10T10:26:48.271924+0000","last_fullsized":"2024-09-10T10:26:48.271924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827265+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:26:47.827265+0000","last_peered":"2024-09-10T10:26:47.827265+0000","last_clean":"2024-09-10T10:26:47.827265+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:26:47.827265+0000","last_undegraded":"2024-09-10T10:26:47.827265+0000","last_fullsized":"2024-09-10T10:26:47.827265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826403+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:26:47.826403+0000","last_peered":"2024-09-10T10:26:47.826403+0000","last_clean":"2024-09-10T10:26:47.826403+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:26:47.826403+0000","last_undegraded":"2024-09-10T10:26:47.826403+0000","last_fullsized":"2024-09-10T10:26:47.826403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827328+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:26:47.827328+0000","last_peered":"2024-09-10T10:26:47.827328+0000","last_clean":"2024-09-10T10:26:47.827328+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:26:47.827328+0000","last_undegraded":"2024-09-10T10:26:47.827328+0000","last_fullsized":"2024-09-10T10:26:47.827328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271728+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:26:48.271728+0000","last_peered":"2024-09-10T10:26:48.271728+0000","last_clean":"2024-09-10T10:26:48.271728+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:26:48.271728+0000","last_undegraded":"2024-09-10T10:26:48.271728+0000","last_fullsized":"2024-09-10T10:26:48.271728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829940+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:26:47.829940+0000","last_peered":"2024-09-10T10:26:47.829940+0000","last_clean":"2024-09-10T10:26:47.829940+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:26:47.829940+0000","last_undegraded":"2024-09-10T10:26:47.829940+0000","last_fullsized":"2024-09-10T10:26:47.829940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437245+0000","last_change":"2024-09-10T10:26:40.796337+0000","last_active":"2024-09-10T10:26:48.437245+0000","last_peered":"2024-09-10T10:26:48.437245+0000","last_clean":"2024-09-10T10:26:48.437245+0000","last_became_active":"2024-09-10T10:26:40.796029+0000","last_became_peered":"2024-09-10T10:26:40.796029+0000","last_unstale":"2024-09-10T10:26:48.437245+0000","last_undegraded":"2024-09-10T10:26:48.437245+0000","last_fullsized":"2024-09-10T10:26:48.437245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829990+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:26:47.829990+0000","last_peered":"2024-09-10T10:26:47.829990+0000","last_clean":"2024-09-10T10:26:47.829990+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:26:47.829990+0000","last_undegraded":"2024-09-10T10:26:47.829990+0000","last_fullsized":"2024-09-10T10:26:47.829990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831277+0000","last_change":"2024-09-10T10:26:45.814368+0000","last_active":"2024-09-10T10:26:47.831277+0000","last_peered":"2024-09-10T10:26:47.831277+0000","last_clean":"2024-09-10T10:26:47.831277+0000","last_became_active":"2024-09-10T10:26:45.813953+0000","last_became_peered":"2024-09-10T10:26:45.813953+0000","last_unstale":"2024-09-10T10:26:47.831277+0000","last_undegraded":"2024-09-10T10:26:47.831277+0000","last_fullsized":"2024-09-10T10:26:47.831277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436841+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:26:48.436841+0000","last_peered":"2024-09-10T10:26:48.436841+0000","last_clean":"2024-09-10T10:26:48.436841+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:26:48.436841+0000","last_undegraded":"2024-09-10T10:26:48.436841+0000","last_fullsized":"2024-09-10T10:26:48.436841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830425+0000","last_change":"2024-09-10T10:26:41.789145+0000","last_active":"2024-09-10T10:26:47.830425+0000","last_peered":"2024-09-10T10:26:47.830425+0000","last_clean":"2024-09-10T10:26:47.830425+0000","last_became_active":"2024-09-10T10:26:41.788874+0000","last_became_peered":"2024-09-10T10:26:41.788874+0000","last_unstale":"2024-09-10T10:26:47.830425+0000","last_undegraded":"2024-09-10T10:26:47.830425+0000","last_fullsized":"2024-09-10T10:26:47.830425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830738+0000","last_change":"2024-09-10T10:26:45.816587+0000","last_active":"2024-09-10T10:26:47.830738+0000","last_peered":"2024-09-10T10:26:47.830738+0000","last_clean":"2024-09-10T10:26:47.830738+0000","last_became_active":"2024-09-10T10:26:45.816138+0000","last_became_peered":"2024-09-10T10:26:45.816138+0000","last_unstale":"2024-09-10T10:26:47.830738+0000","last_undegraded":"2024-09-10T10:26:47.830738+0000","last_fullsized":"2024-09-10T10:26:47.830738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436252+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:26:48.436252+0000","last_peered":"2024-09-10T10:26:48.436252+0000","last_clean":"2024-09-10T10:26:48.436252+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:26:48.436252+0000","last_undegraded":"2024-09-10T10:26:48.436252+0000","last_fullsized":"2024-09-10T10:26:48.436252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436878+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:26:48.436878+0000","last_peered":"2024-09-10T10:26:48.436878+0000","last_clean":"2024-09-10T10:26:48.436878+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:26:48.436878+0000","last_undegraded":"2024-09-10T10:26:48.436878+0000","last_fullsized":"2024-09-10T10:26:48.436878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831602+0000","last_change":"2024-09-10T10:26:45.816967+0000","last_active":"2024-09-10T10:26:47.831602+0000","last_peered":"2024-09-10T10:26:47.831602+0000","last_clean":"2024-09-10T10:26:47.831602+0000","last_became_active":"2024-09-10T10:26:45.816161+0000","last_became_peered":"2024-09-10T10:26:45.816161+0000","last_unstale":"2024-09-10T10:26:47.831602+0000","last_undegraded":"2024-09-10T10:26:47.831602+0000","last_fullsized":"2024-09-10T10:26:47.831602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826732+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:26:47.826732+0000","last_peered":"2024-09-10T10:26:47.826732+0000","last_clean":"2024-09-10T10:26:47.826732+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:26:47.826732+0000","last_undegraded":"2024-09-10T10:26:47.826732+0000","last_fullsized":"2024-09-10T10:26:47.826732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436716+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:26:48.436716+0000","last_peered":"2024-09-10T10:26:48.436716+0000","last_clean":"2024-09-10T10:26:48.436716+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:26:48.436716+0000","last_undegraded":"2024-09-10T10:26:48.436716+0000","last_fullsized":"2024-09-10T10:26:48.436716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830916+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:26:47.830916+0000","last_peered":"2024-09-10T10:26:47.830916+0000","last_clean":"2024-09-10T10:26:47.830916+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:26:47.830916+0000","last_undegraded":"2024-09-10T10:26:47.830916+0000","last_fullsized":"2024-09-10T10:26:47.830916+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826532+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:26:47.826532+0000","last_peered":"2024-09-10T10:26:47.826532+0000","last_clean":"2024-09-10T10:26:47.826532+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:26:47.826532+0000","last_undegraded":"2024-09-10T10:26:47.826532+0000","last_fullsized":"2024-09-10T10:26:47.826532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436164+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:26:48.436164+0000","last_peered":"2024-09-10T10:26:48.436164+0000","last_clean":"2024-09-10T10:26:48.436164+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:26:48.436164+0000","last_undegraded":"2024-09-10T10:26:48.436164+0000","last_fullsized":"2024-09-10T10:26:48.436164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436960+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:26:48.436960+0000","last_peered":"2024-09-10T10:26:48.436960+0000","last_clean":"2024-09-10T10:26:48.436960+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:26:48.436960+0000","last_undegraded":"2024-09-10T10:26:48.436960+0000","last_fullsized":"2024-09-10T10:26:48.436960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830283+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:26:47.830283+0000","last_peered":"2024-09-10T10:26:47.830283+0000","last_clean":"2024-09-10T10:26:47.830283+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:26:47.830283+0000","last_undegraded":"2024-09-10T10:26:47.830283+0000","last_fullsized":"2024-09-10T10:26:47.830283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436570+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:26:48.436570+0000","last_peered":"2024-09-10T10:26:48.436570+0000","last_clean":"2024-09-10T10:26:48.436570+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:26:48.436570+0000","last_undegraded":"2024-09-10T10:26:48.436570+0000","last_fullsized":"2024-09-10T10:26:48.436570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830167+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:26:47.830167+0000","last_peered":"2024-09-10T10:26:47.830167+0000","last_clean":"2024-09-10T10:26:47.830167+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:26:47.830167+0000","last_undegraded":"2024-09-10T10:26:47.830167+0000","last_fullsized":"2024-09-10T10:26:47.830167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271909+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:26:48.271909+0000","last_peered":"2024-09-10T10:26:48.271909+0000","last_clean":"2024-09-10T10:26:48.271909+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:26:48.271909+0000","last_undegraded":"2024-09-10T10:26:48.271909+0000","last_fullsized":"2024-09-10T10:26:48.271909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437755+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:26:48.437755+0000","last_peered":"2024-09-10T10:26:48.437755+0000","last_clean":"2024-09-10T10:26:48.437755+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:26:48.437755+0000","last_undegraded":"2024-09-10T10:26:48.437755+0000","last_fullsized":"2024-09-10T10:26:48.437755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829937+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:26:47.829937+0000","last_peered":"2024-09-10T10:26:47.829937+0000","last_clean":"2024-09-10T10:26:47.829937+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:26:47.829937+0000","last_undegraded":"2024-09-10T10:26:47.829937+0000","last_fullsized":"2024-09-10T10:26:47.829937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827469+0000","last_change":"2024-09-10T10:26:41.795085+0000","last_active":"2024-09-10T10:26:47.827469+0000","last_peered":"2024-09-10T10:26:47.827469+0000","last_clean":"2024-09-10T10:26:47.827469+0000","last_became_active":"2024-09-10T10:26:41.794819+0000","last_became_peered":"2024-09-10T10:26:41.794819+0000","last_unstale":"2024-09-10T10:26:47.827469+0000","last_undegraded":"2024-09-10T10:26:47.827469+0000","last_fullsized":"2024-09-10T10:26:47.827469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436983+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:26:48.436983+0000","last_peered":"2024-09-10T10:26:48.436983+0000","last_clean":"2024-09-10T10:26:48.436983+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:26:48.436983+0000","last_undegraded":"2024-09-10T10:26:48.436983+0000","last_fullsized":"2024-09-10T10:26:48.436983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826343+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:26:47.826343+0000","last_peered":"2024-09-10T10:26:47.826343+0000","last_clean":"2024-09-10T10:26:47.826343+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:26:47.826343+0000","last_undegraded":"2024-09-10T10:26:47.826343+0000","last_fullsized":"2024-09-10T10:26:47.826343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436707+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:26:48.436707+0000","last_peered":"2024-09-10T10:26:48.436707+0000","last_clean":"2024-09-10T10:26:48.436707+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:26:48.436707+0000","last_undegraded":"2024-09-10T10:26:48.436707+0000","last_fullsized":"2024-09-10T10:26:48.436707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437399+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:26:48.437399+0000","last_peered":"2024-09-10T10:26:48.437399+0000","last_clean":"2024-09-10T10:26:48.437399+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:26:48.437399+0000","last_undegraded":"2024-09-10T10:26:48.437399+0000","last_fullsized":"2024-09-10T10:26:48.437399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271566+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:26:48.271566+0000","last_peered":"2024-09-10T10:26:48.271566+0000","last_clean":"2024-09-10T10:26:48.271566+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:26:48.271566+0000","last_undegraded":"2024-09-10T10:26:48.271566+0000","last_fullsized":"2024-09-10T10:26:48.271566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830633+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:26:47.830633+0000","last_peered":"2024-09-10T10:26:47.830633+0000","last_clean":"2024-09-10T10:26:47.830633+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:26:47.830633+0000","last_undegraded":"2024-09-10T10:26:47.830633+0000","last_fullsized":"2024-09-10T10:26:47.830633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436761+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:26:48.436761+0000","last_peered":"2024-09-10T10:26:48.436761+0000","last_clean":"2024-09-10T10:26:48.436761+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:26:48.436761+0000","last_undegraded":"2024-09-10T10:26:48.436761+0000","last_fullsized":"2024-09-10T10:26:48.436761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436477+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:26:48.436477+0000","last_peered":"2024-09-10T10:26:48.436477+0000","last_clean":"2024-09-10T10:26:48.436477+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:26:48.436477+0000","last_undegraded":"2024-09-10T10:26:48.436477+0000","last_fullsized":"2024-09-10T10:26:48.436477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831395+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:26:47.831395+0000","last_peered":"2024-09-10T10:26:47.831395+0000","last_clean":"2024-09-10T10:26:47.831395+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:26:47.831395+0000","last_undegraded":"2024-09-10T10:26:47.831395+0000","last_fullsized":"2024-09-10T10:26:47.831395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830140+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:26:47.830140+0000","last_peered":"2024-09-10T10:26:47.830140+0000","last_clean":"2024-09-10T10:26:47.830140+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:26:47.830140+0000","last_undegraded":"2024-09-10T10:26:47.830140+0000","last_fullsized":"2024-09-10T10:26:47.830140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436656+0000","last_change":"2024-09-10T10:26:45.814008+0000","last_active":"2024-09-10T10:26:48.436656+0000","last_peered":"2024-09-10T10:26:48.436656+0000","last_clean":"2024-09-10T10:26:48.436656+0000","last_became_active":"2024-09-10T10:26:45.813627+0000","last_became_peered":"2024-09-10T10:26:45.813627+0000","last_unstale":"2024-09-10T10:26:48.436656+0000","last_undegraded":"2024-09-10T10:26:48.436656+0000","last_fullsized":"2024-09-10T10:26:48.436656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436224+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:26:48.436224+0000","last_peered":"2024-09-10T10:26:48.436224+0000","last_clean":"2024-09-10T10:26:48.436224+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:26:48.436224+0000","last_undegraded":"2024-09-10T10:26:48.436224+0000","last_fullsized":"2024-09-10T10:26:48.436224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436034+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:26:48.436034+0000","last_peered":"2024-09-10T10:26:48.436034+0000","last_clean":"2024-09-10T10:26:48.436034+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:26:48.436034+0000","last_undegraded":"2024-09-10T10:26:48.436034+0000","last_fullsized":"2024-09-10T10:26:48.436034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436987+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:26:48.436987+0000","last_peered":"2024-09-10T10:26:48.436987+0000","last_clean":"2024-09-10T10:26:48.436987+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:26:48.436987+0000","last_undegraded":"2024-09-10T10:26:48.436987+0000","last_fullsized":"2024-09-10T10:26:48.436987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271316+0000","last_change":"2024-09-10T10:26:41.799224+0000","last_active":"2024-09-10T10:26:48.271316+0000","last_peered":"2024-09-10T10:26:48.271316+0000","last_clean":"2024-09-10T10:26:48.271316+0000","last_became_active":"2024-09-10T10:26:41.798948+0000","last_became_peered":"2024-09-10T10:26:41.798948+0000","last_unstale":"2024-09-10T10:26:48.271316+0000","last_undegraded":"2024-09-10T10:26:48.271316+0000","last_fullsized":"2024-09-10T10:26:48.271316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436688+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:26:48.436688+0000","last_peered":"2024-09-10T10:26:48.436688+0000","last_clean":"2024-09-10T10:26:48.436688+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:26:48.436688+0000","last_undegraded":"2024-09-10T10:26:48.436688+0000","last_fullsized":"2024-09-10T10:26:48.436688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271901+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:26:48.271901+0000","last_peered":"2024-09-10T10:26:48.271901+0000","last_clean":"2024-09-10T10:26:48.271901+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:26:48.271901+0000","last_undegraded":"2024-09-10T10:26:48.271901+0000","last_fullsized":"2024-09-10T10:26:48.271901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826634+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:26:47.826634+0000","last_peered":"2024-09-10T10:26:47.826634+0000","last_clean":"2024-09-10T10:26:47.826634+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:26:47.826634+0000","last_undegraded":"2024-09-10T10:26:47.826634+0000","last_fullsized":"2024-09-10T10:26:47.826634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830090+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:26:47.830090+0000","last_peered":"2024-09-10T10:26:47.830090+0000","last_clean":"2024-09-10T10:26:47.830090+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:26:47.830090+0000","last_undegraded":"2024-09-10T10:26:47.830090+0000","last_fullsized":"2024-09-10T10:26:47.830090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437071+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:26:48.437071+0000","last_peered":"2024-09-10T10:26:48.437071+0000","last_clean":"2024-09-10T10:26:48.437071+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:26:48.437071+0000","last_undegraded":"2024-09-10T10:26:48.437071+0000","last_fullsized":"2024-09-10T10:26:48.437071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437061+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:26:48.437061+0000","last_peered":"2024-09-10T10:26:48.437061+0000","last_clean":"2024-09-10T10:26:48.437061+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:26:48.437061+0000","last_undegraded":"2024-09-10T10:26:48.437061+0000","last_fullsized":"2024-09-10T10:26:48.437061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437121+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:26:48.437121+0000","last_peered":"2024-09-10T10:26:48.437121+0000","last_clean":"2024-09-10T10:26:48.437121+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:26:48.437121+0000","last_undegraded":"2024-09-10T10:26:48.437121+0000","last_fullsized":"2024-09-10T10:26:48.437121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436307+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:26:48.436307+0000","last_peered":"2024-09-10T10:26:48.436307+0000","last_clean":"2024-09-10T10:26:48.436307+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:26:48.436307+0000","last_undegraded":"2024-09-10T10:26:48.436307+0000","last_fullsized":"2024-09-10T10:26:48.436307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436298+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:26:48.436298+0000","last_peered":"2024-09-10T10:26:48.436298+0000","last_clean":"2024-09-10T10:26:48.436298+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:26:48.436298+0000","last_undegraded":"2024-09-10T10:26:48.436298+0000","last_fullsized":"2024-09-10T10:26:48.436298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830079+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:26:47.830079+0000","last_peered":"2024-09-10T10:26:47.830079+0000","last_clean":"2024-09-10T10:26:47.830079+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:26:47.830079+0000","last_undegraded":"2024-09-10T10:26:47.830079+0000","last_fullsized":"2024-09-10T10:26:47.830079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436713+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:26:48.436713+0000","last_peered":"2024-09-10T10:26:48.436713+0000","last_clean":"2024-09-10T10:26:48.436713+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:26:48.436713+0000","last_undegraded":"2024-09-10T10:26:48.436713+0000","last_fullsized":"2024-09-10T10:26:48.436713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271787+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:26:48.271787+0000","last_peered":"2024-09-10T10:26:48.271787+0000","last_clean":"2024-09-10T10:26:48.271787+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:26:48.271787+0000","last_undegraded":"2024-09-10T10:26:48.271787+0000","last_fullsized":"2024-09-10T10:26:48.271787+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437650+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:26:48.437650+0000","last_peered":"2024-09-10T10:26:48.437650+0000","last_clean":"2024-09-10T10:26:48.437650+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:26:48.437650+0000","last_undegraded":"2024-09-10T10:26:48.437650+0000","last_fullsized":"2024-09-10T10:26:48.437650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436651+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:26:48.436651+0000","last_peered":"2024-09-10T10:26:48.436651+0000","last_clean":"2024-09-10T10:26:48.436651+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:26:48.436651+0000","last_undegraded":"2024-09-10T10:26:48.436651+0000","last_fullsized":"2024-09-10T10:26:48.436651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829425+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:26:47.829425+0000","last_peered":"2024-09-10T10:26:47.829425+0000","last_clean":"2024-09-10T10:26:47.829425+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:26:47.829425+0000","last_undegraded":"2024-09-10T10:26:47.829425+0000","last_fullsized":"2024-09-10T10:26:47.829425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437218+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:26:48.437218+0000","last_peered":"2024-09-10T10:26:48.437218+0000","last_clean":"2024-09-10T10:26:48.437218+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:26:48.437218+0000","last_undegraded":"2024-09-10T10:26:48.437218+0000","last_fullsized":"2024-09-10T10:26:48.437218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830430+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:26:47.830430+0000","last_peered":"2024-09-10T10:26:47.830430+0000","last_clean":"2024-09-10T10:26:47.830430+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:26:47.830430+0000","last_undegraded":"2024-09-10T10:26:47.830430+0000","last_fullsized":"2024-09-10T10:26:47.830430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829765+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:26:47.829765+0000","last_peered":"2024-09-10T10:26:47.829765+0000","last_clean":"2024-09-10T10:26:47.829765+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:26:47.829765+0000","last_undegraded":"2024-09-10T10:26:47.829765+0000","last_fullsized":"2024-09-10T10:26:47.829765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437037+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:26:48.437037+0000","last_peered":"2024-09-10T10:26:48.437037+0000","last_clean":"2024-09-10T10:26:48.437037+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:26:48.437037+0000","last_undegraded":"2024-09-10T10:26:48.437037+0000","last_fullsized":"2024-09-10T10:26:48.437037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.435841+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:26:48.435841+0000","last_peered":"2024-09-10T10:26:48.435841+0000","last_clean":"2024-09-10T10:26:48.435841+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:26:48.435841+0000","last_undegraded":"2024-09-10T10:26:48.435841+0000","last_fullsized":"2024-09-10T10:26:48.435841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271756+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:26:48.271756+0000","last_peered":"2024-09-10T10:26:48.271756+0000","last_clean":"2024-09-10T10:26:48.271756+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:26:48.271756+0000","last_undegraded":"2024-09-10T10:26:48.271756+0000","last_fullsized":"2024-09-10T10:26:48.271756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436218+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:26:48.436218+0000","last_peered":"2024-09-10T10:26:48.436218+0000","last_clean":"2024-09-10T10:26:48.436218+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:26:48.436218+0000","last_undegraded":"2024-09-10T10:26:48.436218+0000","last_fullsized":"2024-09-10T10:26:48.436218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436280+0000","last_change":"2024-09-10T10:26:40.794739+0000","last_active":"2024-09-10T10:26:48.436280+0000","last_peered":"2024-09-10T10:26:48.436280+0000","last_clean":"2024-09-10T10:26:48.436280+0000","last_became_active":"2024-09-10T10:26:40.794430+0000","last_became_peered":"2024-09-10T10:26:40.794430+0000","last_unstale":"2024-09-10T10:26:48.436280+0000","last_undegraded":"2024-09-10T10:26:48.436280+0000","last_fullsized":"2024-09-10T10:26:48.436280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827508+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:26:47.827508+0000","last_peered":"2024-09-10T10:26:47.827508+0000","last_clean":"2024-09-10T10:26:47.827508+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:26:47.827508+0000","last_undegraded":"2024-09-10T10:26:47.827508+0000","last_fullsized":"2024-09-10T10:26:47.827508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436803+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:26:48.436803+0000","last_peered":"2024-09-10T10:26:48.436803+0000","last_clean":"2024-09-10T10:26:48.436803+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:26:48.436803+0000","last_undegraded":"2024-09-10T10:26:48.436803+0000","last_fullsized":"2024-09-10T10:26:48.436803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830046+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:26:47.830046+0000","last_peered":"2024-09-10T10:26:47.830046+0000","last_clean":"2024-09-10T10:26:47.830046+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:26:47.830046+0000","last_undegraded":"2024-09-10T10:26:47.830046+0000","last_fullsized":"2024-09-10T10:26:47.830046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827010+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:26:47.827010+0000","last_peered":"2024-09-10T10:26:47.827010+0000","last_clean":"2024-09-10T10:26:47.827010+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:26:47.827010+0000","last_undegraded":"2024-09-10T10:26:47.827010+0000","last_fullsized":"2024-09-10T10:26:47.827010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271524+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:26:48.271524+0000","last_peered":"2024-09-10T10:26:48.271524+0000","last_clean":"2024-09-10T10:26:48.271524+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:26:48.271524+0000","last_undegraded":"2024-09-10T10:26:48.271524+0000","last_fullsized":"2024-09-10T10:26:48.271524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271879+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:26:48.271879+0000","last_peered":"2024-09-10T10:26:48.271879+0000","last_clean":"2024-09-10T10:26:48.271879+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:26:48.271879+0000","last_undegraded":"2024-09-10T10:26:48.271879+0000","last_fullsized":"2024-09-10T10:26:48.271879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830480+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:26:47.830480+0000","last_peered":"2024-09-10T10:26:47.830480+0000","last_clean":"2024-09-10T10:26:47.830480+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:26:47.830480+0000","last_undegraded":"2024-09-10T10:26:47.830480+0000","last_fullsized":"2024-09-10T10:26:47.830480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436991+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:26:48.436991+0000","last_peered":"2024-09-10T10:26:48.436991+0000","last_clean":"2024-09-10T10:26:48.436991+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:26:48.436991+0000","last_undegraded":"2024-09-10T10:26:48.436991+0000","last_fullsized":"2024-09-10T10:26:48.436991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:54.051970+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:26:54.051970+0000","last_peered":"2024-09-10T10:26:54.051970+0000","last_clean":"2024-09-10T10:26:54.051970+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:26:54.051970+0000","last_undegraded":"2024-09-10T10:26:54.051970+0000","last_fullsized":"2024-09-10T10:26:54.051970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436702+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:26:48.436702+0000","last_peered":"2024-09-10T10:26:48.436702+0000","last_clean":"2024-09-10T10:26:48.436702+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:26:48.436702+0000","last_undegraded":"2024-09-10T10:26:48.436702+0000","last_fullsized":"2024-09-10T10:26:48.436702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831260+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:26:47.831260+0000","last_peered":"2024-09-10T10:26:47.831260+0000","last_clean":"2024-09-10T10:26:47.831260+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:26:47.831260+0000","last_undegraded":"2024-09-10T10:26:47.831260+0000","last_fullsized":"2024-09-10T10:26:47.831260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436571+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:26:48.436571+0000","last_peered":"2024-09-10T10:26:48.436571+0000","last_clean":"2024-09-10T10:26:48.436571+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:26:48.436571+0000","last_undegraded":"2024-09-10T10:26:48.436571+0000","last_fullsized":"2024-09-10T10:26:48.436571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831349+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:26:47.831349+0000","last_peered":"2024-09-10T10:26:47.831349+0000","last_clean":"2024-09-10T10:26:47.831349+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:26:47.831349+0000","last_undegraded":"2024-09-10T10:26:47.831349+0000","last_fullsized":"2024-09-10T10:26:47.831349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271920+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:26:48.271920+0000","last_peered":"2024-09-10T10:26:48.271920+0000","last_clean":"2024-09-10T10:26:48.271920+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:26:48.271920+0000","last_undegraded":"2024-09-10T10:26:48.271920+0000","last_fullsized":"2024-09-10T10:26:48.271920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830756+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:26:47.830756+0000","last_peered":"2024-09-10T10:26:47.830756+0000","last_clean":"2024-09-10T10:26:47.830756+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:26:47.830756+0000","last_undegraded":"2024-09-10T10:26:47.830756+0000","last_fullsized":"2024-09-10T10:26:47.830756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826511+0000","last_change":"2024-09-10T10:26:45.819680+0000","last_active":"2024-09-10T10:26:47.826511+0000","last_peered":"2024-09-10T10:26:47.826511+0000","last_clean":"2024-09-10T10:26:47.826511+0000","last_became_active":"2024-09-10T10:26:45.819319+0000","last_became_peered":"2024-09-10T10:26:45.819319+0000","last_unstale":"2024-09-10T10:26:47.826511+0000","last_undegraded":"2024-09-10T10:26:47.826511+0000","last_fullsized":"2024-09-10T10:26:47.826511+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:43.492188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830718+0000","last_change":"2024-09-10T10:26:41.792618+0000","last_active":"2024-09-10T10:26:47.830718+0000","last_peered":"2024-09-10T10:26:47.830718+0000","last_clean":"2024-09-10T10:26:47.830718+0000","last_became_active":"2024-09-10T10:26:41.792323+0000","last_became_peered":"2024-09-10T10:26:41.792323+0000","last_unstale":"2024-09-10T10:26:47.830718+0000","last_undegraded":"2024-09-10T10:26:47.830718+0000","last_fullsized":"2024-09-10T10:26:47.830718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437639+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:26:48.437639+0000","last_peered":"2024-09-10T10:26:48.437639+0000","last_clean":"2024-09-10T10:26:48.437639+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:26:48.437639+0000","last_undegraded":"2024-09-10T10:26:48.437639+0000","last_fullsized":"2024-09-10T10:26:48.437639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826609+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:26:47.826609+0000","last_peered":"2024-09-10T10:26:47.826609+0000","last_clean":"2024-09-10T10:26:47.826609+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:26:47.826609+0000","last_undegraded":"2024-09-10T10:26:47.826609+0000","last_fullsized":"2024-09-10T10:26:47.826609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271720+0000","last_change":"2024-09-10T10:26:46.193656+0000","last_active":"2024-09-10T10:26:48.271720+0000","last_peered":"2024-09-10T10:26:48.271720+0000","last_clean":"2024-09-10T10:26:48.271720+0000","last_became_active":"2024-09-10T10:26:46.193109+0000","last_became_peered":"2024-09-10T10:26:46.193109+0000","last_unstale":"2024-09-10T10:26:48.271720+0000","last_undegraded":"2024-09-10T10:26:48.271720+0000","last_fullsized":"2024-09-10T10:26:48.271720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436035+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:26:48.436035+0000","last_peered":"2024-09-10T10:26:48.436035+0000","last_clean":"2024-09-10T10:26:48.436035+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:26:48.436035+0000","last_undegraded":"2024-09-10T10:26:48.436035+0000","last_fullsized":"2024-09-10T10:26:48.436035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829601+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:26:47.829601+0000","last_peered":"2024-09-10T10:26:47.829601+0000","last_clean":"2024-09-10T10:26:47.829601+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:26:47.829601+0000","last_undegraded":"2024-09-10T10:26:47.829601+0000","last_fullsized":"2024-09-10T10:26:47.829601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437648+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:26:48.437648+0000","last_peered":"2024-09-10T10:26:48.437648+0000","last_clean":"2024-09-10T10:26:48.437648+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:26:48.437648+0000","last_undegraded":"2024-09-10T10:26:48.437648+0000","last_fullsized":"2024-09-10T10:26:48.437648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826450+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:26:47.826450+0000","last_peered":"2024-09-10T10:26:47.826450+0000","last_clean":"2024-09-10T10:26:47.826450+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:26:47.826450+0000","last_undegraded":"2024-09-10T10:26:47.826450+0000","last_fullsized":"2024-09-10T10:26:47.826450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826672+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:26:47.826672+0000","last_peered":"2024-09-10T10:26:47.826672+0000","last_clean":"2024-09-10T10:26:47.826672+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:26:47.826672+0000","last_undegraded":"2024-09-10T10:26:47.826672+0000","last_fullsized":"2024-09-10T10:26:47.826672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436533+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:26:48.436533+0000","last_peered":"2024-09-10T10:26:48.436533+0000","last_clean":"2024-09-10T10:26:48.436533+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:26:48.436533+0000","last_undegraded":"2024-09-10T10:26:48.436533+0000","last_fullsized":"2024-09-10T10:26:48.436533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271816+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:26:48.271816+0000","last_peered":"2024-09-10T10:26:48.271816+0000","last_clean":"2024-09-10T10:26:48.271816+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:26:48.271816+0000","last_undegraded":"2024-09-10T10:26:48.271816+0000","last_fullsized":"2024-09-10T10:26:48.271816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436252+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:26:48.436252+0000","last_peered":"2024-09-10T10:26:48.436252+0000","last_clean":"2024-09-10T10:26:48.436252+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:26:48.436252+0000","last_undegraded":"2024-09-10T10:26:48.436252+0000","last_fullsized":"2024-09-10T10:26:48.436252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826785+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:26:47.826785+0000","last_peered":"2024-09-10T10:26:47.826785+0000","last_clean":"2024-09-10T10:26:47.826785+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:26:47.826785+0000","last_undegraded":"2024-09-10T10:26:47.826785+0000","last_fullsized":"2024-09-10T10:26:47.826785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830947+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:26:47.830947+0000","last_peered":"2024-09-10T10:26:47.830947+0000","last_clean":"2024-09-10T10:26:47.830947+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:26:47.830947+0000","last_undegraded":"2024-09-10T10:26:47.830947+0000","last_fullsized":"2024-09-10T10:26:47.830947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827120+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:26:47.827120+0000","last_peered":"2024-09-10T10:26:47.827120+0000","last_clean":"2024-09-10T10:26:47.827120+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:26:47.827120+0000","last_undegraded":"2024-09-10T10:26:47.827120+0000","last_fullsized":"2024-09-10T10:26:47.827120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436082+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:26:48.436082+0000","last_peered":"2024-09-10T10:26:48.436082+0000","last_clean":"2024-09-10T10:26:48.436082+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:26:48.436082+0000","last_undegraded":"2024-09-10T10:26:48.436082+0000","last_fullsized":"2024-09-10T10:26:48.436082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.435938+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:26:48.435938+0000","last_peered":"2024-09-10T10:26:48.435938+0000","last_clean":"2024-09-10T10:26:48.435938+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:26:48.435938+0000","last_undegraded":"2024-09-10T10:26:48.435938+0000","last_fullsized":"2024-09-10T10:26:48.435938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830004+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:26:47.830004+0000","last_peered":"2024-09-10T10:26:47.830004+0000","last_clean":"2024-09-10T10:26:47.830004+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:26:47.830004+0000","last_undegraded":"2024-09-10T10:26:47.830004+0000","last_fullsized":"2024-09-10T10:26:47.830004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271738+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:26:48.271738+0000","last_peered":"2024-09-10T10:26:48.271738+0000","last_clean":"2024-09-10T10:26:48.271738+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:26:48.271738+0000","last_undegraded":"2024-09-10T10:26:48.271738+0000","last_fullsized":"2024-09-10T10:26:48.271738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827172+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:26:47.827172+0000","last_peered":"2024-09-10T10:26:47.827172+0000","last_clean":"2024-09-10T10:26:47.827172+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:26:47.827172+0000","last_undegraded":"2024-09-10T10:26:47.827172+0000","last_fullsized":"2024-09-10T10:26:47.827172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436417+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:26:48.436417+0000","last_peered":"2024-09-10T10:26:48.436417+0000","last_clean":"2024-09-10T10:26:48.436417+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:26:48.436417+0000","last_undegraded":"2024-09-10T10:26:48.436417+0000","last_fullsized":"2024-09-10T10:26:48.436417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437644+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:26:48.437644+0000","last_peered":"2024-09-10T10:26:48.437644+0000","last_clean":"2024-09-10T10:26:48.437644+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:26:48.437644+0000","last_undegraded":"2024-09-10T10:26:48.437644+0000","last_fullsized":"2024-09-10T10:26:48.437644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827413+0000","last_change":"2024-09-10T10:26:41.802903+0000","last_active":"2024-09-10T10:26:47.827413+0000","last_peered":"2024-09-10T10:26:47.827413+0000","last_clean":"2024-09-10T10:26:47.827413+0000","last_became_active":"2024-09-10T10:26:41.802586+0000","last_became_peered":"2024-09-10T10:26:41.802586+0000","last_unstale":"2024-09-10T10:26:47.827413+0000","last_undegraded":"2024-09-10T10:26:47.827413+0000","last_fullsized":"2024-09-10T10:26:47.827413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829500+0000","last_change":"2024-09-10T10:26:41.800060+0000","last_active":"2024-09-10T10:26:47.829500+0000","last_peered":"2024-09-10T10:26:47.829500+0000","last_clean":"2024-09-10T10:26:47.829500+0000","last_became_active":"2024-09-10T10:26:41.799729+0000","last_became_peered":"2024-09-10T10:26:41.799729+0000","last_unstale":"2024-09-10T10:26:47.829500+0000","last_undegraded":"2024-09-10T10:26:47.829500+0000","last_fullsized":"2024-09-10T10:26:47.829500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826575+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:26:47.826575+0000","last_peered":"2024-09-10T10:26:47.826575+0000","last_clean":"2024-09-10T10:26:47.826575+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:26:47.826575+0000","last_undegraded":"2024-09-10T10:26:47.826575+0000","last_fullsized":"2024-09-10T10:26:47.826575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829948+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:26:47.829948+0000","last_peered":"2024-09-10T10:26:47.829948+0000","last_clean":"2024-09-10T10:26:47.829948+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:26:47.829948+0000","last_undegraded":"2024-09-10T10:26:47.829948+0000","last_fullsized":"2024-09-10T10:26:47.829948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271616+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:26:48.271616+0000","last_peered":"2024-09-10T10:26:48.271616+0000","last_clean":"2024-09-10T10:26:48.271616+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:26:48.271616+0000","last_undegraded":"2024-09-10T10:26:48.271616+0000","last_fullsized":"2024-09-10T10:26:48.271616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436780+0000","last_change":"2024-09-10T10:26:41.787435+0000","last_active":"2024-09-10T10:26:48.436780+0000","last_peered":"2024-09-10T10:26:48.436780+0000","last_clean":"2024-09-10T10:26:48.436780+0000","last_became_active":"2024-09-10T10:26:41.787083+0000","last_became_peered":"2024-09-10T10:26:41.787083+0000","last_unstale":"2024-09-10T10:26:48.436780+0000","last_undegraded":"2024-09-10T10:26:48.436780+0000","last_fullsized":"2024-09-10T10:26:48.436780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437419+0000","last_change":"2024-09-10T10:26:41.792587+0000","last_active":"2024-09-10T10:26:48.437419+0000","last_peered":"2024-09-10T10:26:48.437419+0000","last_clean":"2024-09-10T10:26:48.437419+0000","last_became_active":"2024-09-10T10:26:41.792320+0000","last_became_peered":"2024-09-10T10:26:41.792320+0000","last_unstale":"2024-09-10T10:26:48.437419+0000","last_undegraded":"2024-09-10T10:26:48.437419+0000","last_fullsized":"2024-09-10T10:26:48.437419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436604+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:26:48.436604+0000","last_peered":"2024-09-10T10:26:48.436604+0000","last_clean":"2024-09-10T10:26:48.436604+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:26:48.436604+0000","last_undegraded":"2024-09-10T10:26:48.436604+0000","last_fullsized":"2024-09-10T10:26:48.436604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830936+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:26:47.830936+0000","last_peered":"2024-09-10T10:26:47.830936+0000","last_clean":"2024-09-10T10:26:47.830936+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:26:47.830936+0000","last_undegraded":"2024-09-10T10:26:47.830936+0000","last_fullsized":"2024-09-10T10:26:47.830936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829378+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:26:47.829378+0000","last_peered":"2024-09-10T10:26:47.829378+0000","last_clean":"2024-09-10T10:26:47.829378+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:26:47.829378+0000","last_undegraded":"2024-09-10T10:26:47.829378+0000","last_fullsized":"2024-09-10T10:26:47.829378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436127+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:26:48.436127+0000","last_peered":"2024-09-10T10:26:48.436127+0000","last_clean":"2024-09-10T10:26:48.436127+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:26:48.436127+0000","last_undegraded":"2024-09-10T10:26:48.436127+0000","last_fullsized":"2024-09-10T10:26:48.436127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271505+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:26:48.271505+0000","last_peered":"2024-09-10T10:26:48.271505+0000","last_clean":"2024-09-10T10:26:48.271505+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:26:48.271505+0000","last_undegraded":"2024-09-10T10:26:48.271505+0000","last_fullsized":"2024-09-10T10:26:48.271505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826694+0000","last_change":"2024-09-10T10:26:41.802117+0000","last_active":"2024-09-10T10:26:47.826694+0000","last_peered":"2024-09-10T10:26:47.826694+0000","last_clean":"2024-09-10T10:26:47.826694+0000","last_became_active":"2024-09-10T10:26:41.801798+0000","last_became_peered":"2024-09-10T10:26:41.801798+0000","last_unstale":"2024-09-10T10:26:47.826694+0000","last_undegraded":"2024-09-10T10:26:47.826694+0000","last_fullsized":"2024-09-10T10:26:47.826694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831168+0000","last_change":"2024-09-10T10:26:41.791156+0000","last_active":"2024-09-10T10:26:47.831168+0000","last_peered":"2024-09-10T10:26:47.831168+0000","last_clean":"2024-09-10T10:26:47.831168+0000","last_became_active":"2024-09-10T10:26:41.790893+0000","last_became_peered":"2024-09-10T10:26:41.790893+0000","last_unstale":"2024-09-10T10:26:47.831168+0000","last_undegraded":"2024-09-10T10:26:47.831168+0000","last_fullsized":"2024-09-10T10:26:47.831168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:46.923082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437194+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:26:48.437194+0000","last_peered":"2024-09-10T10:26:48.437194+0000","last_clean":"2024-09-10T10:26:48.437194+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:26:48.437194+0000","last_undegraded":"2024-09-10T10:26:48.437194+0000","last_fullsized":"2024-09-10T10:26:48.437194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437640+0000","last_change":"2024-09-10T10:26:41.795268+0000","last_active":"2024-09-10T10:26:48.437640+0000","last_peered":"2024-09-10T10:26:48.437640+0000","last_clean":"2024-09-10T10:26:48.437640+0000","last_became_active":"2024-09-10T10:26:41.794772+0000","last_became_peered":"2024-09-10T10:26:41.794772+0000","last_unstale":"2024-09-10T10:26:48.437640+0000","last_undegraded":"2024-09-10T10:26:48.437640+0000","last_fullsized":"2024-09-10T10:26:48.437640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437511+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:26:48.437511+0000","last_peered":"2024-09-10T10:26:48.437511+0000","last_clean":"2024-09-10T10:26:48.437511+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:26:48.437511+0000","last_undegraded":"2024-09-10T10:26:48.437511+0000","last_fullsized":"2024-09-10T10:26:48.437511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826362+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:26:47.826362+0000","last_peered":"2024-09-10T10:26:47.826362+0000","last_clean":"2024-09-10T10:26:47.826362+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:26:47.826362+0000","last_undegraded":"2024-09-10T10:26:47.826362+0000","last_fullsized":"2024-09-10T10:26:47.826362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436935+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:26:48.436935+0000","last_peered":"2024-09-10T10:26:48.436935+0000","last_clean":"2024-09-10T10:26:48.436935+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:26:48.436935+0000","last_undegraded":"2024-09-10T10:26:48.436935+0000","last_fullsized":"2024-09-10T10:26:48.436935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829829+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:26:47.829829+0000","last_peered":"2024-09-10T10:26:47.829829+0000","last_clean":"2024-09-10T10:26:47.829829+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:26:47.829829+0000","last_undegraded":"2024-09-10T10:26:47.829829+0000","last_fullsized":"2024-09-10T10:26:47.829829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436704+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:26:48.436704+0000","last_peered":"2024-09-10T10:26:48.436704+0000","last_clean":"2024-09-10T10:26:48.436704+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:26:48.436704+0000","last_undegraded":"2024-09-10T10:26:48.436704+0000","last_fullsized":"2024-09-10T10:26:48.436704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829829+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:26:47.829829+0000","last_peered":"2024-09-10T10:26:47.829829+0000","last_clean":"2024-09-10T10:26:47.829829+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:26:47.829829+0000","last_undegraded":"2024-09-10T10:26:47.829829+0000","last_fullsized":"2024-09-10T10:26:47.829829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271530+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:26:48.271530+0000","last_peered":"2024-09-10T10:26:48.271530+0000","last_clean":"2024-09-10T10:26:48.271530+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:26:48.271530+0000","last_undegraded":"2024-09-10T10:26:48.271530+0000","last_fullsized":"2024-09-10T10:26:48.271530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436873+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:26:48.436873+0000","last_peered":"2024-09-10T10:26:48.436873+0000","last_clean":"2024-09-10T10:26:48.436873+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:26:48.436873+0000","last_undegraded":"2024-09-10T10:26:48.436873+0000","last_fullsized":"2024-09-10T10:26:48.436873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437504+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:26:48.437504+0000","last_peered":"2024-09-10T10:26:48.437504+0000","last_clean":"2024-09-10T10:26:48.437504+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:26:48.437504+0000","last_undegraded":"2024-09-10T10:26:48.437504+0000","last_fullsized":"2024-09-10T10:26:48.437504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":348160,"data_stored":703014,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":115908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":115908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":356352,"data_stored":703070,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:27:06.806 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:27:06.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.937+0000 7f67338ac640 1 Processor -- start 2024-09-10T10:27:06.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.938+0000 7f67338ac640 1 -- start start 2024-09-10T10:27:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.938+0000 7f67338ac640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f672c070950 0x7f672c0c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.938+0000 7f67338ac640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672c0c33c0 con 0x7f672c0c8c00 2024-09-10T10:27:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.939+0000 7f67338ac640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672c0c36a0 con 0x7f672c1528e0 2024-09-10T10:27:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.939+0000 7f67338ac640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f672c1596d0 con 0x7f672c070950 2024-09-10T10:27:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.939+0000 7f6730e47640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f672c070950 0x7f672c0c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.939+0000 7f6731e49640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f672c0c8c00 0x7f672c0c93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:48518/0 (socket says 172.21.15.120:48518) 2024-09-10T10:27:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.939+0000 7f6731e49640 1 -- 172.21.15.120:0/88558977 learned_addr learned my addr 172.21.15.120:0/88558977 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.939+0000 7f6730e47640 1 -- 172.21.15.120:0/88558977 >> v1:172.21.15.134:6789/0 conn(0x7f672c1528e0 legacy=0x7f672c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.939+0000 7f6730e47640 1 -- 172.21.15.120:0/88558977 >> v1:172.21.15.120:6789/0 conn(0x7f672c0c8c00 legacy=0x7f672c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.939+0000 7f6730e47640 1 -- 172.21.15.120:0/88558977 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c1587c0 con 0x7f672c070950 2024-09-10T10:27:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.939+0000 7f672bfff640 1 -- 172.21.15.120:0/88558977 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 492293350 0 0) 0x7f672c0c33c0 con 0x7f672c0c8c00 2024-09-10T10:27:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f6730e47640 1 --2- 172.21.15.120:0/88558977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 0x7f672c0c4fa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67200097a0 tx=0x7f672002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2455ae78d90cc0e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f672bfff640 1 -- 172.21.15.120:0/88558977 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720037070 con 0x7f672c070950 2024-09-10T10:27:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f672bfff640 1 -- 172.21.15.120:0/88558977 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6720032640 con 0x7f672c070950 2024-09-10T10:27:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f672bfff640 1 -- 172.21.15.120:0/88558977 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720032a10 con 0x7f672c070950 2024-09-10T10:27:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f67338ac640 1 -- 172.21.15.120:0/88558977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 msgr2=0x7f672c0c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f67338ac640 1 --2- 172.21.15.120:0/88558977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 0x7f672c0c4fa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67200097a0 tx=0x7f672002a2c0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f67338ac640 1 -- 172.21.15.120:0/88558977 shutdown_connections 2024-09-10T10:27:06.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f67338ac640 1 --2- 172.21.15.120:0/88558977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 0x7f672c0c4fa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f67338ac640 1 -- 172.21.15.120:0/88558977 >> 172.21.15.120:0/88558977 conn(0x7f672c067950 msgr2=0x7f672c067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:06.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f67338ac640 1 -- 172.21.15.120:0/88558977 shutdown_connections 2024-09-10T10:27:06.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.940+0000 7f67338ac640 1 -- 172.21.15.120:0/88558977 wait complete. 2024-09-10T10:27:06.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.941+0000 7f67338ac640 1 Processor -- start 2024-09-10T10:27:06.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.941+0000 7f67338ac640 1 -- start start 2024-09-10T10:27:06.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f67338ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 0x7f672c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f67338ac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f672c0c8c00 0x7f672c14ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f67338ac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f672c1528e0 0x7f672c1ee090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f67338ac640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f672c15bca0 con 0x7f672c070950 2024-09-10T10:27:06.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f67338ac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f672c0c3980 con 0x7f672c1528e0 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f67338ac640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f672c1ee990 con 0x7f672c0c8c00 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f6731648640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 0x7f672c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f6731648640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 0x7f672c0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34978/0 (socket says 172.21.15.120:34978) 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f6731648640 1 -- 172.21.15.120:0/1751735922 learned_addr learned my addr 172.21.15.120:0/1751735922 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f6731e49640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f672c1528e0 0x7f672c1ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f6730e47640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f672c0c8c00 0x7f672c14ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f6731648640 1 -- 172.21.15.120:0/1751735922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f672c0c8c00 msgr2=0x7f672c14ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f6731648640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f672c0c8c00 0x7f672c14ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f6731648640 1 -- 172.21.15.120:0/1751735922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f672c1528e0 msgr2=0x7f672c1ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.942+0000 7f6731648640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f672c1528e0 0x7f672c1ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.943+0000 7f6731648640 1 -- 172.21.15.120:0/1751735922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c1eec70 con 0x7f672c070950 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.943+0000 7f6730e47640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f672c0c8c00 0x7f672c14ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.943+0000 7f6731648640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 0x7f672c0b1160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f672400c430 tx=0x7f672400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.943+0000 7f6729ffb640 1 -- 172.21.15.120:0/1751735922 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724017070 con 0x7f672c070950 2024-09-10T10:27:06.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.943+0000 7f6729ffb640 1 -- 172.21.15.120:0/1751735922 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67240052d0 con 0x7f672c070950 2024-09-10T10:27:06.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.943+0000 7f67338ac640 1 -- 172.21.15.120:0/1751735922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672c1eef50 con 0x7f672c070950 2024-09-10T10:27:06.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.943+0000 7f6729ffb640 1 -- 172.21.15.120:0/1751735922 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724003830 con 0x7f672c070950 2024-09-10T10:27:06.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.943+0000 7f67338ac640 1 -- 172.21.15.120:0/1751735922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672c0b0960 con 0x7f672c070950 2024-09-10T10:27:06.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.944+0000 7f67338ac640 1 -- 172.21.15.120:0/1751735922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f672c0c4fa0 con 0x7f672c070950 2024-09-10T10:27:06.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.944+0000 7f6729ffb640 1 -- 172.21.15.120:0/1751735922 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f672401a070 con 0x7f672c070950 2024-09-10T10:27:06.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.945+0000 7f6729ffb640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f671806be60 0x7f671806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:06.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.945+0000 7f6729ffb640 1 -- 172.21.15.120:0/1751735922 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f672408a460 con 0x7f672c070950 2024-09-10T10:27:06.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.945+0000 7f6730e47640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f671806be60 0x7f671806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:06.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.945+0000 7f6730e47640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f671806be60 0x7f671806e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6720034000 tx=0x7f672002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:06.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:06.949+0000 7f6729ffb640 1 -- 172.21.15.120:0/1751735922 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6724058480 con 0x7f672c070950 2024-09-10T10:27:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.111+0000 7f67338ac640 1 -- 172.21.15.120:0/1751735922 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f672c0cb130 con 0x7f671806be60 2024-09-10T10:27:07.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.118+0000 7f6729ffb640 1 -- 172.21.15.120:0/1751735922 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332611 (secure 0 0 0) 0x7f672c0cb130 con 0x7f671806be60 2024-09-10T10:27:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:07.125 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:27:07.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 -- 172.21.15.120:0/1751735922 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f671806be60 msgr2=0x7f671806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f671806be60 0x7f671806e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6720034000 tx=0x7f672002e000 comp rx=0 tx=0).stop 2024-09-10T10:27:07.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 -- 172.21.15.120:0/1751735922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 msgr2=0x7f672c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 0x7f672c0b1160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f672400c430 tx=0x7f672400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 -- 172.21.15.120:0/1751735922 shutdown_connections 2024-09-10T10:27:07.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f671806be60 0x7f671806e320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f672c1528e0 0x7f672c1ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f672c0c8c00 0x7f672c14ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 --2- 172.21.15.120:0/1751735922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f672c070950 0x7f672c0b1160 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 -- 172.21.15.120:0/1751735922 >> 172.21.15.120:0/1751735922 conn(0x7f672c067950 msgr2=0x7f672c157ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:07.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 -- 172.21.15.120:0/1751735922 shutdown_connections 2024-09-10T10:27:07.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.122+0000 7f67338ac640 1 -- 172.21.15.120:0/1751735922 wait complete. 2024-09-10T10:27:07.134 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-10T10:27:06.095174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218616,"kb_used_data":2368,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756104,"statfs":{"total":773094113280,"available":772870250496,"internally_reserved":0,"allocated":2424832,"data_stored":2086532,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002745"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271924+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:26:48.271924+0000","last_peered":"2024-09-10T10:26:48.271924+0000","last_clean":"2024-09-10T10:26:48.271924+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:26:48.271924+0000","last_undegraded":"2024-09-10T10:26:48.271924+0000","last_fullsized":"2024-09-10T10:26:48.271924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827265+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:26:47.827265+0000","last_peered":"2024-09-10T10:26:47.827265+0000","last_clean":"2024-09-10T10:26:47.827265+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:26:47.827265+0000","last_undegraded":"2024-09-10T10:26:47.827265+0000","last_fullsized":"2024-09-10T10:26:47.827265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826403+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:26:47.826403+0000","last_peered":"2024-09-10T10:26:47.826403+0000","last_clean":"2024-09-10T10:26:47.826403+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:26:47.826403+0000","last_undegraded":"2024-09-10T10:26:47.826403+0000","last_fullsized":"2024-09-10T10:26:47.826403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827328+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:26:47.827328+0000","last_peered":"2024-09-10T10:26:47.827328+0000","last_clean":"2024-09-10T10:26:47.827328+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:26:47.827328+0000","last_undegraded":"2024-09-10T10:26:47.827328+0000","last_fullsized":"2024-09-10T10:26:47.827328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271728+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:26:48.271728+0000","last_peered":"2024-09-10T10:26:48.271728+0000","last_clean":"2024-09-10T10:26:48.271728+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:26:48.271728+0000","last_undegraded":"2024-09-10T10:26:48.271728+0000","last_fullsized":"2024-09-10T10:26:48.271728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829940+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:26:47.829940+0000","last_peered":"2024-09-10T10:26:47.829940+0000","last_clean":"2024-09-10T10:26:47.829940+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:26:47.829940+0000","last_undegraded":"2024-09-10T10:26:47.829940+0000","last_fullsized":"2024-09-10T10:26:47.829940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437245+0000","last_change":"2024-09-10T10:26:40.796337+0000","last_active":"2024-09-10T10:26:48.437245+0000","last_peered":"2024-09-10T10:26:48.437245+0000","last_clean":"2024-09-10T10:26:48.437245+0000","last_became_active":"2024-09-10T10:26:40.796029+0000","last_became_peered":"2024-09-10T10:26:40.796029+0000","last_unstale":"2024-09-10T10:26:48.437245+0000","last_undegraded":"2024-09-10T10:26:48.437245+0000","last_fullsized":"2024-09-10T10:26:48.437245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829990+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:26:47.829990+0000","last_peered":"2024-09-10T10:26:47.829990+0000","last_clean":"2024-09-10T10:26:47.829990+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:26:47.829990+0000","last_undegraded":"2024-09-10T10:26:47.829990+0000","last_fullsized":"2024-09-10T10:26:47.829990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831277+0000","last_change":"2024-09-10T10:26:45.814368+0000","last_active":"2024-09-10T10:26:47.831277+0000","last_peered":"2024-09-10T10:26:47.831277+0000","last_clean":"2024-09-10T10:26:47.831277+0000","last_became_active":"2024-09-10T10:26:45.813953+0000","last_became_peered":"2024-09-10T10:26:45.813953+0000","last_unstale":"2024-09-10T10:26:47.831277+0000","last_undegraded":"2024-09-10T10:26:47.831277+0000","last_fullsized":"2024-09-10T10:26:47.831277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436841+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:26:48.436841+0000","last_peered":"2024-09-10T10:26:48.436841+0000","last_clean":"2024-09-10T10:26:48.436841+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:26:48.436841+0000","last_undegraded":"2024-09-10T10:26:48.436841+0000","last_fullsized":"2024-09-10T10:26:48.436841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830425+0000","last_change":"2024-09-10T10:26:41.789145+0000","last_active":"2024-09-10T10:26:47.830425+0000","last_peered":"2024-09-10T10:26:47.830425+0000","last_clean":"2024-09-10T10:26:47.830425+0000","last_became_active":"2024-09-10T10:26:41.788874+0000","last_became_peered":"2024-09-10T10:26:41.788874+0000","last_unstale":"2024-09-10T10:26:47.830425+0000","last_undegraded":"2024-09-10T10:26:47.830425+0000","last_fullsized":"2024-09-10T10:26:47.830425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830738+0000","last_change":"2024-09-10T10:26:45.816587+0000","last_active":"2024-09-10T10:26:47.830738+0000","last_peered":"2024-09-10T10:26:47.830738+0000","last_clean":"2024-09-10T10:26:47.830738+0000","last_became_active":"2024-09-10T10:26:45.816138+0000","last_became_peered":"2024-09-10T10:26:45.816138+0000","last_unstale":"2024-09-10T10:26:47.830738+0000","last_undegraded":"2024-09-10T10:26:47.830738+0000","last_fullsized":"2024-09-10T10:26:47.830738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436252+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:26:48.436252+0000","last_peered":"2024-09-10T10:26:48.436252+0000","last_clean":"2024-09-10T10:26:48.436252+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:26:48.436252+0000","last_undegraded":"2024-09-10T10:26:48.436252+0000","last_fullsized":"2024-09-10T10:26:48.436252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436878+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:26:48.436878+0000","last_peered":"2024-09-10T10:26:48.436878+0000","last_clean":"2024-09-10T10:26:48.436878+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:26:48.436878+0000","last_undegraded":"2024-09-10T10:26:48.436878+0000","last_fullsized":"2024-09-10T10:26:48.436878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831602+0000","last_change":"2024-09-10T10:26:45.816967+0000","last_active":"2024-09-10T10:26:47.831602+0000","last_peered":"2024-09-10T10:26:47.831602+0000","last_clean":"2024-09-10T10:26:47.831602+0000","last_became_active":"2024-09-10T10:26:45.816161+0000","last_became_peered":"2024-09-10T10:26:45.816161+0000","last_unstale":"2024-09-10T10:26:47.831602+0000","last_undegraded":"2024-09-10T10:26:47.831602+0000","last_fullsized":"2024-09-10T10:26:47.831602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826732+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:26:47.826732+0000","last_peered":"2024-09-10T10:26:47.826732+0000","last_clean":"2024-09-10T10:26:47.826732+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:26:47.826732+0000","last_undegraded":"2024-09-10T10:26:47.826732+0000","last_fullsized":"2024-09-10T10:26:47.826732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436716+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:26:48.436716+0000","last_peered":"2024-09-10T10:26:48.436716+0000","last_clean":"2024-09-10T10:26:48.436716+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:26:48.436716+0000","last_undegraded":"2024-09-10T10:26:48.436716+0000","last_fullsized":"2024-09-10T10:26:48.436716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830916+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:26:47.830916+0000","last_peered":"2024-09-10T10:26:47.830916+0000","last_clean":"2024-09-10T10:26:47.830916+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:26:47.830916+0000","last_undegraded":"2024-09-10T10:26:47.830916+0000","last_fullsized":"2024-09-10T10:26:47.830916+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826532+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:26:47.826532+0000","last_peered":"2024-09-10T10:26:47.826532+0000","last_clean":"2024-09-10T10:26:47.826532+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:26:47.826532+0000","last_undegraded":"2024-09-10T10:26:47.826532+0000","last_fullsized":"2024-09-10T10:26:47.826532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436164+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:26:48.436164+0000","last_peered":"2024-09-10T10:26:48.436164+0000","last_clean":"2024-09-10T10:26:48.436164+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:26:48.436164+0000","last_undegraded":"2024-09-10T10:26:48.436164+0000","last_fullsized":"2024-09-10T10:26:48.436164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436960+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:26:48.436960+0000","last_peered":"2024-09-10T10:26:48.436960+0000","last_clean":"2024-09-10T10:26:48.436960+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:26:48.436960+0000","last_undegraded":"2024-09-10T10:26:48.436960+0000","last_fullsized":"2024-09-10T10:26:48.436960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830283+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:26:47.830283+0000","last_peered":"2024-09-10T10:26:47.830283+0000","last_clean":"2024-09-10T10:26:47.830283+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:26:47.830283+0000","last_undegraded":"2024-09-10T10:26:47.830283+0000","last_fullsized":"2024-09-10T10:26:47.830283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436570+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:26:48.436570+0000","last_peered":"2024-09-10T10:26:48.436570+0000","last_clean":"2024-09-10T10:26:48.436570+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:26:48.436570+0000","last_undegraded":"2024-09-10T10:26:48.436570+0000","last_fullsized":"2024-09-10T10:26:48.436570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830167+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:26:47.830167+0000","last_peered":"2024-09-10T10:26:47.830167+0000","last_clean":"2024-09-10T10:26:47.830167+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:26:47.830167+0000","last_undegraded":"2024-09-10T10:26:47.830167+0000","last_fullsized":"2024-09-10T10:26:47.830167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271909+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:26:48.271909+0000","last_peered":"2024-09-10T10:26:48.271909+0000","last_clean":"2024-09-10T10:26:48.271909+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:26:48.271909+0000","last_undegraded":"2024-09-10T10:26:48.271909+0000","last_fullsized":"2024-09-10T10:26:48.271909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437755+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:26:48.437755+0000","last_peered":"2024-09-10T10:26:48.437755+0000","last_clean":"2024-09-10T10:26:48.437755+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:26:48.437755+0000","last_undegraded":"2024-09-10T10:26:48.437755+0000","last_fullsized":"2024-09-10T10:26:48.437755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829937+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:26:47.829937+0000","last_peered":"2024-09-10T10:26:47.829937+0000","last_clean":"2024-09-10T10:26:47.829937+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:26:47.829937+0000","last_undegraded":"2024-09-10T10:26:47.829937+0000","last_fullsized":"2024-09-10T10:26:47.829937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827469+0000","last_change":"2024-09-10T10:26:41.795085+0000","last_active":"2024-09-10T10:26:47.827469+0000","last_peered":"2024-09-10T10:26:47.827469+0000","last_clean":"2024-09-10T10:26:47.827469+0000","last_became_active":"2024-09-10T10:26:41.794819+0000","last_became_peered":"2024-09-10T10:26:41.794819+0000","last_unstale":"2024-09-10T10:26:47.827469+0000","last_undegraded":"2024-09-10T10:26:47.827469+0000","last_fullsized":"2024-09-10T10:26:47.827469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436983+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:26:48.436983+0000","last_peered":"2024-09-10T10:26:48.436983+0000","last_clean":"2024-09-10T10:26:48.436983+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:26:48.436983+0000","last_undegraded":"2024-09-10T10:26:48.436983+0000","last_fullsized":"2024-09-10T10:26:48.436983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826343+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:26:47.826343+0000","last_peered":"2024-09-10T10:26:47.826343+0000","last_clean":"2024-09-10T10:26:47.826343+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:26:47.826343+0000","last_undegraded":"2024-09-10T10:26:47.826343+0000","last_fullsized":"2024-09-10T10:26:47.826343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436707+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:26:48.436707+0000","last_peered":"2024-09-10T10:26:48.436707+0000","last_clean":"2024-09-10T10:26:48.436707+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:26:48.436707+0000","last_undegraded":"2024-09-10T10:26:48.436707+0000","last_fullsized":"2024-09-10T10:26:48.436707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437399+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:26:48.437399+0000","last_peered":"2024-09-10T10:26:48.437399+0000","last_clean":"2024-09-10T10:26:48.437399+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:26:48.437399+0000","last_undegraded":"2024-09-10T10:26:48.437399+0000","last_fullsized":"2024-09-10T10:26:48.437399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271566+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:26:48.271566+0000","last_peered":"2024-09-10T10:26:48.271566+0000","last_clean":"2024-09-10T10:26:48.271566+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:26:48.271566+0000","last_undegraded":"2024-09-10T10:26:48.271566+0000","last_fullsized":"2024-09-10T10:26:48.271566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830633+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:26:47.830633+0000","last_peered":"2024-09-10T10:26:47.830633+0000","last_clean":"2024-09-10T10:26:47.830633+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:26:47.830633+0000","last_undegraded":"2024-09-10T10:26:47.830633+0000","last_fullsized":"2024-09-10T10:26:47.830633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436761+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:26:48.436761+0000","last_peered":"2024-09-10T10:26:48.436761+0000","last_clean":"2024-09-10T10:26:48.436761+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:26:48.436761+0000","last_undegraded":"2024-09-10T10:26:48.436761+0000","last_fullsized":"2024-09-10T10:26:48.436761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436477+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:26:48.436477+0000","last_peered":"2024-09-10T10:26:48.436477+0000","last_clean":"2024-09-10T10:26:48.436477+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:26:48.436477+0000","last_undegraded":"2024-09-10T10:26:48.436477+0000","last_fullsized":"2024-09-10T10:26:48.436477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831395+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:26:47.831395+0000","last_peered":"2024-09-10T10:26:47.831395+0000","last_clean":"2024-09-10T10:26:47.831395+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:26:47.831395+0000","last_undegraded":"2024-09-10T10:26:47.831395+0000","last_fullsized":"2024-09-10T10:26:47.831395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830140+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:26:47.830140+0000","last_peered":"2024-09-10T10:26:47.830140+0000","last_clean":"2024-09-10T10:26:47.830140+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:26:47.830140+0000","last_undegraded":"2024-09-10T10:26:47.830140+0000","last_fullsized":"2024-09-10T10:26:47.830140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436656+0000","last_change":"2024-09-10T10:26:45.814008+0000","last_active":"2024-09-10T10:26:48.436656+0000","last_peered":"2024-09-10T10:26:48.436656+0000","last_clean":"2024-09-10T10:26:48.436656+0000","last_became_active":"2024-09-10T10:26:45.813627+0000","last_became_peered":"2024-09-10T10:26:45.813627+0000","last_unstale":"2024-09-10T10:26:48.436656+0000","last_undegraded":"2024-09-10T10:26:48.436656+0000","last_fullsized":"2024-09-10T10:26:48.436656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436224+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:26:48.436224+0000","last_peered":"2024-09-10T10:26:48.436224+0000","last_clean":"2024-09-10T10:26:48.436224+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:26:48.436224+0000","last_undegraded":"2024-09-10T10:26:48.436224+0000","last_fullsized":"2024-09-10T10:26:48.436224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436034+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:26:48.436034+0000","last_peered":"2024-09-10T10:26:48.436034+0000","last_clean":"2024-09-10T10:26:48.436034+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:26:48.436034+0000","last_undegraded":"2024-09-10T10:26:48.436034+0000","last_fullsized":"2024-09-10T10:26:48.436034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436987+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:26:48.436987+0000","last_peered":"2024-09-10T10:26:48.436987+0000","last_clean":"2024-09-10T10:26:48.436987+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:26:48.436987+0000","last_undegraded":"2024-09-10T10:26:48.436987+0000","last_fullsized":"2024-09-10T10:26:48.436987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271316+0000","last_change":"2024-09-10T10:26:41.799224+0000","last_active":"2024-09-10T10:26:48.271316+0000","last_peered":"2024-09-10T10:26:48.271316+0000","last_clean":"2024-09-10T10:26:48.271316+0000","last_became_active":"2024-09-10T10:26:41.798948+0000","last_became_peered":"2024-09-10T10:26:41.798948+0000","last_unstale":"2024-09-10T10:26:48.271316+0000","last_undegraded":"2024-09-10T10:26:48.271316+0000","last_fullsized":"2024-09-10T10:26:48.271316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436688+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:26:48.436688+0000","last_peered":"2024-09-10T10:26:48.436688+0000","last_clean":"2024-09-10T10:26:48.436688+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:26:48.436688+0000","last_undegraded":"2024-09-10T10:26:48.436688+0000","last_fullsized":"2024-09-10T10:26:48.436688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271901+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:26:48.271901+0000","last_peered":"2024-09-10T10:26:48.271901+0000","last_clean":"2024-09-10T10:26:48.271901+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:26:48.271901+0000","last_undegraded":"2024-09-10T10:26:48.271901+0000","last_fullsized":"2024-09-10T10:26:48.271901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826634+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:26:47.826634+0000","last_peered":"2024-09-10T10:26:47.826634+0000","last_clean":"2024-09-10T10:26:47.826634+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:26:47.826634+0000","last_undegraded":"2024-09-10T10:26:47.826634+0000","last_fullsized":"2024-09-10T10:26:47.826634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830090+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:26:47.830090+0000","last_peered":"2024-09-10T10:26:47.830090+0000","last_clean":"2024-09-10T10:26:47.830090+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:26:47.830090+0000","last_undegraded":"2024-09-10T10:26:47.830090+0000","last_fullsized":"2024-09-10T10:26:47.830090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437071+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:26:48.437071+0000","last_peered":"2024-09-10T10:26:48.437071+0000","last_clean":"2024-09-10T10:26:48.437071+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:26:48.437071+0000","last_undegraded":"2024-09-10T10:26:48.437071+0000","last_fullsized":"2024-09-10T10:26:48.437071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437061+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:26:48.437061+0000","last_peered":"2024-09-10T10:26:48.437061+0000","last_clean":"2024-09-10T10:26:48.437061+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:26:48.437061+0000","last_undegraded":"2024-09-10T10:26:48.437061+0000","last_fullsized":"2024-09-10T10:26:48.437061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437121+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:26:48.437121+0000","last_peered":"2024-09-10T10:26:48.437121+0000","last_clean":"2024-09-10T10:26:48.437121+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:26:48.437121+0000","last_undegraded":"2024-09-10T10:26:48.437121+0000","last_fullsized":"2024-09-10T10:26:48.437121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436307+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:26:48.436307+0000","last_peered":"2024-09-10T10:26:48.436307+0000","last_clean":"2024-09-10T10:26:48.436307+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:26:48.436307+0000","last_undegraded":"2024-09-10T10:26:48.436307+0000","last_fullsized":"2024-09-10T10:26:48.436307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436298+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:26:48.436298+0000","last_peered":"2024-09-10T10:26:48.436298+0000","last_clean":"2024-09-10T10:26:48.436298+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:26:48.436298+0000","last_undegraded":"2024-09-10T10:26:48.436298+0000","last_fullsized":"2024-09-10T10:26:48.436298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830079+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:26:47.830079+0000","last_peered":"2024-09-10T10:26:47.830079+0000","last_clean":"2024-09-10T10:26:47.830079+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:26:47.830079+0000","last_undegraded":"2024-09-10T10:26:47.830079+0000","last_fullsized":"2024-09-10T10:26:47.830079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436713+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:26:48.436713+0000","last_peered":"2024-09-10T10:26:48.436713+0000","last_clean":"2024-09-10T10:26:48.436713+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:26:48.436713+0000","last_undegraded":"2024-09-10T10:26:48.436713+0000","last_fullsized":"2024-09-10T10:26:48.436713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271787+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:26:48.271787+0000","last_peered":"2024-09-10T10:26:48.271787+0000","last_clean":"2024-09-10T10:26:48.271787+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:26:48.271787+0000","last_undegraded":"2024-09-10T10:26:48.271787+0000","last_fullsized":"2024-09-10T10:26:48.271787+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437650+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:26:48.437650+0000","last_peered":"2024-09-10T10:26:48.437650+0000","last_clean":"2024-09-10T10:26:48.437650+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:26:48.437650+0000","last_undegraded":"2024-09-10T10:26:48.437650+0000","last_fullsized":"2024-09-10T10:26:48.437650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436651+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:26:48.436651+0000","last_peered":"2024-09-10T10:26:48.436651+0000","last_clean":"2024-09-10T10:26:48.436651+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:26:48.436651+0000","last_undegraded":"2024-09-10T10:26:48.436651+0000","last_fullsized":"2024-09-10T10:26:48.436651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829425+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:26:47.829425+0000","last_peered":"2024-09-10T10:26:47.829425+0000","last_clean":"2024-09-10T10:26:47.829425+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:26:47.829425+0000","last_undegraded":"2024-09-10T10:26:47.829425+0000","last_fullsized":"2024-09-10T10:26:47.829425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437218+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:26:48.437218+0000","last_peered":"2024-09-10T10:26:48.437218+0000","last_clean":"2024-09-10T10:26:48.437218+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:26:48.437218+0000","last_undegraded":"2024-09-10T10:26:48.437218+0000","last_fullsized":"2024-09-10T10:26:48.437218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830430+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:26:47.830430+0000","last_peered":"2024-09-10T10:26:47.830430+0000","last_clean":"2024-09-10T10:26:47.830430+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:26:47.830430+0000","last_undegraded":"2024-09-10T10:26:47.830430+0000","last_fullsized":"2024-09-10T10:26:47.830430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829765+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:26:47.829765+0000","last_peered":"2024-09-10T10:26:47.829765+0000","last_clean":"2024-09-10T10:26:47.829765+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:26:47.829765+0000","last_undegraded":"2024-09-10T10:26:47.829765+0000","last_fullsized":"2024-09-10T10:26:47.829765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437037+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:26:48.437037+0000","last_peered":"2024-09-10T10:26:48.437037+0000","last_clean":"2024-09-10T10:26:48.437037+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:26:48.437037+0000","last_undegraded":"2024-09-10T10:26:48.437037+0000","last_fullsized":"2024-09-10T10:26:48.437037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.435841+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:26:48.435841+0000","last_peered":"2024-09-10T10:26:48.435841+0000","last_clean":"2024-09-10T10:26:48.435841+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:26:48.435841+0000","last_undegraded":"2024-09-10T10:26:48.435841+0000","last_fullsized":"2024-09-10T10:26:48.435841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271756+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:26:48.271756+0000","last_peered":"2024-09-10T10:26:48.271756+0000","last_clean":"2024-09-10T10:26:48.271756+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:26:48.271756+0000","last_undegraded":"2024-09-10T10:26:48.271756+0000","last_fullsized":"2024-09-10T10:26:48.271756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436218+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:26:48.436218+0000","last_peered":"2024-09-10T10:26:48.436218+0000","last_clean":"2024-09-10T10:26:48.436218+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:26:48.436218+0000","last_undegraded":"2024-09-10T10:26:48.436218+0000","last_fullsized":"2024-09-10T10:26:48.436218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436280+0000","last_change":"2024-09-10T10:26:40.794739+0000","last_active":"2024-09-10T10:26:48.436280+0000","last_peered":"2024-09-10T10:26:48.436280+0000","last_clean":"2024-09-10T10:26:48.436280+0000","last_became_active":"2024-09-10T10:26:40.794430+0000","last_became_peered":"2024-09-10T10:26:40.794430+0000","last_unstale":"2024-09-10T10:26:48.436280+0000","last_undegraded":"2024-09-10T10:26:48.436280+0000","last_fullsized":"2024-09-10T10:26:48.436280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827508+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:26:47.827508+0000","last_peered":"2024-09-10T10:26:47.827508+0000","last_clean":"2024-09-10T10:26:47.827508+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:26:47.827508+0000","last_undegraded":"2024-09-10T10:26:47.827508+0000","last_fullsized":"2024-09-10T10:26:47.827508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436803+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:26:48.436803+0000","last_peered":"2024-09-10T10:26:48.436803+0000","last_clean":"2024-09-10T10:26:48.436803+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:26:48.436803+0000","last_undegraded":"2024-09-10T10:26:48.436803+0000","last_fullsized":"2024-09-10T10:26:48.436803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830046+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:26:47.830046+0000","last_peered":"2024-09-10T10:26:47.830046+0000","last_clean":"2024-09-10T10:26:47.830046+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:26:47.830046+0000","last_undegraded":"2024-09-10T10:26:47.830046+0000","last_fullsized":"2024-09-10T10:26:47.830046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827010+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:26:47.827010+0000","last_peered":"2024-09-10T10:26:47.827010+0000","last_clean":"2024-09-10T10:26:47.827010+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:26:47.827010+0000","last_undegraded":"2024-09-10T10:26:47.827010+0000","last_fullsized":"2024-09-10T10:26:47.827010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271524+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:26:48.271524+0000","last_peered":"2024-09-10T10:26:48.271524+0000","last_clean":"2024-09-10T10:26:48.271524+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:26:48.271524+0000","last_undegraded":"2024-09-10T10:26:48.271524+0000","last_fullsized":"2024-09-10T10:26:48.271524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271879+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:26:48.271879+0000","last_peered":"2024-09-10T10:26:48.271879+0000","last_clean":"2024-09-10T10:26:48.271879+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:26:48.271879+0000","last_undegraded":"2024-09-10T10:26:48.271879+0000","last_fullsized":"2024-09-10T10:26:48.271879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830480+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:26:47.830480+0000","last_peered":"2024-09-10T10:26:47.830480+0000","last_clean":"2024-09-10T10:26:47.830480+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:26:47.830480+0000","last_undegraded":"2024-09-10T10:26:47.830480+0000","last_fullsized":"2024-09-10T10:26:47.830480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436991+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:26:48.436991+0000","last_peered":"2024-09-10T10:26:48.436991+0000","last_clean":"2024-09-10T10:26:48.436991+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:26:48.436991+0000","last_undegraded":"2024-09-10T10:26:48.436991+0000","last_fullsized":"2024-09-10T10:26:48.436991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:54.051970+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:26:54.051970+0000","last_peered":"2024-09-10T10:26:54.051970+0000","last_clean":"2024-09-10T10:26:54.051970+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:26:54.051970+0000","last_undegraded":"2024-09-10T10:26:54.051970+0000","last_fullsized":"2024-09-10T10:26:54.051970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436702+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:26:48.436702+0000","last_peered":"2024-09-10T10:26:48.436702+0000","last_clean":"2024-09-10T10:26:48.436702+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:26:48.436702+0000","last_undegraded":"2024-09-10T10:26:48.436702+0000","last_fullsized":"2024-09-10T10:26:48.436702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831260+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:26:47.831260+0000","last_peered":"2024-09-10T10:26:47.831260+0000","last_clean":"2024-09-10T10:26:47.831260+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:26:47.831260+0000","last_undegraded":"2024-09-10T10:26:47.831260+0000","last_fullsized":"2024-09-10T10:26:47.831260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436571+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:26:48.436571+0000","last_peered":"2024-09-10T10:26:48.436571+0000","last_clean":"2024-09-10T10:26:48.436571+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:26:48.436571+0000","last_undegraded":"2024-09-10T10:26:48.436571+0000","last_fullsized":"2024-09-10T10:26:48.436571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831349+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:26:47.831349+0000","last_peered":"2024-09-10T10:26:47.831349+0000","last_clean":"2024-09-10T10:26:47.831349+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:26:47.831349+0000","last_undegraded":"2024-09-10T10:26:47.831349+0000","last_fullsized":"2024-09-10T10:26:47.831349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271920+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:26:48.271920+0000","last_peered":"2024-09-10T10:26:48.271920+0000","last_clean":"2024-09-10T10:26:48.271920+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:26:48.271920+0000","last_undegraded":"2024-09-10T10:26:48.271920+0000","last_fullsized":"2024-09-10T10:26:48.271920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830756+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:26:47.830756+0000","last_peered":"2024-09-10T10:26:47.830756+0000","last_clean":"2024-09-10T10:26:47.830756+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:26:47.830756+0000","last_undegraded":"2024-09-10T10:26:47.830756+0000","last_fullsized":"2024-09-10T10:26:47.830756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826511+0000","last_change":"2024-09-10T10:26:45.819680+0000","last_active":"2024-09-10T10:26:47.826511+0000","last_peered":"2024-09-10T10:26:47.826511+0000","last_clean":"2024-09-10T10:26:47.826511+0000","last_became_active":"2024-09-10T10:26:45.819319+0000","last_became_peered":"2024-09-10T10:26:45.819319+0000","last_unstale":"2024-09-10T10:26:47.826511+0000","last_undegraded":"2024-09-10T10:26:47.826511+0000","last_fullsized":"2024-09-10T10:26:47.826511+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:43.492188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830718+0000","last_change":"2024-09-10T10:26:41.792618+0000","last_active":"2024-09-10T10:26:47.830718+0000","last_peered":"2024-09-10T10:26:47.830718+0000","last_clean":"2024-09-10T10:26:47.830718+0000","last_became_active":"2024-09-10T10:26:41.792323+0000","last_became_peered":"2024-09-10T10:26:41.792323+0000","last_unstale":"2024-09-10T10:26:47.830718+0000","last_undegraded":"2024-09-10T10:26:47.830718+0000","last_fullsized":"2024-09-10T10:26:47.830718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437639+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:26:48.437639+0000","last_peered":"2024-09-10T10:26:48.437639+0000","last_clean":"2024-09-10T10:26:48.437639+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:26:48.437639+0000","last_undegraded":"2024-09-10T10:26:48.437639+0000","last_fullsized":"2024-09-10T10:26:48.437639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826609+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:26:47.826609+0000","last_peered":"2024-09-10T10:26:47.826609+0000","last_clean":"2024-09-10T10:26:47.826609+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:26:47.826609+0000","last_undegraded":"2024-09-10T10:26:47.826609+0000","last_fullsized":"2024-09-10T10:26:47.826609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271720+0000","last_change":"2024-09-10T10:26:46.193656+0000","last_active":"2024-09-10T10:26:48.271720+0000","last_peered":"2024-09-10T10:26:48.271720+0000","last_clean":"2024-09-10T10:26:48.271720+0000","last_became_active":"2024-09-10T10:26:46.193109+0000","last_became_peered":"2024-09-10T10:26:46.193109+0000","last_unstale":"2024-09-10T10:26:48.271720+0000","last_undegraded":"2024-09-10T10:26:48.271720+0000","last_fullsized":"2024-09-10T10:26:48.271720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436035+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:26:48.436035+0000","last_peered":"2024-09-10T10:26:48.436035+0000","last_clean":"2024-09-10T10:26:48.436035+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:26:48.436035+0000","last_undegraded":"2024-09-10T10:26:48.436035+0000","last_fullsized":"2024-09-10T10:26:48.436035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829601+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:26:47.829601+0000","last_peered":"2024-09-10T10:26:47.829601+0000","last_clean":"2024-09-10T10:26:47.829601+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:26:47.829601+0000","last_undegraded":"2024-09-10T10:26:47.829601+0000","last_fullsized":"2024-09-10T10:26:47.829601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437648+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:26:48.437648+0000","last_peered":"2024-09-10T10:26:48.437648+0000","last_clean":"2024-09-10T10:26:48.437648+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:26:48.437648+0000","last_undegraded":"2024-09-10T10:26:48.437648+0000","last_fullsized":"2024-09-10T10:26:48.437648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826450+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:26:47.826450+0000","last_peered":"2024-09-10T10:26:47.826450+0000","last_clean":"2024-09-10T10:26:47.826450+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:26:47.826450+0000","last_undegraded":"2024-09-10T10:26:47.826450+0000","last_fullsized":"2024-09-10T10:26:47.826450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826672+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:26:47.826672+0000","last_peered":"2024-09-10T10:26:47.826672+0000","last_clean":"2024-09-10T10:26:47.826672+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:26:47.826672+0000","last_undegraded":"2024-09-10T10:26:47.826672+0000","last_fullsized":"2024-09-10T10:26:47.826672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436533+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:26:48.436533+0000","last_peered":"2024-09-10T10:26:48.436533+0000","last_clean":"2024-09-10T10:26:48.436533+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:26:48.436533+0000","last_undegraded":"2024-09-10T10:26:48.436533+0000","last_fullsized":"2024-09-10T10:26:48.436533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271816+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:26:48.271816+0000","last_peered":"2024-09-10T10:26:48.271816+0000","last_clean":"2024-09-10T10:26:48.271816+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:26:48.271816+0000","last_undegraded":"2024-09-10T10:26:48.271816+0000","last_fullsized":"2024-09-10T10:26:48.271816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436252+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:26:48.436252+0000","last_peered":"2024-09-10T10:26:48.436252+0000","last_clean":"2024-09-10T10:26:48.436252+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:26:48.436252+0000","last_undegraded":"2024-09-10T10:26:48.436252+0000","last_fullsized":"2024-09-10T10:26:48.436252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826785+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:26:47.826785+0000","last_peered":"2024-09-10T10:26:47.826785+0000","last_clean":"2024-09-10T10:26:47.826785+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:26:47.826785+0000","last_undegraded":"2024-09-10T10:26:47.826785+0000","last_fullsized":"2024-09-10T10:26:47.826785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830947+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:26:47.830947+0000","last_peered":"2024-09-10T10:26:47.830947+0000","last_clean":"2024-09-10T10:26:47.830947+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:26:47.830947+0000","last_undegraded":"2024-09-10T10:26:47.830947+0000","last_fullsized":"2024-09-10T10:26:47.830947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827120+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:26:47.827120+0000","last_peered":"2024-09-10T10:26:47.827120+0000","last_clean":"2024-09-10T10:26:47.827120+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:26:47.827120+0000","last_undegraded":"2024-09-10T10:26:47.827120+0000","last_fullsized":"2024-09-10T10:26:47.827120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436082+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:26:48.436082+0000","last_peered":"2024-09-10T10:26:48.436082+0000","last_clean":"2024-09-10T10:26:48.436082+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:26:48.436082+0000","last_undegraded":"2024-09-10T10:26:48.436082+0000","last_fullsized":"2024-09-10T10:26:48.436082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.435938+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:26:48.435938+0000","last_peered":"2024-09-10T10:26:48.435938+0000","last_clean":"2024-09-10T10:26:48.435938+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:26:48.435938+0000","last_undegraded":"2024-09-10T10:26:48.435938+0000","last_fullsized":"2024-09-10T10:26:48.435938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830004+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:26:47.830004+0000","last_peered":"2024-09-10T10:26:47.830004+0000","last_clean":"2024-09-10T10:26:47.830004+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:26:47.830004+0000","last_undegraded":"2024-09-10T10:26:47.830004+0000","last_fullsized":"2024-09-10T10:26:47.830004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271738+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:26:48.271738+0000","last_peered":"2024-09-10T10:26:48.271738+0000","last_clean":"2024-09-10T10:26:48.271738+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:26:48.271738+0000","last_undegraded":"2024-09-10T10:26:48.271738+0000","last_fullsized":"2024-09-10T10:26:48.271738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827172+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:26:47.827172+0000","last_peered":"2024-09-10T10:26:47.827172+0000","last_clean":"2024-09-10T10:26:47.827172+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:26:47.827172+0000","last_undegraded":"2024-09-10T10:26:47.827172+0000","last_fullsized":"2024-09-10T10:26:47.827172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436417+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:26:48.436417+0000","last_peered":"2024-09-10T10:26:48.436417+0000","last_clean":"2024-09-10T10:26:48.436417+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:26:48.436417+0000","last_undegraded":"2024-09-10T10:26:48.436417+0000","last_fullsized":"2024-09-10T10:26:48.436417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437644+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:26:48.437644+0000","last_peered":"2024-09-10T10:26:48.437644+0000","last_clean":"2024-09-10T10:26:48.437644+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:26:48.437644+0000","last_undegraded":"2024-09-10T10:26:48.437644+0000","last_fullsized":"2024-09-10T10:26:48.437644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827413+0000","last_change":"2024-09-10T10:26:41.802903+0000","last_active":"2024-09-10T10:26:47.827413+0000","last_peered":"2024-09-10T10:26:47.827413+0000","last_clean":"2024-09-10T10:26:47.827413+0000","last_became_active":"2024-09-10T10:26:41.802586+0000","last_became_peered":"2024-09-10T10:26:41.802586+0000","last_unstale":"2024-09-10T10:26:47.827413+0000","last_undegraded":"2024-09-10T10:26:47.827413+0000","last_fullsized":"2024-09-10T10:26:47.827413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829500+0000","last_change":"2024-09-10T10:26:41.800060+0000","last_active":"2024-09-10T10:26:47.829500+0000","last_peered":"2024-09-10T10:26:47.829500+0000","last_clean":"2024-09-10T10:26:47.829500+0000","last_became_active":"2024-09-10T10:26:41.799729+0000","last_became_peered":"2024-09-10T10:26:41.799729+0000","last_unstale":"2024-09-10T10:26:47.829500+0000","last_undegraded":"2024-09-10T10:26:47.829500+0000","last_fullsized":"2024-09-10T10:26:47.829500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826575+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:26:47.826575+0000","last_peered":"2024-09-10T10:26:47.826575+0000","last_clean":"2024-09-10T10:26:47.826575+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:26:47.826575+0000","last_undegraded":"2024-09-10T10:26:47.826575+0000","last_fullsized":"2024-09-10T10:26:47.826575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829948+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:26:47.829948+0000","last_peered":"2024-09-10T10:26:47.829948+0000","last_clean":"2024-09-10T10:26:47.829948+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:26:47.829948+0000","last_undegraded":"2024-09-10T10:26:47.829948+0000","last_fullsized":"2024-09-10T10:26:47.829948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271616+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:26:48.271616+0000","last_peered":"2024-09-10T10:26:48.271616+0000","last_clean":"2024-09-10T10:26:48.271616+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:26:48.271616+0000","last_undegraded":"2024-09-10T10:26:48.271616+0000","last_fullsized":"2024-09-10T10:26:48.271616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436780+0000","last_change":"2024-09-10T10:26:41.787435+0000","last_active":"2024-09-10T10:26:48.436780+0000","last_peered":"2024-09-10T10:26:48.436780+0000","last_clean":"2024-09-10T10:26:48.436780+0000","last_became_active":"2024-09-10T10:26:41.787083+0000","last_became_peered":"2024-09-10T10:26:41.787083+0000","last_unstale":"2024-09-10T10:26:48.436780+0000","last_undegraded":"2024-09-10T10:26:48.436780+0000","last_fullsized":"2024-09-10T10:26:48.436780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437419+0000","last_change":"2024-09-10T10:26:41.792587+0000","last_active":"2024-09-10T10:26:48.437419+0000","last_peered":"2024-09-10T10:26:48.437419+0000","last_clean":"2024-09-10T10:26:48.437419+0000","last_became_active":"2024-09-10T10:26:41.792320+0000","last_became_peered":"2024-09-10T10:26:41.792320+0000","last_unstale":"2024-09-10T10:26:48.437419+0000","last_undegraded":"2024-09-10T10:26:48.437419+0000","last_fullsized":"2024-09-10T10:26:48.437419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436604+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:26:48.436604+0000","last_peered":"2024-09-10T10:26:48.436604+0000","last_clean":"2024-09-10T10:26:48.436604+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:26:48.436604+0000","last_undegraded":"2024-09-10T10:26:48.436604+0000","last_fullsized":"2024-09-10T10:26:48.436604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830936+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:26:47.830936+0000","last_peered":"2024-09-10T10:26:47.830936+0000","last_clean":"2024-09-10T10:26:47.830936+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:26:47.830936+0000","last_undegraded":"2024-09-10T10:26:47.830936+0000","last_fullsized":"2024-09-10T10:26:47.830936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829378+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:26:47.829378+0000","last_peered":"2024-09-10T10:26:47.829378+0000","last_clean":"2024-09-10T10:26:47.829378+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:26:47.829378+0000","last_undegraded":"2024-09-10T10:26:47.829378+0000","last_fullsized":"2024-09-10T10:26:47.829378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436127+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:26:48.436127+0000","last_peered":"2024-09-10T10:26:48.436127+0000","last_clean":"2024-09-10T10:26:48.436127+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:26:48.436127+0000","last_undegraded":"2024-09-10T10:26:48.436127+0000","last_fullsized":"2024-09-10T10:26:48.436127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271505+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:26:48.271505+0000","last_peered":"2024-09-10T10:26:48.271505+0000","last_clean":"2024-09-10T10:26:48.271505+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:26:48.271505+0000","last_undegraded":"2024-09-10T10:26:48.271505+0000","last_fullsized":"2024-09-10T10:26:48.271505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826694+0000","last_change":"2024-09-10T10:26:41.802117+0000","last_active":"2024-09-10T10:26:47.826694+0000","last_peered":"2024-09-10T10:26:47.826694+0000","last_clean":"2024-09-10T10:26:47.826694+0000","last_became_active":"2024-09-10T10:26:41.801798+0000","last_became_peered":"2024-09-10T10:26:41.801798+0000","last_unstale":"2024-09-10T10:26:47.826694+0000","last_undegraded":"2024-09-10T10:26:47.826694+0000","last_fullsized":"2024-09-10T10:26:47.826694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831168+0000","last_change":"2024-09-10T10:26:41.791156+0000","last_active":"2024-09-10T10:26:47.831168+0000","last_peered":"2024-09-10T10:26:47.831168+0000","last_clean":"2024-09-10T10:26:47.831168+0000","last_became_active":"2024-09-10T10:26:41.790893+0000","last_became_peered":"2024-09-10T10:26:41.790893+0000","last_unstale":"2024-09-10T10:26:47.831168+0000","last_undegraded":"2024-09-10T10:26:47.831168+0000","last_fullsized":"2024-09-10T10:26:47.831168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:46.923082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437194+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:26:48.437194+0000","last_peered":"2024-09-10T10:26:48.437194+0000","last_clean":"2024-09-10T10:26:48.437194+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:26:48.437194+0000","last_undegraded":"2024-09-10T10:26:48.437194+0000","last_fullsized":"2024-09-10T10:26:48.437194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437640+0000","last_change":"2024-09-10T10:26:41.795268+0000","last_active":"2024-09-10T10:26:48.437640+0000","last_peered":"2024-09-10T10:26:48.437640+0000","last_clean":"2024-09-10T10:26:48.437640+0000","last_became_active":"2024-09-10T10:26:41.794772+0000","last_became_peered":"2024-09-10T10:26:41.794772+0000","last_unstale":"2024-09-10T10:26:48.437640+0000","last_undegraded":"2024-09-10T10:26:48.437640+0000","last_fullsized":"2024-09-10T10:26:48.437640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437511+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:26:48.437511+0000","last_peered":"2024-09-10T10:26:48.437511+0000","last_clean":"2024-09-10T10:26:48.437511+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:26:48.437511+0000","last_undegraded":"2024-09-10T10:26:48.437511+0000","last_fullsized":"2024-09-10T10:26:48.437511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826362+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:26:47.826362+0000","last_peered":"2024-09-10T10:26:47.826362+0000","last_clean":"2024-09-10T10:26:47.826362+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:26:47.826362+0000","last_undegraded":"2024-09-10T10:26:47.826362+0000","last_fullsized":"2024-09-10T10:26:47.826362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436935+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:26:48.436935+0000","last_peered":"2024-09-10T10:26:48.436935+0000","last_clean":"2024-09-10T10:26:48.436935+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:26:48.436935+0000","last_undegraded":"2024-09-10T10:26:48.436935+0000","last_fullsized":"2024-09-10T10:26:48.436935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829829+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:26:47.829829+0000","last_peered":"2024-09-10T10:26:47.829829+0000","last_clean":"2024-09-10T10:26:47.829829+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:26:47.829829+0000","last_undegraded":"2024-09-10T10:26:47.829829+0000","last_fullsized":"2024-09-10T10:26:47.829829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436704+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:26:48.436704+0000","last_peered":"2024-09-10T10:26:48.436704+0000","last_clean":"2024-09-10T10:26:48.436704+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:26:48.436704+0000","last_undegraded":"2024-09-10T10:26:48.436704+0000","last_fullsized":"2024-09-10T10:26:48.436704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829829+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:26:47.829829+0000","last_peered":"2024-09-10T10:26:47.829829+0000","last_clean":"2024-09-10T10:26:47.829829+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:26:47.829829+0000","last_undegraded":"2024-09-10T10:26:47.829829+0000","last_fullsized":"2024-09-10T10:26:47.829829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271530+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:26:48.271530+0000","last_peered":"2024-09-10T10:26:48.271530+0000","last_clean":"2024-09-10T10:26:48.271530+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:26:48.271530+0000","last_undegraded":"2024-09-10T10:26:48.271530+0000","last_fullsized":"2024-09-10T10:26:48.271530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436873+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:26:48.436873+0000","last_peered":"2024-09-10T10:26:48.436873+0000","last_clean":"2024-09-10T10:26:48.436873+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:26:48.436873+0000","last_undegraded":"2024-09-10T10:26:48.436873+0000","last_fullsized":"2024-09-10T10:26:48.436873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437504+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:26:48.437504+0000","last_peered":"2024-09-10T10:26:48.437504+0000","last_clean":"2024-09-10T10:26:48.437504+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:26:48.437504+0000","last_undegraded":"2024-09-10T10:26:48.437504+0000","last_fullsized":"2024-09-10T10:26:48.437504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":348160,"data_stored":703014,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":115908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":115908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":356352,"data_stored":703070,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:27:07.138 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:27:07.139 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T10:27:07.139 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T10:27:07.139 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T10:27:07.139 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T10:27:07.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.270+0000 7f9dbcfd6640 1 Processor -- start 2024-09-10T10:27:07.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.270+0000 7f9dbcfd6640 1 -- start start 2024-09-10T10:27:07.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.271+0000 7f9dbcfd6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9db8154e40 0x7f9db806cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.271+0000 7f9dbcfd6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db80c7b40 con 0x7f9db80c0760 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.271+0000 7f9dbcfd6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db80c8ba0 con 0x7f9db815ad60 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.271+0000 7f9dbcfd6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9db80c8e80 con 0x7f9db8154e40 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.271+0000 7f9db5d9b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f9db80c0760 0x7f9db806ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:48530/0 (socket says 172.21.15.120:48530) 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.271+0000 7f9db5d9b640 1 -- 172.21.15.120:0/1316044722 learned_addr learned my addr 172.21.15.120:0/1316044722 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db659c640 1 --2- 172.21.15.120:0/1316044722 >> v2:172.21.15.134:3300/0 conn(0x7f9db8154e40 0x7f9db806cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 911017421 0 0) 0x7f9db80c7b40 con 0x7f9db80c0760 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9da4003520 con 0x7f9db80c0760 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2458006011 0 0) 0x7f9da4003520 con 0x7f9db80c0760 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 >> v2:172.21.15.134:3300/0 conn(0x7f9db8154e40 msgr2=0x7f9db806cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 --2- 172.21.15.120:0/1316044722 >> v2:172.21.15.134:3300/0 conn(0x7f9db8154e40 0x7f9db806cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 >> v1:172.21.15.134:6789/0 conn(0x7f9db815ad60 legacy=0x7f9db815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db80c7e00 con 0x7f9db80c0760 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f9da0001670 con 0x7f9db80c0760 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 >> v1:172.21.15.120:6789/0 conn(0x7f9db80c0760 legacy=0x7f9db806ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9da40039b0 0x7f9da4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da4004800 0x7f9da4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da40051c0 0x7f9da400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9db80c8e80 con 0x7f9da40051c0 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9da400c060 con 0x7f9da4004800 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9da400c260 con 0x7f9da40039b0 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db6d9d640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da40051c0 0x7f9da400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db5d9b640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da4004800 0x7f9da4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db659c640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9da40039b0 0x7f9da4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db6d9d640 1 -- 172.21.15.120:0/1316044722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9da40039b0 msgr2=0x7f9da4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:07.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db6d9d640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9da40039b0 0x7f9da4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db6d9d640 1 -- 172.21.15.120:0/1316044722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da4004800 msgr2=0x7f9da4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db6d9d640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da4004800 0x7f9da4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.272+0000 7f9db6d9d640 1 -- 172.21.15.120:0/1316044722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da4003520 con 0x7f9da40051c0 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.273+0000 7f9db6d9d640 1 -- 172.21.15.120:0/1316044722 reap_dead start 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.273+0000 7f9db6d9d640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da40051c0 0x7f9da400b9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9da8001750 tx=0x7f9da4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb7eb6b68a4d102 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.273+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8001370 con 0x7f9da40051c0 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.273+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da80052f0 con 0x7f9da40051c0 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.273+0000 7f9db559a640 1 -- 172.21.15.120:0/1316044722 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da802abe0 con 0x7f9da40051c0 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.274+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/1316044722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da40051c0 msgr2=0x7f9da400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.274+0000 7f9dbcfd6640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da40051c0 0x7f9da400b9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9da8001750 tx=0x7f9da4005830 comp rx=0 tx=0).stop 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.274+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/1316044722 shutdown_connections 2024-09-10T10:27:07.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.274+0000 7f9dbcfd6640 1 --2- 172.21.15.120:0/1316044722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da40051c0 0x7f9da400b9c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.274+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/1316044722 >> 172.21.15.120:0/1316044722 conn(0x7f9db8067950 msgr2=0x7f9db8067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:07.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.274+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/1316044722 shutdown_connections 2024-09-10T10:27:07.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.274+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/1316044722 wait complete. 2024-09-10T10:27:07.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.275+0000 7f9dbcfd6640 1 Processor -- start 2024-09-10T10:27:07.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.275+0000 7f9dbcfd6640 1 -- start start 2024-09-10T10:27:07.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9dbcfd6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da40051c0 0x7f9db81f07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9dbcfd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9db80be6e0 0x7f9db80c1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9dbcfd6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9db80bde60 0x7f9db80bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9dbcfd6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9db80c8620 con 0x7f9db80be6e0 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9dbcfd6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9db806bd30 con 0x7f9da40051c0 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9dbcfd6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9db8155220 con 0x7f9db80bde60 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9db80be6e0 0x7f9db80c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9db80be6e0 0x7f9db80c1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34996/0 (socket says 172.21.15.120:34996) 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db5d9b640 1 -- 172.21.15.120:0/3031938372 learned_addr learned my addr 172.21.15.120:0/3031938372 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db6d9d640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9db80bde60 0x7f9db80bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db659c640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da40051c0 0x7f9db81f07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db5d9b640 1 -- 172.21.15.120:0/3031938372 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9db80bde60 msgr2=0x7f9db80bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db5d9b640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9db80bde60 0x7f9db80bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db5d9b640 1 -- 172.21.15.120:0/3031938372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da40051c0 msgr2=0x7f9db81f07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db5d9b640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da40051c0 0x7f9db81f07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db5d9b640 1 -- 172.21.15.120:0/3031938372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db80c0500 con 0x7f9db80be6e0 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db6d9d640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9db80bde60 0x7f9db80bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.276+0000 7f9db5d9b640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9db80be6e0 0x7f9db80c1170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9da0007370 tx=0x7f9da0007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.277+0000 7f9daeffd640 1 -- 172.21.15.120:0/3031938372 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da001a070 con 0x7f9db80be6e0 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.277+0000 7f9daeffd640 1 -- 172.21.15.120:0/3031938372 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da000f940 con 0x7f9db80be6e0 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.277+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/3031938372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db81f0de0 con 0x7f9db80be6e0 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.277+0000 7f9daeffd640 1 -- 172.21.15.120:0/3031938372 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da000fd50 con 0x7f9db80be6e0 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.277+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/3031938372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db81f13a0 con 0x7f9db80be6e0 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.278+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/3031938372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db815bd50 con 0x7f9db80be6e0 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.278+0000 7f9daeffd640 1 -- 172.21.15.120:0/3031938372 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9da001d070 con 0x7f9db80be6e0 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.278+0000 7f9daeffd640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9d9806be60 0x7f9d9806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.279+0000 7f9daeffd640 1 -- 172.21.15.120:0/3031938372 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9da008aa20 con 0x7f9db80be6e0 2024-09-10T10:27:07.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.279+0000 7f9db659c640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9d9806be60 0x7f9d9806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.279+0000 7f9db659c640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9d9806be60 0x7f9d9806e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9db0014450 tx=0x7f9db000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:07.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.283+0000 7f9daeffd640 1 -- 172.21.15.120:0/3031938372 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9db815bd50 con 0x7f9db80be6e0 2024-09-10T10:27:07.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.491+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/3031938372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9db806beb0 con 0x7f9db80be6e0 2024-09-10T10:27:07.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.493+0000 7f9daeffd640 1 -- 172.21.15.120:0/3031938372 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172945 (secure 0 0 0) 0x7f9db806beb0 con 0x7f9db80be6e0 2024-09-10T10:27:07.498 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:07.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/3031938372 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9d9806be60 msgr2=0x7f9d9806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9d9806be60 0x7f9d9806e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9db0014450 tx=0x7f9db000b040 comp rx=0 tx=0).stop 2024-09-10T10:27:07.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/3031938372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9db80be6e0 msgr2=0x7f9db80c1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9db80be6e0 0x7f9db80c1170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9da0007370 tx=0x7f9da0007830 comp rx=0 tx=0).stop 2024-09-10T10:27:07.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/3031938372 shutdown_connections 2024-09-10T10:27:07.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9d9806be60 0x7f9d9806e320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9db80bde60 0x7f9db80bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9db80be6e0 0x7f9db80c1170 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 --2- 172.21.15.120:0/3031938372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da40051c0 0x7f9db81f07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/3031938372 >> 172.21.15.120:0/3031938372 conn(0x7f9db8067950 msgr2=0x7f9db8159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:07.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/3031938372 shutdown_connections 2024-09-10T10:27:07.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.496+0000 7f9dbcfd6640 1 -- 172.21.15.120:0/3031938372 wait complete. 2024-09-10T10:27:07.508 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3379691116},{"type":"v1","addr":"172.21.15.120:6833","nonce":3379691116}]},"active_addr":"172.21.15.120:6833/3379691116","active_change":"2024-09-10T10:26:22.079781+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":2608512187}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":121713597}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":3973768135}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":1865267468}]}]} 2024-09-10T10:27:07.511 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T10:27:07.511 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T10:27:07.511 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:27:07.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.608+0000 7f769e79d640 1 Processor -- start 2024-09-10T10:27:07.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.608+0000 7f769e79d640 1 -- start start 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.609+0000 7f769e79d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f769806bc10 0x7f76980c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.609+0000 7f769e79d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76980c8120 con 0x7f769815a810 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.609+0000 7f769e79d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76980c8dd0 con 0x7f76981549c0 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.609+0000 7f769e79d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f76980c9700 con 0x7f769806bc10 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.609+0000 7f769df9c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f769815a810 0x7f769815ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49086/0 (socket says 172.21.15.120:49086) 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.609+0000 7f769df9c640 1 -- 172.21.15.120:0/2568580448 learned_addr learned my addr 172.21.15.120:0/2568580448 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.609+0000 7f769cf9a640 1 --2- 172.21.15.120:0/2568580448 >> v2:172.21.15.134:3300/0 conn(0x7f769806bc10 0x7f76980c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 648615098 0 0) 0x7f76980c8120 con 0x7f769815a810 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7680003520 con 0x7f769815a810 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1230146233 0 0) 0x7f7680003520 con 0x7f769815a810 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 >> v2:172.21.15.134:3300/0 conn(0x7f769806bc10 msgr2=0x7f76980c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 --2- 172.21.15.120:0/2568580448 >> v2:172.21.15.134:3300/0 conn(0x7f769806bc10 0x7f76980c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 >> v1:172.21.15.134:6789/0 conn(0x7f76981549c0 legacy=0x7f7698072920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76980c90b0 con 0x7f769815a810 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f7690001670 con 0x7f769815a810 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 >> v1:172.21.15.120:6789/0 conn(0x7f769815a810 legacy=0x7f769815ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76800039b0 0x7f7680003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7680004800 0x7f7680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f768000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76980c9700 con 0x7f76800051c0 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f768000c070 con 0x7f7680004800 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f768000c270 con 0x7f76800039b0 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f769df9c640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f768000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f769cf9a640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7680004800 0x7f7680004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f769d79b640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76800039b0 0x7f7680003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f769df9c640 1 -- 172.21.15.120:0/2568580448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76800039b0 msgr2=0x7f7680003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f769df9c640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76800039b0 0x7f7680003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f769df9c640 1 -- 172.21.15.120:0/2568580448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7680004800 msgr2=0x7f7680004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f769df9c640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7680004800 0x7f7680004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.610+0000 7f769df9c640 1 -- 172.21.15.120:0/2568580448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7680003520 con 0x7f76800051c0 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.611+0000 7f769df9c640 1 -- 172.21.15.120:0/2568580448 reap_dead start 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.611+0000 7f769df9c640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f768000b9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f769000c210 tx=0x7f7680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1512a1d520331dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.611+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690007d30 con 0x7f76800051c0 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.611+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7690005ad0 con 0x7f76800051c0 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.611+0000 7f7697fff640 1 -- 172.21.15.120:0/2568580448 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690007070 con 0x7f76800051c0 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.612+0000 7f769e79d640 1 -- 172.21.15.120:0/2568580448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 msgr2=0x7f768000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.612+0000 7f769e79d640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f768000b9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f769000c210 tx=0x7f7680005830 comp rx=0 tx=0).stop 2024-09-10T10:27:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.612+0000 7f769e79d640 1 -- 172.21.15.120:0/2568580448 shutdown_connections 2024-09-10T10:27:07.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.612+0000 7f769e79d640 1 --2- 172.21.15.120:0/2568580448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f768000b9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.612+0000 7f769e79d640 1 -- 172.21.15.120:0/2568580448 >> 172.21.15.120:0/2568580448 conn(0x7f7698067950 msgr2=0x7f7698067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:07.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.612+0000 7f769e79d640 1 -- 172.21.15.120:0/2568580448 shutdown_connections 2024-09-10T10:27:07.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.612+0000 7f769e79d640 1 -- 172.21.15.120:0/2568580448 wait complete. 2024-09-10T10:27:07.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.613+0000 7f769e79d640 1 Processor -- start 2024-09-10T10:27:07.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.613+0000 7f769e79d640 1 -- start start 2024-09-10T10:27:07.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.613+0000 7f769e79d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f76981f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.613+0000 7f769e79d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7698156010 0x7f7698156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.613+0000 7f769e79d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f769815dae0 0x7f769815df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.613+0000 7f769e79d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f769806cea0 con 0x7f76800051c0 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.613+0000 7f769e79d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7698154da0 con 0x7f769815dae0 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769e79d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f76980ca8a0 con 0x7f7698156010 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769d79b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f76981f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769d79b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f76981f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34244/0 (socket says 172.21.15.120:34244) 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769d79b640 1 -- 172.21.15.120:0/3138055271 learned_addr learned my addr 172.21.15.120:0/3138055271 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769d79b640 1 -- 172.21.15.120:0/3138055271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7698156010 msgr2=0x7f7698156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769df9c640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f769815dae0 0x7f769815df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769d79b640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7698156010 0x7f7698156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769d79b640 1 -- 172.21.15.120:0/3138055271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f769815dae0 msgr2=0x7f769815df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769d79b640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f769815dae0 0x7f769815df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769d79b640 1 -- 172.21.15.120:0/3138055271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7698157d90 con 0x7f76800051c0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769d79b640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f76981f3820 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f768800f960 tx=0x7f768800fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f7695ffb640 1 -- 172.21.15.120:0/3138055271 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688003b80 con 0x7f76800051c0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f7695ffb640 1 -- 172.21.15.120:0/3138055271 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7688018070 con 0x7f76800051c0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769e79d640 1 -- 172.21.15.120:0/3138055271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76981f14b0 con 0x7f76800051c0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f7695ffb640 1 -- 172.21.15.120:0/3138055271 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768800d7f0 con 0x7f76800051c0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.614+0000 7f769e79d640 1 -- 172.21.15.120:0/3138055271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76981f43c0 con 0x7f76800051c0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.615+0000 7f769e79d640 1 -- 172.21.15.120:0/3138055271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76980c8080 con 0x7f76800051c0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.616+0000 7f7695ffb640 1 -- 172.21.15.120:0/3138055271 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f768800d990 con 0x7f76800051c0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.616+0000 7f7695ffb640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f768406be30 0x7f768406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.616+0000 7f769cf9a640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f768406be30 0x7f768406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.616+0000 7f7695ffb640 1 -- 172.21.15.120:0/3138055271 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f768808b390 con 0x7f76800051c0 2024-09-10T10:27:07.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.617+0000 7f769cf9a640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f768406be30 0x7f768406e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f76980cd600 tx=0x7f768c009320 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:07.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.621+0000 7f7695ffb640 1 -- 172.21.15.120:0/3138055271 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f768801a310 con 0x7f76800051c0 2024-09-10T10:27:07.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.788+0000 7f769e79d640 1 -- 172.21.15.120:0/3138055271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7698072f00 con 0x7f76800051c0 2024-09-10T10:27:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.790+0000 7f7695ffb640 1 -- 172.21.15.120:0/3138055271 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f7688059970 con 0x7f76800051c0 2024-09-10T10:27:07.791 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:07.791 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:47.823215+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:27:07.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 -- 172.21.15.120:0/3138055271 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f768406be30 msgr2=0x7f768406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f768406be30 0x7f768406e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f76980cd600 tx=0x7f768c009320 comp rx=0 tx=0).stop 2024-09-10T10:27:07.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 -- 172.21.15.120:0/3138055271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 msgr2=0x7f76981f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f76981f3820 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f768800f960 tx=0x7f768800fe20 comp rx=0 tx=0).stop 2024-09-10T10:27:07.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 -- 172.21.15.120:0/3138055271 shutdown_connections 2024-09-10T10:27:07.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f768406be30 0x7f768406e2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f769815dae0 0x7f769815df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7698156010 0x7f7698156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 --2- 172.21.15.120:0/3138055271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76800051c0 0x7f76981f3820 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 -- 172.21.15.120:0/3138055271 >> 172.21.15.120:0/3138055271 conn(0x7f7698067950 msgr2=0x7f769814aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:07.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 -- 172.21.15.120:0/3138055271 shutdown_connections 2024-09-10T10:27:07.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.792+0000 7f769e79d640 1 -- 172.21.15.120:0/3138055271 wait complete. 2024-09-10T10:27:07.804 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T10:27:07.804 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:27:07.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.902+0000 7f3615f2a640 1 Processor -- start 2024-09-10T10:27:07.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.902+0000 7f3615f2a640 1 -- start start 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f3615f2a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f36100c7770 0x7f36100cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f3615f2a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f361015b1a0 con 0x7f36100cb410 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f3615f2a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f361015b420 con 0x7f36100c6980 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f3615f2a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f361015b9e0 con 0x7f36100c7770 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f360effd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f36100c7770 0x7f36100cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f360ffff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f36100cb410 0x7f36100cbfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49092/0 (socket says 172.21.15.120:49092) 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f360ffff640 1 -- 172.21.15.120:0/3675644355 learned_addr learned my addr 172.21.15.120:0/3675644355 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f360effd640 1 -- 172.21.15.120:0/3675644355 >> v1:172.21.15.134:6789/0 conn(0x7f36100c6980 legacy=0x7f361015aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f360effd640 1 -- 172.21.15.120:0/3675644355 >> v1:172.21.15.120:6789/0 conn(0x7f36100cb410 legacy=0x7f36100cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f360effd640 1 -- 172.21.15.120:0/3675644355 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361015b700 con 0x7f36100c7770 2024-09-10T10:27:07.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.903+0000 7f360e7fc640 1 -- 172.21.15.120:0/3675644355 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2974076600 0 0) 0x7f3608001490 con 0x7f36100cb410 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.904+0000 7f360effd640 1 --2- 172.21.15.120:0/3675644355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 0x7f36100cd730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f36040097a0 tx=0x7f360402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e945b15c1a633a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.904+0000 7f360e7fc640 1 -- 172.21.15.120:0/3675644355 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604037070 con 0x7f36100c7770 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.904+0000 7f360e7fc640 1 -- 172.21.15.120:0/3675644355 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360402ae50 con 0x7f36100c7770 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.904+0000 7f360e7fc640 1 -- 172.21.15.120:0/3675644355 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604032660 con 0x7f36100c7770 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.904+0000 7f3615f2a640 1 -- 172.21.15.120:0/3675644355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 msgr2=0x7f36100cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.904+0000 7f3615f2a640 1 --2- 172.21.15.120:0/3675644355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 0x7f36100cd730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f36040097a0 tx=0x7f360402a140 comp rx=0 tx=0).stop 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.904+0000 7f3615f2a640 1 -- 172.21.15.120:0/3675644355 shutdown_connections 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.905+0000 7f3615f2a640 1 --2- 172.21.15.120:0/3675644355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 0x7f36100cd730 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.905+0000 7f3615f2a640 1 -- 172.21.15.120:0/3675644355 >> 172.21.15.120:0/3675644355 conn(0x7f361006cb70 msgr2=0x7f361006e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.905+0000 7f3615f2a640 1 -- 172.21.15.120:0/3675644355 shutdown_connections 2024-09-10T10:27:07.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.905+0000 7f3615f2a640 1 -- 172.21.15.120:0/3675644355 wait complete. 2024-09-10T10:27:07.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.906+0000 7f3615f2a640 1 Processor -- start 2024-09-10T10:27:07.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.906+0000 7f3615f2a640 1 -- start start 2024-09-10T10:27:07.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.906+0000 7f3615f2a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36100c6980 0x7f36101e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.906+0000 7f3615f2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 0x7f36101e5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.906+0000 7f3615f2a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36100cb410 0x7f36101e5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f3615f2a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36100cd210 con 0x7f36100c7770 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f3615f2a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36100ccf30 con 0x7f36100c6980 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f3615f2a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f36100d0fe0 con 0x7f36100cb410 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 0x7f36101e5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 0x7f36101e5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34260/0 (socket says 172.21.15.120:34260) 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360effd640 1 -- 172.21.15.120:0/143594136 learned_addr learned my addr 172.21.15.120:0/143594136 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360f7fe640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36100c6980 0x7f36101e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360ffff640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36100cb410 0x7f36101e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360effd640 1 -- 172.21.15.120:0/143594136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36100cb410 msgr2=0x7f36101e5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360effd640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36100cb410 0x7f36101e5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360effd640 1 -- 172.21.15.120:0/143594136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36100c6980 msgr2=0x7f36101e22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360effd640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36100c6980 0x7f36101e22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360effd640 1 -- 172.21.15.120:0/143594136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36101f3ce0 con 0x7f36100c7770 2024-09-10T10:27:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360f7fe640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36100c6980 0x7f36101e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f360effd640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 0x7f36101e5440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3604035040 tx=0x7f36040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f35f7fff640 1 -- 172.21.15.120:0/143594136 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604037040 con 0x7f36100c7770 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f35f7fff640 1 -- 172.21.15.120:0/143594136 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3604004030 con 0x7f36100c7770 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f3615f2a640 1 -- 172.21.15.120:0/143594136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36101f3f80 con 0x7f36100c7770 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f35f7fff640 1 -- 172.21.15.120:0/143594136 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604006e10 con 0x7f36100c7770 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.907+0000 7f3615f2a640 1 -- 172.21.15.120:0/143594136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36101f0b60 con 0x7f36100c7770 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.908+0000 7f3615f2a640 1 -- 172.21.15.120:0/143594136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36100cd730 con 0x7f36100c7770 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.909+0000 7f35f7fff640 1 -- 172.21.15.120:0/143594136 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f360403d650 con 0x7f36100c7770 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.909+0000 7f35f7fff640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f35fc06be30 0x7f35fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.909+0000 7f35f7fff640 1 -- 172.21.15.120:0/143594136 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f36040ab930 con 0x7f36100c7770 2024-09-10T10:27:07.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.909+0000 7f360f7fe640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f35fc06be30 0x7f35fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:07.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.910+0000 7f360f7fe640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f35fc06be30 0x7f35fc06e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3600003eb0 tx=0x7f3600009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:07.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:07.914+0000 7f35f7fff640 1 -- 172.21.15.120:0/143594136 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3604042020 con 0x7f36100c7770 2024-09-10T10:27:08.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.080+0000 7f3615f2a640 1 -- 172.21.15.120:0/143594136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36100c3080 con 0x7f36100c7770 2024-09-10T10:27:08.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.081+0000 7f35f7fff640 1 -- 172.21.15.120:0/143594136 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f360407a030 con 0x7f36100c7770 2024-09-10T10:27:08.083 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:08.083 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:26:47.823215+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:27:08.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 -- 172.21.15.120:0/143594136 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f35fc06be30 msgr2=0x7f35fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f35fc06be30 0x7f35fc06e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3600003eb0 tx=0x7f3600009040 comp rx=0 tx=0).stop 2024-09-10T10:27:08.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 -- 172.21.15.120:0/143594136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 msgr2=0x7f36101e5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 0x7f36101e5440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3604035040 tx=0x7f36040046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 -- 172.21.15.120:0/143594136 shutdown_connections 2024-09-10T10:27:08.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f35fc06be30 0x7f35fc06e2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36100cb410 0x7f36101e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36100c7770 0x7f36101e5440 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 --2- 172.21.15.120:0/143594136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36100c6980 0x7f36101e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 -- 172.21.15.120:0/143594136 >> 172.21.15.120:0/143594136 conn(0x7f361006cb70 msgr2=0x7f36100c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.083+0000 7f3615f2a640 1 -- 172.21.15.120:0/143594136 shutdown_connections 2024-09-10T10:27:08.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.084+0000 7f3615f2a640 1 -- 172.21.15.120:0/143594136 wait complete. 2024-09-10T10:27:08.096 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T10:27:08.096 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T10:27:08.097 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T10:27:08.097 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T10:27:08.098 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T10:27:08.098 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T10:27:08.098 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T10:27:08.099 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T10:27:08.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.264+0000 7ff142933640 1 Processor -- start 2024-09-10T10:27:08.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.264+0000 7ff142933640 1 -- start start 2024-09-10T10:27:08.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.264+0000 7fcd2359e640 1 Processor -- start 2024-09-10T10:27:08.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.265+0000 7fcd2359e640 1 -- start start 2024-09-10T10:27:08.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.264+0000 7f98adbf1640 1 Processor -- start 2024-09-10T10:27:08.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.264+0000 7f98adbf1640 1 -- start start 2024-09-10T10:27:08.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98adbf1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98a00dce90 0x7f98a0005a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98adbf1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f98a004aa40 0x7f98a0046390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98adbf1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a00e6030 0x7f98a00e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98adbf1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f98a0045660 con 0x7f98a004aa40 2024-09-10T10:27:08.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98adbf1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f98a0045f00 con 0x7f98a00dce90 2024-09-10T10:27:08.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98adbf1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98a00dd270 con 0x7f98a00e6030 2024-09-10T10:27:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.265+0000 7ff142933640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c158f40 0x7ff13c15ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.265+0000 7ff142933640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff13c159320 0x7ff13c0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.265+0000 7ff142933640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff13c162c00 0x7ff13c0c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.265+0000 7ff142933640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff13c162770 con 0x7ff13c159320 2024-09-10T10:27:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.265+0000 7ff142933640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff13c15f430 con 0x7ff13c162c00 2024-09-10T10:27:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.265+0000 7ff142933640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff13c15f710 con 0x7ff13c158f40 2024-09-10T10:27:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7ff140ed0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff13c162c00 0x7ff13c0c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7ff140ed0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff13c162c00 0x7ff13c0c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41624/0 (socket says 172.21.15.120:41624) 2024-09-10T10:27:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7ff140ed0640 1 -- 172.21.15.120:0/2971902395 learned_addr learned my addr 172.21.15.120:0/2971902395 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7ff13bfff640 1 --2- 172.21.15.120:0/2971902395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c158f40 0x7ff13c15ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.265+0000 7f3bbaac9640 1 Processor -- start 2024-09-10T10:27:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.265+0000 7f3bbaac9640 1 -- start start 2024-09-10T10:27:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7ff13b7fe640 1 --2- 172.21.15.120:0/2971902395 >> v2:172.21.15.120:3300/0 conn(0x7ff13c159320 0x7ff13c0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7ff140ed0640 1 -- 172.21.15.120:0/2971902395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c158f40 msgr2=0x7ff13c15ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7ff140ed0640 1 --2- 172.21.15.120:0/2971902395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c158f40 0x7ff13c15ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98a7fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f98a004aa40 0x7f98a0046390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98acbef640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98a00dce90 0x7f98a0005a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98acbef640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98a00dce90 0x7f98a0005a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41640/0 (socket says 172.21.15.120:41640) 2024-09-10T10:27:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98acbef640 1 -- 172.21.15.120:0/3673350389 learned_addr learned my addr 172.21.15.120:0/3673350389 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7f98ad3f0640 1 --2- 172.21.15.120:0/3673350389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a00e6030 0x7f98a00e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7fcd2359e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcd1c04b820 0x7fcd1c04bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7ff140ed0640 1 -- 172.21.15.120:0/2971902395 >> v2:172.21.15.120:3300/0 conn(0x7ff13c159320 msgr2=0x7ff13c0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7ff140ed0640 1 --2- 172.21.15.120:0/2971902395 >> v2:172.21.15.120:3300/0 conn(0x7ff13c159320 0x7ff13c0c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f13634e7640 1 Processor -- start 2024-09-10T10:27:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f3bbaac9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3bac04b6e0 0x7f3bac0e2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f3bbaac9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac04aac0 con 0x7f3bac005a30 2024-09-10T10:27:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f3bbaac9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac005e80 con 0x7f3bac0d8ad0 2024-09-10T10:27:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f3bbaac9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f3bac04b040 con 0x7f3bac04b6e0 2024-09-10T10:27:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f98acbef640 1 -- 172.21.15.120:0/3673350389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a00e6030 msgr2=0x7f98a00e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f98acbef640 1 --2- 172.21.15.120:0/3673350389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a00e6030 0x7f98a00e64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f98acbef640 1 -- 172.21.15.120:0/3673350389 >> v2:172.21.15.120:3300/0 conn(0x7f98a004aa40 msgr2=0x7f98a0046390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f98acbef640 1 --2- 172.21.15.120:0/3673350389 >> v2:172.21.15.120:3300/0 conn(0x7f98a004aa40 0x7f98a0046390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f98acbef640 1 -- 172.21.15.120:0/3673350389 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a00468d0 con 0x7f98a00dce90 2024-09-10T10:27:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f13634e7640 1 -- start start 2024-09-10T10:27:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f85091a6640 1 Processor -- start 2024-09-10T10:27:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7ff140ed0640 1 -- 172.21.15.120:0/2971902395 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c15f9f0 con 0x7ff13c162c00 2024-09-10T10:27:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7ff13bfff640 1 --2- 172.21.15.120:0/2971902395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c158f40 0x7ff13c15ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7fcd2359e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcd1c045a30 0x7fcd1c045ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7fcd2359e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1c0dd1d0 con 0x7fcd1c04c360 2024-09-10T10:27:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7fcd2359e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1c0e39f0 con 0x7fcd1c045a30 2024-09-10T10:27:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7fcd2359e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1c0463e0 con 0x7fcd1c04b820 2024-09-10T10:27:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7fcd22d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcd1c045a30 0x7fcd1c045ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7fcd21d9b640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fcd1c04c360 0x7fcd1c044f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:35764/0 (socket says 172.21.15.120:35764) 2024-09-10T10:27:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.266+0000 7fcd21d9b640 1 -- 172.21.15.120:0/461973504 learned_addr learned my addr 172.21.15.120:0/461973504 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7fcd2259c640 1 --2- 172.21.15.120:0/461973504 >> v2:172.21.15.134:3300/0 conn(0x7fcd1c04b820 0x7fcd1c04bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7fcd2159a640 1 -- 172.21.15.120:0/461973504 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2510106225 0 0) 0x7fcd1c0dd1d0 con 0x7fcd1c04c360 2024-09-10T10:27:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7fcd2159a640 1 -- 172.21.15.120:0/461973504 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd0c003520 con 0x7fcd1c04c360 2024-09-10T10:27:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f3bb3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3bac04b6e0 0x7f3bac0e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f3bb3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3bac04b6e0 0x7f3bac0e2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34286/0 (socket says 172.21.15.120:34286) 2024-09-10T10:27:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f3bb3fff640 1 -- 172.21.15.120:0/3466574342 learned_addr learned my addr 172.21.15.120:0/3466574342 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7fcd2259c640 1 -- 172.21.15.120:0/461973504 >> v2:172.21.15.120:3300/0 conn(0x7fcd1c045a30 msgr2=0x7fcd1c045ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7fcd2259c640 1 --2- 172.21.15.120:0/461973504 >> v2:172.21.15.120:3300/0 conn(0x7fcd1c045a30 0x7fcd1c045ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7fcd2259c640 1 -- 172.21.15.120:0/461973504 >> v1:172.21.15.134:6789/0 conn(0x7fcd1c04c360 legacy=0x7fcd1c044f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7fcd2259c640 1 -- 172.21.15.120:0/461973504 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c0e2b40 con 0x7fcd1c04b820 2024-09-10T10:27:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.267+0000 7f85091a6640 1 -- start start 2024-09-10T10:27:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f3bb37fe640 1 -- 172.21.15.120:0/3466574342 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399074330 0 0) 0x7f3bac005e80 con 0x7f3bac0d8ad0 2024-09-10T10:27:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f3bb37fe640 1 -- 172.21.15.120:0/3466574342 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ba4003520 con 0x7f3bac0d8ad0 2024-09-10T10:27:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f3bb3fff640 1 -- 172.21.15.120:0/3466574342 >> v1:172.21.15.134:6789/0 conn(0x7f3bac0d8ad0 legacy=0x7f3bac0df030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f3bb3fff640 1 -- 172.21.15.120:0/3466574342 >> v1:172.21.15.120:6789/0 conn(0x7f3bac005a30 legacy=0x7f3bac04bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f89076dd640 1 Processor -- start 2024-09-10T10:27:08.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7ff140ed0640 1 --2- 172.21.15.120:0/2971902395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff13c162c00 0x7ff13c0c7430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0c4000 tx=0x7ff13402a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a276c0c5bb1563f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f98acbef640 1 --2- 172.21.15.120:0/3673350389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00dce90 0x7f98a0005a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f98a0007530 tx=0x7f989c02a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3c355edeb736cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f89076dd640 1 -- start start 2024-09-10T10:27:08.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f3bb3fff640 1 -- 172.21.15.120:0/3466574342 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac04ad60 con 0x7f3bac04b6e0 2024-09-10T10:27:08.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f98a77fe640 1 -- 172.21.15.120:0/3673350389 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c037070 con 0x7f98a00dce90 2024-09-10T10:27:08.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f98a77fe640 1 -- 172.21.15.120:0/3673350389 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989c032440 con 0x7f98a00dce90 2024-09-10T10:27:08.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f98a77fe640 1 -- 172.21.15.120:0/3673350389 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c032810 con 0x7f98a00dce90 2024-09-10T10:27:08.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.268+0000 7f13634e7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f135c157ec0 0x7f135c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7fcd2259c640 1 --2- 172.21.15.120:0/461973504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd1c04b820 0x7fcd1c04bc00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00d790 tx=0x7fcd1402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec94e3b093c9520 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f13634e7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f135c0cd920 0x7f135c0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f85091a6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8504158f40 0x7f850415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f85091a6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850415f430 con 0x7f8504159320 2024-09-10T10:27:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f85091a6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850415f710 con 0x7f85040c8b10 2024-09-10T10:27:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f85091a6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f8504162770 con 0x7f8504158f40 2024-09-10T10:27:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7ff13affd640 1 -- 172.21.15.120:0/2971902395 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134036070 con 0x7ff13c162c00 2024-09-10T10:27:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7ff13affd640 1 -- 172.21.15.120:0/2971902395 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff134031600 con 0x7ff13c162c00 2024-09-10T10:27:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7ff13affd640 1 -- 172.21.15.120:0/2971902395 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340319d0 con 0x7ff13c162c00 2024-09-10T10:27:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f13634e7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f135c0c65a0 con 0x7f135c0c9cf0 2024-09-10T10:27:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f13634e7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f135c0cab20 con 0x7f135c157ec0 2024-09-10T10:27:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f1362ce6640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f135c0c9cf0 0x7f135c06c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:35776/0 (socket says 172.21.15.120:35776) 2024-09-10T10:27:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f8502d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8504158f40 0x7f850415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f8502d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8504158f40 0x7f850415ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41662/0 (socket says 172.21.15.120:41662) 2024-09-10T10:27:08.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f8502d9d640 1 -- 172.21.15.120:0/574655285 learned_addr learned my addr 172.21.15.120:0/574655285 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f1362ce6640 1 -- 172.21.15.120:0/3660622396 learned_addr learned my addr 172.21.15.120:0/3660622396 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:08.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f13624e5640 1 --2- 172.21.15.120:0/3660622396 >> v2:172.21.15.120:3300/0 conn(0x7f135c157ec0 0x7f135c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7fcd2159a640 1 -- 172.21.15.120:0/461973504 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14037070 con 0x7fcd1c04b820 2024-09-10T10:27:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f1361ce4640 1 --2- 172.21.15.120:0/3660622396 >> v2:172.21.15.134:3300/0 conn(0x7f135c0cd920 0x7f135c0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f89076dd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f89000c6980 0x7f89000cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7fcd2159a640 1 -- 172.21.15.120:0/461973504 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd14032440 con 0x7fcd1c04b820 2024-09-10T10:27:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7fcd2159a640 1 -- 172.21.15.120:0/461973504 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1402ae60 con 0x7fcd1c04b820 2024-09-10T10:27:08.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f13634e7640 1 -- 172.21.15.120:0/3660622396 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f135c0cae00 con 0x7f135c0cd920 2024-09-10T10:27:08.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f89076dd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89000cb410 0x7f89000cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f89076dd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f890015b1a0 con 0x7f89000c7770 2024-09-10T10:27:08.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f89076dd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f890015b9e0 con 0x7f89000c6980 2024-09-10T10:27:08.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f89076dd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f890015b420 con 0x7f89000cb410 2024-09-10T10:27:08.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8904c78640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f89000c7770 0x7f890015aa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49114/0 (socket says 172.21.15.120:49114) 2024-09-10T10:27:08.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.269+0000 7f3bb3fff640 1 --2- 172.21.15.120:0/3466574342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bac04b6e0 0x7f3bac0e2780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009870 tx=0x7f3ba802a3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e7e95fdf7904bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8904c78640 1 -- 172.21.15.120:0/2144679104 learned_addr learned my addr 172.21.15.120:0/2144679104 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:08.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8905c7a640 1 --2- 172.21.15.120:0/2144679104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89000cb410 0x7f89000cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7ff142933640 1 -- 172.21.15.120:0/2971902395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff13c162c00 msgr2=0x7ff13c0c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7ff142933640 1 --2- 172.21.15.120:0/2971902395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff13c162c00 0x7ff13c0c7430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0c4000 tx=0x7ff13402a730 comp rx=0 tx=0).stop 2024-09-10T10:27:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f98adbf1640 1 -- 172.21.15.120:0/3673350389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00dce90 msgr2=0x7f98a0005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f98adbf1640 1 --2- 172.21.15.120:0/3673350389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00dce90 0x7f98a0005a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f98a0007530 tx=0x7f989c02a1d0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f2a37e74640 1 Processor -- start 2024-09-10T10:27:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7ff142933640 1 -- 172.21.15.120:0/2971902395 shutdown_connections 2024-09-10T10:27:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7ff142933640 1 --2- 172.21.15.120:0/2971902395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff13c162c00 0x7ff13c0c7430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7ff142933640 1 --2- 172.21.15.120:0/2971902395 >> v2:172.21.15.120:3300/0 conn(0x7ff13c159320 0x7ff13c0c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7ff142933640 1 --2- 172.21.15.120:0/2971902395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c158f40 0x7ff13c15ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7ff142933640 1 -- 172.21.15.120:0/2971902395 >> 172.21.15.120:0/2971902395 conn(0x7ff13c06f3c0 msgr2=0x7ff13c06f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8905479640 1 --2- 172.21.15.120:0/2144679104 >> v2:172.21.15.120:3300/0 conn(0x7f89000c6980 0x7f89000cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f13624e5640 1 -- 172.21.15.120:0/3660622396 >> v2:172.21.15.134:3300/0 conn(0x7f135c0cd920 msgr2=0x7f135c0c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f13624e5640 1 --2- 172.21.15.120:0/3660622396 >> v2:172.21.15.134:3300/0 conn(0x7f135c0cd920 0x7f135c0c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f13614e3640 1 -- 172.21.15.120:0/3660622396 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2110351268 0 0) 0x7f1354001490 con 0x7f135c0c9cf0 2024-09-10T10:27:08.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f13624e5640 1 -- 172.21.15.120:0/3660622396 >> v1:172.21.15.134:6789/0 conn(0x7f135c0c9cf0 legacy=0x7f135c06c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f13624e5640 1 -- 172.21.15.120:0/3660622396 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c0c51c0 con 0x7f135c157ec0 2024-09-10T10:27:08.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f88f3fff640 1 -- 172.21.15.120:0/2144679104 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073273441 0 0) 0x7f88f4001110 con 0x7f89000c7770 2024-09-10T10:27:08.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f2a37e74640 1 -- start start 2024-09-10T10:27:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f3bb37fe640 1 -- 172.21.15.120:0/3466574342 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8037070 con 0x7f3bac04b6e0 2024-09-10T10:27:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f3bb37fe640 1 -- 172.21.15.120:0/3466574342 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba8032640 con 0x7f3bac04b6e0 2024-09-10T10:27:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f88f3fff640 1 -- 172.21.15.120:0/2144679104 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88ec003520 con 0x7f89000c7770 2024-09-10T10:27:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f3bb37fe640 1 -- 172.21.15.120:0/3466574342 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8032a10 con 0x7f3bac04b6e0 2024-09-10T10:27:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8905c7a640 1 -- 172.21.15.120:0/2144679104 >> v2:172.21.15.120:3300/0 conn(0x7f89000c6980 msgr2=0x7f89000cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8905c7a640 1 --2- 172.21.15.120:0/2144679104 >> v2:172.21.15.120:3300/0 conn(0x7f89000c6980 0x7f89000cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8905c7a640 1 -- 172.21.15.120:0/2144679104 >> v1:172.21.15.120:6789/0 conn(0x7f89000c7770 legacy=0x7f890015aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8905c7a640 1 -- 172.21.15.120:0/2144679104 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890015b700 con 0x7f89000cb410 2024-09-10T10:27:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7fcd2359e640 1 -- 172.21.15.120:0/461973504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd1c04b820 msgr2=0x7fcd1c04bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7fcd2359e640 1 --2- 172.21.15.120:0/461973504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd1c04b820 0x7fcd1c04bc00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00d790 tx=0x7fcd1402a110 comp rx=0 tx=0).stop 2024-09-10T10:27:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f98adbf1640 1 -- 172.21.15.120:0/3673350389 shutdown_connections 2024-09-10T10:27:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f98adbf1640 1 --2- 172.21.15.120:0/3673350389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a00e6030 0x7f98a00e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f98adbf1640 1 --2- 172.21.15.120:0/3673350389 >> v2:172.21.15.120:3300/0 conn(0x7f98a004aa40 0x7f98a0046390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f98adbf1640 1 --2- 172.21.15.120:0/3673350389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00dce90 0x7f98a0005a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f98adbf1640 1 -- 172.21.15.120:0/3673350389 >> 172.21.15.120:0/3673350389 conn(0x7f98a004fc50 msgr2=0x7f98a0050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f1361ce4640 1 --2- 172.21.15.120:0/3660622396 >> v2:172.21.15.134:3300/0 conn(0x7f135c0cd920 0x7f135c0c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8502d9d640 1 -- 172.21.15.120:0/574655285 >> v1:172.21.15.134:6789/0 conn(0x7f85040c8b10 legacy=0x7f85040c7430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:27:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8502d9d640 1 -- 172.21.15.120:0/574655285 >> v1:172.21.15.120:6789/0 conn(0x7f8504159320 legacy=0x7f8504162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.270+0000 7f8502d9d640 1 -- 172.21.15.120:0/574655285 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850415f9f0 con 0x7f8504158f40 2024-09-10T10:27:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7fcd2359e640 1 -- 172.21.15.120:0/461973504 shutdown_connections 2024-09-10T10:27:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7fcd2359e640 1 --2- 172.21.15.120:0/461973504 >> v2:172.21.15.120:3300/0 conn(0x7fcd1c045a30 0x7fcd1c045ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7fcd2359e640 1 --2- 172.21.15.120:0/461973504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd1c04b820 0x7fcd1c04bc00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7fcd2359e640 1 -- 172.21.15.120:0/461973504 >> 172.21.15.120:0/461973504 conn(0x7fcd1c050090 msgr2=0x7fcd1c0504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f8502d9d640 1 --2- 172.21.15.120:0/574655285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8504158f40 0x7f850415ee10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009e50 tx=0x7f84fc02a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6824192c9352bf6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7fcd2359e640 1 -- 172.21.15.120:0/461973504 shutdown_connections 2024-09-10T10:27:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f98adbf1640 1 -- 172.21.15.120:0/3673350389 shutdown_connections 2024-09-10T10:27:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7ff142933640 1 -- 172.21.15.120:0/2971902395 shutdown_connections 2024-09-10T10:27:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f3bbaac9640 1 -- 172.21.15.120:0/3466574342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bac04b6e0 msgr2=0x7f3bac0e2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f3bbaac9640 1 --2- 172.21.15.120:0/3466574342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bac04b6e0 0x7f3bac0e2780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009870 tx=0x7f3ba802a3e0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7fcd2359e640 1 -- 172.21.15.120:0/461973504 wait complete. 2024-09-10T10:27:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f98adbf1640 1 -- 172.21.15.120:0/3673350389 wait complete. 2024-09-10T10:27:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f3bbaac9640 1 -- 172.21.15.120:0/3466574342 shutdown_connections 2024-09-10T10:27:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f3bbaac9640 1 --2- 172.21.15.120:0/3466574342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bac04b6e0 0x7f3bac0e2780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f3bbaac9640 1 -- 172.21.15.120:0/3466574342 >> 172.21.15.120:0/3466574342 conn(0x7f3bac04fc80 msgr2=0x7f3bac050090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f13624e5640 1 --2- 172.21.15.120:0/3660622396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c157ec0 0x7f135c15b460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f135c0b5780 tx=0x7f135802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960da062d9056b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f2a37e74640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2a300c6520 0x7f2a300c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f8905c7a640 1 --2- 172.21.15.120:0/2144679104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89000cb410 0x7f89000cbfe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f88f80097a0 tx=0x7f88f802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63647b3b261c7e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f13614e3640 1 -- 172.21.15.120:0/3660622396 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358037070 con 0x7f135c157ec0 2024-09-10T10:27:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f13614e3640 1 -- 172.21.15.120:0/3660622396 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1358032440 con 0x7f135c157ec0 2024-09-10T10:27:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f13614e3640 1 -- 172.21.15.120:0/3660622396 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358032810 con 0x7f135c157ec0 2024-09-10T10:27:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f2a37e74640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a300c9d60 con 0x7f2a300c6900 2024-09-10T10:27:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f2a37e74640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a300ca040 con 0x7f2a3014e180 2024-09-10T10:27:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f2a37e74640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f2a300ca320 con 0x7f2a300c6520 2024-09-10T10:27:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f2a3540f640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f2a300c6900 0x7f2a3006ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49116/0 (socket says 172.21.15.120:49116) 2024-09-10T10:27:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f2a3540f640 1 -- 172.21.15.120:0/198930817 learned_addr learned my addr 172.21.15.120:0/198930817 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f2a35c10640 1 --2- 172.21.15.120:0/198930817 >> v2:172.21.15.120:3300/0 conn(0x7f2a300c6520 0x7f2a300c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7ff142933640 1 -- 172.21.15.120:0/2971902395 wait complete. 2024-09-10T10:27:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f8501d9b640 1 -- 172.21.15.120:0/574655285 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc036070 con 0x7f8504158f40 2024-09-10T10:27:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f8501d9b640 1 -- 172.21.15.120:0/574655285 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84fc031600 con 0x7f8504158f40 2024-09-10T10:27:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.271+0000 7f8501d9b640 1 -- 172.21.15.120:0/574655285 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0319d0 con 0x7f8504158f40 2024-09-10T10:27:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f88f3fff640 1 -- 172.21.15.120:0/2144679104 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8037070 con 0x7f89000cb410 2024-09-10T10:27:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f88f3fff640 1 -- 172.21.15.120:0/2144679104 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f802ae40 con 0x7f89000cb410 2024-09-10T10:27:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f85091a6640 1 -- 172.21.15.120:0/574655285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8504158f40 msgr2=0x7f850415ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f85091a6640 1 --2- 172.21.15.120:0/574655285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8504158f40 0x7f850415ee10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009e50 tx=0x7f84fc02a7a0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f3bbaac9640 1 -- 172.21.15.120:0/3466574342 shutdown_connections 2024-09-10T10:27:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f2a35c10640 1 -- 172.21.15.120:0/198930817 >> v1:172.21.15.134:6789/0 conn(0x7f2a3014e180 legacy=0x7f2a3015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f2a35c10640 1 -- 172.21.15.120:0/198930817 >> v1:172.21.15.120:6789/0 conn(0x7f2a300c6900 legacy=0x7f2a3006ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.272+0000 7f2a35c10640 1 -- 172.21.15.120:0/198930817 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a300ca600 con 0x7f2a300c6520 2024-09-10T10:27:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f88f3fff640 1 -- 172.21.15.120:0/2144679104 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8037070 con 0x7f89000cb410 2024-09-10T10:27:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f3bbaac9640 1 -- 172.21.15.120:0/3466574342 wait complete. 2024-09-10T10:27:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f13634e7640 1 -- 172.21.15.120:0/3660622396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c157ec0 msgr2=0x7f135c15b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f13634e7640 1 --2- 172.21.15.120:0/3660622396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c157ec0 0x7f135c15b460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f135c0b5780 tx=0x7f135802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f98adbf1640 1 Processor -- start 2024-09-10T10:27:08.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7ff142933640 1 Processor -- start 2024-09-10T10:27:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f13634e7640 1 -- 172.21.15.120:0/3660622396 shutdown_connections 2024-09-10T10:27:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f13634e7640 1 --2- 172.21.15.120:0/3660622396 >> v2:172.21.15.134:3300/0 conn(0x7f135c0cd920 0x7f135c0c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f13634e7640 1 --2- 172.21.15.120:0/3660622396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c157ec0 0x7f135c15b460 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f13634e7640 1 -- 172.21.15.120:0/3660622396 >> 172.21.15.120:0/3660622396 conn(0x7f135c067950 msgr2=0x7f135c067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f98adbf1640 1 -- start start 2024-09-10T10:27:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7fcd2359e640 1 Processor -- start 2024-09-10T10:27:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7ff142933640 1 -- start start 2024-09-10T10:27:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f2a35c10640 1 --2- 172.21.15.120:0/198930817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a300c6520 0x7f2a300c46c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0097a0 tx=0x7f2a2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d2712a20c09ce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f89076dd640 1 -- 172.21.15.120:0/2144679104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89000cb410 msgr2=0x7f89000cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f85091a6640 1 -- 172.21.15.120:0/574655285 shutdown_connections 2024-09-10T10:27:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f85091a6640 1 --2- 172.21.15.120:0/574655285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8504158f40 0x7f850415ee10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7f85091a6640 1 -- 172.21.15.120:0/574655285 >> 172.21.15.120:0/574655285 conn(0x7f850406f3c0 msgr2=0x7f850406f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f89076dd640 1 --2- 172.21.15.120:0/2144679104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89000cb410 0x7f89000cbfe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f88f80097a0 tx=0x7f88f802a0f0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f85091a6640 1 -- 172.21.15.120:0/574655285 shutdown_connections 2024-09-10T10:27:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f2a34c0e640 1 -- 172.21.15.120:0/198930817 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c037070 con 0x7f2a300c6520 2024-09-10T10:27:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f85091a6640 1 -- 172.21.15.120:0/574655285 wait complete. 2024-09-10T10:27:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f2a34c0e640 1 -- 172.21.15.120:0/198930817 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a2c032440 con 0x7f2a300c6520 2024-09-10T10:27:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f2a34c0e640 1 -- 172.21.15.120:0/198930817 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c032810 con 0x7f2a300c6520 2024-09-10T10:27:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f13634e7640 1 -- 172.21.15.120:0/3660622396 shutdown_connections 2024-09-10T10:27:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f89076dd640 1 -- 172.21.15.120:0/2144679104 shutdown_connections 2024-09-10T10:27:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f89076dd640 1 --2- 172.21.15.120:0/2144679104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89000cb410 0x7f89000cbfe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f89076dd640 1 --2- 172.21.15.120:0/2144679104 >> v2:172.21.15.120:3300/0 conn(0x7f89000c6980 0x7f89000cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f89076dd640 1 -- 172.21.15.120:0/2144679104 >> 172.21.15.120:0/2144679104 conn(0x7f890006cb70 msgr2=0x7f890006e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f13634e7640 1 -- 172.21.15.120:0/3660622396 wait complete. 2024-09-10T10:27:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f3bbaac9640 1 Processor -- start 2024-09-10T10:27:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.273+0000 7fcd2359e640 1 -- start start 2024-09-10T10:27:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7fcd2359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd1c045a30 0x7fcd1c1851a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7fcd2359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd1c04b820 0x7fcd1c049150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7fcd2359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c04c360 0x7fcd1c0db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7fcd2359e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c0e33a0 con 0x7fcd1c04c360 2024-09-10T10:27:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7fcd2359e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c0e30c0 con 0x7fcd1c045a30 2024-09-10T10:27:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7fcd2359e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c0e3680 con 0x7fcd1c04b820 2024-09-10T10:27:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98adbf1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a004aa40 0x7f98a00db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f3bbaac9640 1 -- start start 2024-09-10T10:27:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7ff142933640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff13c158f40 0x7ff13c161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7ff142933640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff13c159320 0x7ff13c15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.274+0000 7f89076dd640 1 -- 172.21.15.120:0/2144679104 shutdown_connections 2024-09-10T10:27:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98adbf1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98a00dce90 0x7f98a0184700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f89076dd640 1 -- 172.21.15.120:0/2144679104 wait complete. 2024-09-10T10:27:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7fcd21d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd1c04b820 0x7fcd1c049150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7fcd21d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd1c04b820 0x7fcd1c049150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:42736/0 (socket says 172.21.15.120:42736) 2024-09-10T10:27:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7fcd21d9b640 1 -- 172.21.15.120:0/1673824455 learned_addr learned my addr 172.21.15.120:0/1673824455 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7fcd22d9d640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c04c360 0x7fcd1c0db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff13c159320 0x7ff13c15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff13c159320 0x7ff13c15ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34340/0 (socket says 172.21.15.120:34340) 2024-09-10T10:27:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7ff142933640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c162c00 0x7ff13c159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98a00dce90 0x7f98a0184700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98acbef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a004aa40 0x7f98a00db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98acbef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a004aa40 0x7f98a00db0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41668/0 (socket says 172.21.15.120:41668) 2024-09-10T10:27:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7ff13b7fe640 1 -- 172.21.15.120:0/2517997545 learned_addr learned my addr 172.21.15.120:0/2517997545 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98a00dce90 0x7f98a0184700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34330/0 (socket says 172.21.15.120:34330) 2024-09-10T10:27:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98acbef640 1 -- 172.21.15.120:0/414838768 learned_addr learned my addr 172.21.15.120:0/414838768 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c072070 con 0x7ff13c159320 2024-09-10T10:27:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c162400 con 0x7ff13c158f40 2024-09-10T10:27:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98adbf1640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a00e6030 0x7f98a0047170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff13c071930 con 0x7ff13c162c00 2024-09-10T10:27:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7fcd21d9b640 1 -- 172.21.15.120:0/1673824455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd1c045a30 msgr2=0x7fcd1c1851a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:27:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00458b0 con 0x7f98a00dce90 2024-09-10T10:27:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7fcd21d9b640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd1c045a30 0x7fcd1c1851a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00e6dc0 con 0x7f98a004aa40 2024-09-10T10:27:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7fcd21d9b640 1 -- 172.21.15.120:0/1673824455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c04c360 msgr2=0x7fcd1c0db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f98ad3f0640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a00e6030 0x7f98a0047170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98a0045b90 con 0x7f98a00e6030 2024-09-10T10:27:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7fcd21d9b640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c04c360 0x7fcd1c0db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7fcd21d9b640 1 -- 172.21.15.120:0/1673824455 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c1813e0 con 0x7fcd1c04b820 2024-09-10T10:27:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f3bbaac9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bac005a30 0x7f3bac049860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f85091a6640 1 Processor -- start 2024-09-10T10:27:08.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7fcd22d9d640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c04c360 0x7fcd1c0db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:08.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f3bbaac9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bac04b6e0 0x7f3bac179540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f85091a6640 1 -- start start 2024-09-10T10:27:08.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7ff13b7fe640 1 -- 172.21.15.120:0/2517997545 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c162c00 msgr2=0x7ff13c159c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:27:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7ff13b7fe640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff13c162c00 0x7ff13c159c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7ff13b7fe640 1 -- 172.21.15.120:0/2517997545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff13c158f40 msgr2=0x7ff13c161450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:27:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7ff13b7fe640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff13c158f40 0x7ff13c161450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7ff13b7fe640 1 -- 172.21.15.120:0/2517997545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c1f4e60 con 0x7ff13c159320 2024-09-10T10:27:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f3bbaac9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bac0d8ad0 0x7f3bac047d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f3bbaac9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0e1f80 con 0x7f3bac005a30 2024-09-10T10:27:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7ff13b7fe640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff13c159320 0x7ff13c15ffd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00e8d0 tx=0x7ff12c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7fcd21d9b640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd1c04b820 0x7fcd1c049150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcd2406a930 tx=0x7fcd24069890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7fcd12ffd640 1 -- 172.21.15.120:0/1673824455 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24091070 con 0x7fcd1c04b820 2024-09-10T10:27:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f2a37e74640 1 -- 172.21.15.120:0/198930817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a300c6520 msgr2=0x7f2a300c46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.275+0000 7f2a37e74640 1 --2- 172.21.15.120:0/198930817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a300c6520 0x7f2a300c46c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0097a0 tx=0x7f2a2c02a270 comp rx=0 tx=0).stop 2024-09-10T10:27:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f2a37e74640 1 -- 172.21.15.120:0/198930817 shutdown_connections 2024-09-10T10:27:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f2a37e74640 1 --2- 172.21.15.120:0/198930817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a300c6520 0x7f2a300c46c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f2a37e74640 1 -- 172.21.15.120:0/198930817 >> 172.21.15.120:0/198930817 conn(0x7f2a30068140 msgr2=0x7f2a30070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f98a7fff640 1 -- 172.21.15.120:0/414838768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a00e6030 msgr2=0x7f98a0047170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f98a7fff640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a00e6030 0x7f98a0047170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f98a7fff640 1 -- 172.21.15.120:0/414838768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a004aa40 msgr2=0x7f98a00db0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f98a7fff640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a004aa40 0x7f98a00db0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7ff138ff9640 1 -- 172.21.15.120:0/2517997545 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c017020 con 0x7ff13c159320 2024-09-10T10:27:08.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f98a7fff640 1 -- 172.21.15.120:0/414838768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a00db600 con 0x7f98a00dce90 2024-09-10T10:27:08.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7fcd2359e640 1 -- 172.21.15.120:0/1673824455 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1c0d8af0 con 0x7fcd1c04b820 2024-09-10T10:27:08.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f3bb3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bac04b6e0 0x7f3bac179540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f3bb8865640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bac005a30 0x7f3bac049860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f3bb3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bac04b6e0 0x7f3bac179540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41704/0 (socket says 172.21.15.120:41704) 2024-09-10T10:27:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f3bb3fff640 1 -- 172.21.15.120:0/2862873466 learned_addr learned my addr 172.21.15.120:0/2862873466 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7f3bb9066640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bac0d8ad0 0x7f3bac047d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0e22f0 con 0x7f3bac04b6e0 2024-09-10T10:27:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac045130 con 0x7f3bac0d8ad0 2024-09-10T10:27:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f13634e7640 1 Processor -- start 2024-09-10T10:27:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f2a37e74640 1 -- 172.21.15.120:0/198930817 shutdown_connections 2024-09-10T10:27:08.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7fcd2359e640 1 -- 172.21.15.120:0/1673824455 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1c181930 con 0x7fcd1c04b820 2024-09-10T10:27:08.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f2a37e74640 1 -- 172.21.15.120:0/198930817 wait complete. 2024-09-10T10:27:08.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f89076dd640 1 Processor -- start 2024-09-10T10:27:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f85091a6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85040c8b10 0x7f8504151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f85091a6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8504158f40 0x7f85040c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f85091a6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8504159320 0x7f85040c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f85091a6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8504163310 con 0x7f8504158f40 2024-09-10T10:27:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f85091a6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8504162400 con 0x7f8504159320 2024-09-10T10:27:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f85091a6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8504072070 con 0x7f85040c8b10 2024-09-10T10:27:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7fcd12ffd640 1 -- 172.21.15.120:0/1673824455 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd24071840 con 0x7fcd1c04b820 2024-09-10T10:27:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7fcd12ffd640 1 -- 172.21.15.120:0/1673824455 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24071c30 con 0x7fcd1c04b820 2024-09-10T10:27:08.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f89076dd640 1 -- start start 2024-09-10T10:27:08.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f98a7fff640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98a00dce90 0x7f98a0184700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f989400ec10 tx=0x7f989400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.276+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c160510 con 0x7ff13c159320 2024-09-10T10:27:08.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c162000 con 0x7ff13c159320 2024-09-10T10:27:08.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7ff138ff9640 1 -- 172.21.15.120:0/2517997545 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c004240 con 0x7ff13c159320 2024-09-10T10:27:08.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f850259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8504158f40 0x7f85040c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f850259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8504158f40 0x7f85040c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34358/0 (socket says 172.21.15.120:34358) 2024-09-10T10:27:08.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f850259c640 1 -- 172.21.15.120:0/2041471956 learned_addr learned my addr 172.21.15.120:0/2041471956 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7ff138ff9640 1 -- 172.21.15.120:0/2517997545 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00c730 con 0x7ff13c159320 2024-09-10T10:27:08.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff11c001030 con 0x7ff13c159320 2024-09-10T10:27:08.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f8502d9d640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85040c8b10 0x7f8504151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.277+0000 7f13634e7640 1 -- start start 2024-09-10T10:27:08.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f3bb8865640 1 -- 172.21.15.120:0/2862873466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bac0d8ad0 msgr2=0x7f3bac047d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f3bb8865640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bac0d8ad0 0x7f3bac047d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f3bb8865640 1 -- 172.21.15.120:0/2862873466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bac04b6e0 msgr2=0x7f3bac179540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f3bb8865640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bac04b6e0 0x7f3bac179540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f3bb8865640 1 -- 172.21.15.120:0/2862873466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac0d8f50 con 0x7f3bac005a30 2024-09-10T10:27:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f3bb8865640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bac005a30 0x7f3bac049860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c3f0 tx=0x7f3ba000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f8502d9d640 1 -- 172.21.15.120:0/2041471956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8504159320 msgr2=0x7f85040c5690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:27:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f8502d9d640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8504159320 0x7f85040c5690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f8502d9d640 1 -- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8504158f40 msgr2=0x7f85040c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f8502d9d640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8504158f40 0x7f85040c5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f8502d9d640 1 -- 172.21.15.120:0/2041471956 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850415e0e0 con 0x7f85040c8b10 2024-09-10T10:27:08.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f98a57fa640 1 -- 172.21.15.120:0/414838768 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894017070 con 0x7f98a00dce90 2024-09-10T10:27:08.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f98a57fa640 1 -- 172.21.15.120:0/414838768 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989400ed00 con 0x7f98a00dce90 2024-09-10T10:27:08.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f98a57fa640 1 -- 172.21.15.120:0/414838768 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989400ce30 con 0x7f98a00dce90 2024-09-10T10:27:08.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f89076dd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89000c6980 0x7f8900159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f13634e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c0c9cf0 0x7f135c148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.278+0000 7f89076dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89000c7770 0x7f8900159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f8905479640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89000c6980 0x7f8900159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f8905479640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89000c6980 0x7f8900159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41734/0 (socket says 172.21.15.120:41734) 2024-09-10T10:27:08.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f13634e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f135c0cd920 0x7f135c0cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f8905479640 1 -- 172.21.15.120:0/1727437275 learned_addr learned my addr 172.21.15.120:0/1727437275 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f13634e7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f135c157ec0 0x7f135c1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f89076dd640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89000cb410 0x7f890014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f13634e7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f135c158420 con 0x7f135c0c9cf0 2024-09-10T10:27:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f13634e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1582a0 con 0x7f135c0cd920 2024-09-10T10:27:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f13634e7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f135c06c800 con 0x7f135c157ec0 2024-09-10T10:27:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89000cd210 con 0x7f89000c7770 2024-09-10T10:27:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89000ccf30 con 0x7f89000c6980 2024-09-10T10:27:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f2a37e74640 1 Processor -- start 2024-09-10T10:27:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f8905c7a640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89000cb410 0x7f890014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a016be40 con 0x7f98a00dce90 2024-09-10T10:27:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f89000d0fe0 con 0x7f89000cb410 2024-09-10T10:27:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a004f750 con 0x7f98a00dce90 2024-09-10T10:27:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f1361ce4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f135c0cd920 0x7f135c0cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f1362ce6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f135c157ec0 0x7f135c1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f1362ce6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f135c157ec0 0x7f135c1503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:42780/0 (socket says 172.21.15.120:42780) 2024-09-10T10:27:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f1362ce6640 1 -- 172.21.15.120:0/952849144 learned_addr learned my addr 172.21.15.120:0/952849144 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7fcd2359e640 1 -- 172.21.15.120:0/1673824455 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd08001030 con 0x7fcd1c04b820 2024-09-10T10:27:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f2a37e74640 1 -- start start 2024-09-10T10:27:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f8502d9d640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85040c8b10 0x7f8504151d80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f84fc030000 tx=0x7f84fc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f13624e5640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c0c9cf0 0x7f135c148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f84fb7fe640 1 -- 172.21.15.120:0/2041471956 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc036070 con 0x7f85040c8b10 2024-09-10T10:27:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f3bb17fa640 1 -- 172.21.15.120:0/2862873466 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0017070 con 0x7f3bac005a30 2024-09-10T10:27:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f84fb7fe640 1 -- 172.21.15.120:0/2041471956 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84fc037040 con 0x7f85040c8b10 2024-09-10T10:27:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f84fb7fe640 1 -- 172.21.15.120:0/2041471956 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc007c10 con 0x7f85040c8b10 2024-09-10T10:27:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f3bb17fa640 1 -- 172.21.15.120:0/2862873466 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba00035f0 con 0x7f3bac005a30 2024-09-10T10:27:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85041f5710 con 0x7f85040c8b10 2024-09-10T10:27:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f850415ec70 con 0x7f85040c8b10 2024-09-10T10:27:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98a0005a00 con 0x7f98a00dce90 2024-09-10T10:27:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f3bb17fa640 1 -- 172.21.15.120:0/2862873466 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00039e0 con 0x7f3bac005a30 2024-09-10T10:27:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f2a37e74640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a300c6520 0x7f2a300cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7ff138ff9640 1 -- 172.21.15.120:0/2517997545 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff12c01a020 con 0x7ff13c159320 2024-09-10T10:27:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac0471d0 con 0x7f3bac005a30 2024-09-10T10:27:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac0d91f0 con 0x7f3bac005a30 2024-09-10T10:27:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f2a37e74640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a300c6900 0x7f2a30140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f2a37e74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a3014e180 0x7f2a301f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84e0001030 con 0x7f85040c8b10 2024-09-10T10:27:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f2a3540f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a300c6900 0x7f2a30140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f2a3540f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a300c6900 0x7f2a30140d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:42788/0 (socket says 172.21.15.120:42788) 2024-09-10T10:27:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f1362ce6640 1 -- 172.21.15.120:0/952849144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f135c0cd920 msgr2=0x7f135c0cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f2a3540f640 1 -- 172.21.15.120:0/3211200339 learned_addr learned my addr 172.21.15.120:0/3211200339 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.279+0000 7f8904c78640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89000c7770 0x7f8900159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f8905479640 1 -- 172.21.15.120:0/1727437275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89000cb410 msgr2=0x7f890014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f8905479640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89000cb410 0x7f890014ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f8905479640 1 -- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89000c7770 msgr2=0x7f8900159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f8905479640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89000c7770 0x7f8900159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7f8905479640 1 -- 172.21.15.120:0/1727437275 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890014f7a0 con 0x7f89000c6980 2024-09-10T10:27:08.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f8905479640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89000c6980 0x7f8900159510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00efc0 tx=0x7f88fc00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f88f1ffb640 1 -- 172.21.15.120:0/1727437275 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc018020 con 0x7f89000c6980 2024-09-10T10:27:08.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7ff138ff9640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff12806bf00 0x7ff12806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f88f1ffb640 1 -- 172.21.15.120:0/1727437275 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88fc014480 con 0x7f89000c6980 2024-09-10T10:27:08.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.280+0000 7fcd12ffd640 1 -- 172.21.15.120:0/1673824455 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd24094070 con 0x7fcd1c04b820 2024-09-10T10:27:08.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7fcd12ffd640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcd0406be30 0x7fcd0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7fcd2259c640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcd0406be30 0x7fcd0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7fcd12ffd640 1 -- 172.21.15.120:0/1673824455 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcd24107320 con 0x7fcd1c04b820 2024-09-10T10:27:08.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7fcd12ffd640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fcd040755a0 0x7fcd040779e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7fcd12ffd640 1 -- 172.21.15.120:0/1673824455 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd040780b0 con 0x7fcd040755a0 2024-09-10T10:27:08.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7fcd12ffd640 1 -- 172.21.15.120:0/1673824455 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcd2410e050 con 0x7fcd1c04b820 2024-09-10T10:27:08.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3bac046f80 con 0x7f3bac005a30 2024-09-10T10:27:08.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f1362ce6640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f135c0cd920 0x7f135c0cf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f1362ce6640 1 -- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c0c9cf0 msgr2=0x7f135c148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f1362ce6640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c0c9cf0 0x7f135c148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f1362ce6640 1 -- 172.21.15.120:0/952849144 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c1f4840 con 0x7f135c157ec0 2024-09-10T10:27:08.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7ff138ff9640 1 -- 172.21.15.120:0/2517997545 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff12c08b9f0 con 0x7ff13c159320 2024-09-10T10:27:08.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f84fb7fe640 1 -- 172.21.15.120:0/2041471956 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84fc003990 con 0x7f85040c8b10 2024-09-10T10:27:08.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7fcd22d9d640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fcd040755a0 0x7fcd040779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7ff13bfff640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff12806bf00 0x7ff12806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.281+0000 7f2a36411640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a3014e180 0x7f2a301f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f2a35c10640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a300c6520 0x7f2a300cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f1362ce6640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f135c157ec0 0x7f135c1503d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f135400c820 tx=0x7f135400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f84fb7fe640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f84ec06be30 0x7f84ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f84fb7fe640 1 -- 172.21.15.120:0/2041471956 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f84fc0ac2c0 con 0x7f85040c8b10 2024-09-10T10:27:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a300c7590 con 0x7f2a300c6520 2024-09-10T10:27:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f88f1ffb640 1 -- 172.21.15.120:0/1727437275 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc014850 con 0x7f89000c6980 2024-09-10T10:27:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89001e4ba0 con 0x7f89000c6980 2024-09-10T10:27:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7fcd2259c640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcd0406be30 0x7fcd0406e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd14034000 tx=0x7fcd1402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7fcd22d9d640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fcd040755a0 0x7fcd040779e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f1352ffd640 1 -- 172.21.15.120:0/952849144 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354018070 con 0x7f135c157ec0 2024-09-10T10:27:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f1352ffd640 1 -- 172.21.15.120:0/952849144 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135400a5e0 con 0x7f135c157ec0 2024-09-10T10:27:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f1352ffd640 1 -- 172.21.15.120:0/952849144 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135400a9d0 con 0x7f135c157ec0 2024-09-10T10:27:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a300c4cc0 con 0x7f2a3014e180 2024-09-10T10:27:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f13634e7640 1 -- 172.21.15.120:0/952849144 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c14f8f0 con 0x7f135c157ec0 2024-09-10T10:27:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2a300c7710 con 0x7f2a300c6900 2024-09-10T10:27:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f84fb7fe640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f84ec075520 0x7f84ec077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f84fb7fe640 1 -- 172.21.15.120:0/2041471956 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84fc008e50 con 0x7f84ec075520 2024-09-10T10:27:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7f84fb7fe640 1 -- 172.21.15.120:0/2041471956 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84fc0b3050 con 0x7f85040c8b10 2024-09-10T10:27:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89000cfb80 con 0x7f89000c6980 2024-09-10T10:27:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7f13634e7640 1 -- 172.21.15.120:0/952849144 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c14feb0 con 0x7f135c157ec0 2024-09-10T10:27:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7f850259c640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f84ec06be30 0x7f84ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7f850359e640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f84ec075520 0x7f84ec077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7f2a35c10640 1 -- 172.21.15.120:0/3211200339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a300c6900 msgr2=0x7f2a30140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7f2a35c10640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a300c6900 0x7f2a30140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7f2a35c10640 1 -- 172.21.15.120:0/3211200339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a3014e180 msgr2=0x7f2a301f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7f2a35c10640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a3014e180 0x7f2a301f14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7f2a35c10640 1 -- 172.21.15.120:0/3211200339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a301e27c0 con 0x7f2a300c6520 2024-09-10T10:27:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7f2a35c10640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a300c6520 0x7f2a300cd5d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c031000 tx=0x7f2a2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f98a57fa640 1 -- 172.21.15.120:0/414838768 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f989401a070 con 0x7f98a00dce90 2024-09-10T10:27:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.284+0000 7f13634e7640 1 -- 172.21.15.120:0/952849144 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f135c154c30 con 0x7f135c157ec0 2024-09-10T10:27:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f1352ffd640 1 -- 172.21.15.120:0/952849144 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f135400ab70 con 0x7f135c157ec0 2024-09-10T10:27:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f1352ffd640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f134006be30 0x7f134006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f1352ffd640 1 -- 172.21.15.120:0/952849144 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f135408a8d0 con 0x7f135c157ec0 2024-09-10T10:27:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.284+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f890015a690 con 0x7f89000c6980 2024-09-10T10:27:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f88f1ffb640 1 -- 172.21.15.120:0/1727437275 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88fc005d10 con 0x7f89000c6980 2024-09-10T10:27:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.284+0000 7f850359e640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f84ec075520 0x7f84ec077960 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f850259c640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f84ec06be30 0x7f84ec06e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8504162000 tx=0x7f84f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f84fb7fe640 1 -- 172.21.15.120:0/2041471956 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84fc008e50 con 0x7f84ec075520 2024-09-10T10:27:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.283+0000 7fcd12ffd640 1 -- 172.21.15.120:0/1673824455 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd040780b0 con 0x7fcd040755a0 2024-09-10T10:27:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.284+0000 7f2a267fc640 1 -- 172.21.15.120:0/3211200339 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c041070 con 0x7f2a300c6520 2024-09-10T10:27:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.284+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a301e2aa0 con 0x7f2a300c6520 2024-09-10T10:27:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a301e3060 con 0x7f2a300c6520 2024-09-10T10:27:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f2a267fc640 1 -- 172.21.15.120:0/3211200339 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a2c03c870 con 0x7f2a300c6520 2024-09-10T10:27:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f1352ffd640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f13400755a0 0x7f13400779e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f1352ffd640 1 -- 172.21.15.120:0/952849144 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13400780b0 con 0x7f13400755a0 2024-09-10T10:27:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f1352ffd640 1 -- 172.21.15.120:0/952849144 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f135408f2a0 con 0x7f135c157ec0 2024-09-10T10:27:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f2a267fc640 1 -- 172.21.15.120:0/3211200339 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c037040 con 0x7f2a300c6520 2024-09-10T10:27:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f13624e5640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f134006be30 0x7f134006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f3bb17fa640 1 -- 172.21.15.120:0/2862873466 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ba001a070 con 0x7f3bac005a30 2024-09-10T10:27:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f88f1ffb640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f88e406bf00 0x7f88e406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.285+0000 7f88f1ffb640 1 -- 172.21.15.120:0/1727437275 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f88fc08bf40 con 0x7f89000c6980 2024-09-10T10:27:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f88f1ffb640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f88e40755f0 0x7f88e4077a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f88f1ffb640 1 -- 172.21.15.120:0/1727437275 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88fc012e50 con 0x7f88e40755f0 2024-09-10T10:27:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f88f1ffb640 1 -- 172.21.15.120:0/1727437275 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88fc0561c0 con 0x7f89000c6980 2024-09-10T10:27:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a18001030 con 0x7f2a300c6520 2024-09-10T10:27:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f98a57fa640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f988c06bf00 0x7f988c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.282+0000 7ff138ff9640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff128075670 0x7ff128077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.284+0000 7ff138ff9640 1 -- 172.21.15.120:0/2517997545 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff128078180 con 0x7ff128075670 2024-09-10T10:27:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.284+0000 7ff138ff9640 1 -- 172.21.15.120:0/2517997545 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff12c08bde0 con 0x7ff13c159320 2024-09-10T10:27:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.284+0000 7ff140ed0640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff128075670 0x7ff128077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f1361ce4640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f13400755a0 0x7f13400779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f98a57fa640 1 -- 172.21.15.120:0/414838768 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f989408b350 con 0x7f98a00dce90 2024-09-10T10:27:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f8904c78640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f88e406bf00 0x7f88e406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f98a57fa640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f988c075670 0x7f988c077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f3bb17fa640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b9806be80 0x7f3b9806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f98a57fa640 1 -- 172.21.15.120:0/414838768 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9894007e50 con 0x7f988c075670 2024-09-10T10:27:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.286+0000 7f98a57fa640 1 -- 172.21.15.120:0/414838768 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f989408b740 con 0x7f98a00dce90 2024-09-10T10:27:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f3bb17fa640 1 -- 172.21.15.120:0/2862873466 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3ba008abd0 con 0x7f3bac005a30 2024-09-10T10:27:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f1361ce4640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f13400755a0 0x7f13400779e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7ff13bfff640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff12806bf00 0x7ff12806e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff130009560 tx=0x7ff130009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f3bb17fa640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f3b980755f0 0x7f3b98077a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f3bb17fa640 1 -- 172.21.15.120:0/2862873466 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b98078100 con 0x7f3b980755f0 2024-09-10T10:27:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f3bb17fa640 1 -- 172.21.15.120:0/2862873466 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ba008af80 con 0x7f3bac005a30 2024-09-10T10:27:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f3bb3fff640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b9806be80 0x7f3b9806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f8905c7a640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f88e40755f0 0x7f88e4077a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f98acbef640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f988c06bf00 0x7f988c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f98ad3f0640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f988c075670 0x7f988c077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f1352ffd640 1 -- 172.21.15.120:0/952849144 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13400780b0 con 0x7f13400755a0 2024-09-10T10:27:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7ff140ed0640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff128075670 0x7ff128077ab0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.287+0000 7f3bb9066640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f3b980755f0 0x7f3b98077a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.288+0000 7f13624e5640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f134006be30 0x7f134006e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1358034000 tx=0x7f135802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.288+0000 7f8905c7a640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f88e40755f0 0x7f88e4077a30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.288+0000 7f8904c78640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f88e406bf00 0x7f88e406e3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88f4007fd0 tx=0x7f88f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.288+0000 7f98acbef640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f988c06bf00 0x7f988c06e3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f989c009630 tx=0x7f989c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.288+0000 7ff138ff9640 1 -- 172.21.15.120:0/2517997545 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff128078180 con 0x7ff128075670 2024-09-10T10:27:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.289+0000 7f88f1ffb640 1 -- 172.21.15.120:0/1727437275 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f88fc012e50 con 0x7f88e40755f0 2024-09-10T10:27:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.289+0000 7f98ad3f0640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f988c075670 0x7f988c077ab0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.289+0000 7f2a267fc640 1 -- 172.21.15.120:0/3211200339 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a2c044070 con 0x7f2a300c6520 2024-09-10T10:27:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.289+0000 7f3bb9066640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f3b980755f0 0x7f3b98077a30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.289+0000 7f3bb3fff640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b9806be80 0x7f3b9806e340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ba802a420 tx=0x7f3ba802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.289+0000 7f2a267fc640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2a1406be30 0x7f2a1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.289+0000 7f2a267fc640 1 -- 172.21.15.120:0/3211200339 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2a2c0ac220 con 0x7f2a300c6520 2024-09-10T10:27:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.289+0000 7f2a267fc640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2a140755a0 0x7f2a140779e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.289+0000 7f2a267fc640 1 -- 172.21.15.120:0/3211200339 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a2c032440 con 0x7f2a140755a0 2024-09-10T10:27:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.289+0000 7f2a267fc640 1 -- 172.21.15.120:0/3211200339 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a2c0ac610 con 0x7f2a300c6520 2024-09-10T10:27:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.290+0000 7f3bb17fa640 1 -- 172.21.15.120:0/2862873466 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b98078100 con 0x7f3b980755f0 2024-09-10T10:27:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.290+0000 7f2a36411640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2a140755a0 0x7f2a140779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.290+0000 7f2a3540f640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2a1406be30 0x7f2a1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.290+0000 7f98a57fa640 1 -- 172.21.15.120:0/414838768 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9894007e50 con 0x7f988c075670 2024-09-10T10:27:08.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.290+0000 7f2a3540f640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2a1406be30 0x7f2a1406e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2a301e0bf0 tx=0x7f2a20007ee0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.290+0000 7f2a36411640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2a140755a0 0x7f2a140779e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.291+0000 7f2a267fc640 1 -- 172.21.15.120:0/3211200339 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2a2c032440 con 0x7f2a140755a0 2024-09-10T10:27:08.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.314+0000 7fcd2359e640 1 -- 172.21.15.120:0/1673824455 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd08002dd0 con 0x7fcd040755a0 2024-09-10T10:27:08.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.315+0000 7fcd12ffd640 1 -- 172.21.15.120:0/1673824455 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcd08002dd0 con 0x7fcd040755a0 2024-09-10T10:27:08.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.315+0000 7fcd10ff9640 1 -- 172.21.15.120:0/1673824455 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fcd040755a0 msgr2=0x7fcd040779e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.315+0000 7fcd10ff9640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fcd040755a0 0x7fcd040779e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.316+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84e0002dd0 con 0x7f84ec075520 2024-09-10T10:27:08.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.316+0000 7fcd10ff9640 1 -- 172.21.15.120:0/1673824455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcd0406be30 msgr2=0x7fcd0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.316+0000 7fcd10ff9640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcd0406be30 0x7fcd0406e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd14034000 tx=0x7fcd1402e000 comp rx=0 tx=0).stop 2024-09-10T10:27:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.316+0000 7fcd10ff9640 1 -- 172.21.15.120:0/1673824455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd1c04b820 msgr2=0x7fcd1c049150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.316+0000 7fcd10ff9640 1 --2- 172.21.15.120:0/1673824455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd1c04b820 0x7fcd1c049150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcd2406a930 tx=0x7fcd24069890 comp rx=0 tx=0).stop 2024-09-10T10:27:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.316+0000 7fcd22d9d640 1 -- 172.21.15.120:0/1673824455 reap_dead start 2024-09-10T10:27:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.316+0000 7fcd10ff9640 1 -- 172.21.15.120:0/1673824455 shutdown_connections 2024-09-10T10:27:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.316+0000 7fcd10ff9640 1 -- 172.21.15.120:0/1673824455 >> 172.21.15.120:0/1673824455 conn(0x7fcd1c050090 msgr2=0x7fcd1c0e0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.317+0000 7f84fb7fe640 1 -- 172.21.15.120:0/2041471956 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f84e0002dd0 con 0x7f84ec075520 2024-09-10T10:27:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.317+0000 7fcd10ff9640 1 -- 172.21.15.120:0/1673824455 shutdown_connections 2024-09-10T10:27:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.317+0000 7f13634e7640 1 -- 172.21.15.120:0/952849144 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f135c0d1030 con 0x7f13400755a0 2024-09-10T10:27:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.317+0000 7fcd10ff9640 1 -- 172.21.15.120:0/1673824455 wait complete. 2024-09-10T10:27:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f84ec075520 msgr2=0x7f84ec077960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f85091a6640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f84ec075520 0x7f84ec077960 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f84ec06be30 msgr2=0x7f84ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f85091a6640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f84ec06be30 0x7f84ec06e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8504162000 tx=0x7f84f0009040 comp rx=0 tx=0).stop 2024-09-10T10:27:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85040c8b10 msgr2=0x7f8504151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f85091a6640 1 --2- 172.21.15.120:0/2041471956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85040c8b10 0x7f8504151d80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f84fc030000 tx=0x7f84fc004a40 comp rx=0 tx=0).stop 2024-09-10T10:27:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f850359e640 1 -- 172.21.15.120:0/2041471956 reap_dead start 2024-09-10T10:27:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f1352ffd640 1 -- 172.21.15.120:0/952849144 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f135c0d1030 con 0x7f13400755a0 2024-09-10T10:27:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 shutdown_connections 2024-09-10T10:27:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 >> 172.21.15.120:0/2041471956 conn(0x7f850406f3c0 msgr2=0x7f8504156de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 shutdown_connections 2024-09-10T10:27:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f85091a6640 1 -- 172.21.15.120:0/2041471956 wait complete. 2024-09-10T10:27:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f1350ff9640 1 -- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f13400755a0 msgr2=0x7f13400779e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.318+0000 7f1350ff9640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f13400755a0 0x7f13400779e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f1350ff9640 1 -- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f134006be30 msgr2=0x7f134006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f1350ff9640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f134006be30 0x7f134006e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1358034000 tx=0x7f135802e000 comp rx=0 tx=0).stop 2024-09-10T10:27:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f1350ff9640 1 -- 172.21.15.120:0/952849144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f135c157ec0 msgr2=0x7f135c1503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f1350ff9640 1 --2- 172.21.15.120:0/952849144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f135c157ec0 0x7f135c1503d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f135400c820 tx=0x7f135400cce0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f1362ce6640 1 -- 172.21.15.120:0/952849144 reap_dead start 2024-09-10T10:27:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3bac04d5a0 con 0x7f3b980755f0 2024-09-10T10:27:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f1350ff9640 1 -- 172.21.15.120:0/952849144 shutdown_connections 2024-09-10T10:27:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f1350ff9640 1 -- 172.21.15.120:0/952849144 >> 172.21.15.120:0/952849144 conn(0x7f135c067950 msgr2=0x7f135c154520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f1350ff9640 1 -- 172.21.15.120:0/952849144 shutdown_connections 2024-09-10T10:27:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f1350ff9640 1 -- 172.21.15.120:0/952849144 wait complete. 2024-09-10T10:27:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.319+0000 7f3bb17fa640 1 -- 172.21.15.120:0/2862873466 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3bac04d5a0 con 0x7f3b980755f0 2024-09-10T10:27:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff11c002dd0 con 0x7ff128075670 2024-09-10T10:27:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f3b980755f0 msgr2=0x7f3b98077a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7f3bbaac9640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f3b980755f0 0x7f3b98077a30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b9806be80 msgr2=0x7f3b9806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7f3bbaac9640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b9806be80 0x7f3b9806e340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ba802a420 tx=0x7f3ba802e000 comp rx=0 tx=0).stop 2024-09-10T10:27:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bac005a30 msgr2=0x7f3bac049860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7f3bbaac9640 1 --2- 172.21.15.120:0/2862873466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bac005a30 0x7f3bac049860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c3f0 tx=0x7f3ba000c8b0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7f3bb9066640 1 -- 172.21.15.120:0/2862873466 reap_dead start 2024-09-10T10:27:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7ff138ff9640 1 -- 172.21.15.120:0/2517997545 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff11c002dd0 con 0x7ff128075670 2024-09-10T10:27:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 shutdown_connections 2024-09-10T10:27:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.320+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 >> 172.21.15.120:0/2862873466 conn(0x7f3bac04fc80 msgr2=0x7f3bac04ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.321+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 shutdown_connections 2024-09-10T10:27:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.321+0000 7f3bbaac9640 1 -- 172.21.15.120:0/2862873466 wait complete. 2024-09-10T10:27:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.321+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f89000cbdc0 con 0x7f88e40755f0 2024-09-10T10:27:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.321+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff128075670 msgr2=0x7ff128077ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.321+0000 7ff142933640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff128075670 0x7ff128077ab0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.321+0000 7f88f1ffb640 1 -- 172.21.15.120:0/1727437275 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f89000cbdc0 con 0x7f88e40755f0 2024-09-10T10:27:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f98a0005a00 con 0x7f988c075670 2024-09-10T10:27:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f88e40755f0 msgr2=0x7f88e4077a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7f89076dd640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f88e40755f0 0x7f88e4077a30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f88e406bf00 msgr2=0x7f88e406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7f89076dd640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f88e406bf00 0x7f88e406e3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88f4007fd0 tx=0x7f88f4009040 comp rx=0 tx=0).stop 2024-09-10T10:27:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89000c6980 msgr2=0x7f8900159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7f89076dd640 1 --2- 172.21.15.120:0/1727437275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89000c6980 0x7f8900159510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00efc0 tx=0x7f88fc00c490 comp rx=0 tx=0).stop 2024-09-10T10:27:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7f8905c7a640 1 -- 172.21.15.120:0/1727437275 reap_dead start 2024-09-10T10:27:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff12806bf00 msgr2=0x7ff12806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7ff142933640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff12806bf00 0x7ff12806e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff130009560 tx=0x7ff130009190 comp rx=0 tx=0).stop 2024-09-10T10:27:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff13c159320 msgr2=0x7ff13c15ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7ff142933640 1 --2- 172.21.15.120:0/2517997545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff13c159320 0x7ff13c15ffd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00e8d0 tx=0x7ff12c00ed90 comp rx=0 tx=0).stop 2024-09-10T10:27:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7ff140ed0640 1 -- 172.21.15.120:0/2517997545 reap_dead start 2024-09-10T10:27:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 shutdown_connections 2024-09-10T10:27:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 >> 172.21.15.120:0/2517997545 conn(0x7ff13c06f3c0 msgr2=0x7ff13c06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 shutdown_connections 2024-09-10T10:27:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.322+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2a18002d50 con 0x7f2a140755a0 2024-09-10T10:27:08.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 shutdown_connections 2024-09-10T10:27:08.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 >> 172.21.15.120:0/1727437275 conn(0x7f890006cb70 msgr2=0x7f89000d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7ff142933640 1 -- 172.21.15.120:0/2517997545 wait complete. 2024-09-10T10:27:08.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7f98a57fa640 1 -- 172.21.15.120:0/414838768 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f98a0005a00 con 0x7f988c075670 2024-09-10T10:27:08.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 shutdown_connections 2024-09-10T10:27:08.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7f89076dd640 1 -- 172.21.15.120:0/1727437275 wait complete. 2024-09-10T10:27:08.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f988c075670 msgr2=0x7f988c077ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7f98adbf1640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f988c075670 0x7f988c077ab0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7f2a267fc640 1 -- 172.21.15.120:0/3211200339 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2a18002d50 con 0x7f2a140755a0 2024-09-10T10:27:08.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2a140755a0 msgr2=0x7f2a140779e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.323+0000 7f2a37e74640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2a140755a0 0x7f2a140779e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f988c06bf00 msgr2=0x7f988c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f98adbf1640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f988c06bf00 0x7f988c06e3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f989c009630 tx=0x7f989c02e000 comp rx=0 tx=0).stop 2024-09-10T10:27:08.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98a00dce90 msgr2=0x7f98a0184700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f98adbf1640 1 --2- 172.21.15.120:0/414838768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98a00dce90 0x7f98a0184700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f989400ec10 tx=0x7f989400c520 comp rx=0 tx=0).stop 2024-09-10T10:27:08.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f98ad3f0640 1 -- 172.21.15.120:0/414838768 reap_dead start 2024-09-10T10:27:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2a1406be30 msgr2=0x7f2a1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f2a37e74640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2a1406be30 0x7f2a1406e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2a301e0bf0 tx=0x7f2a20007ee0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a300c6520 msgr2=0x7f2a300cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f2a37e74640 1 --2- 172.21.15.120:0/3211200339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a300c6520 0x7f2a300cd5d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c031000 tx=0x7f2a2c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f2a36411640 1 -- 172.21.15.120:0/3211200339 reap_dead start 2024-09-10T10:27:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 shutdown_connections 2024-09-10T10:27:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 >> 172.21.15.120:0/3211200339 conn(0x7f2a30068140 msgr2=0x7f2a300c92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.324+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 shutdown_connections 2024-09-10T10:27:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.325+0000 7f2a37e74640 1 -- 172.21.15.120:0/3211200339 wait complete. 2024-09-10T10:27:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.326+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 shutdown_connections 2024-09-10T10:27:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.326+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 >> 172.21.15.120:0/414838768 conn(0x7f98a004fc50 msgr2=0x7f98a004c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.326+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 shutdown_connections 2024-09-10T10:27:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.326+0000 7f98adbf1640 1 -- 172.21.15.120:0/414838768 wait complete. 2024-09-10T10:27:08.383 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:08.383 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T10:27:08.384 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:08.384 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T10:27:08.385 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:08.385 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T10:27:08.386 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:08.386 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T10:27:08.387 INFO:teuthology.orchestra.run.smithi120.stdout:55834574858 2024-09-10T10:27:08.387 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T10:27:08.388 INFO:teuthology.orchestra.run.smithi120.stdout:55834574858 2024-09-10T10:27:08.388 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T10:27:08.389 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:08.389 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T10:27:08.390 INFO:teuthology.orchestra.run.smithi120.stdout:55834574858 2024-09-10T10:27:08.391 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T10:27:08.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.587+0000 7f9cd452c640 1 Processor -- start 2024-09-10T10:27:08.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.588+0000 7f9cd452c640 1 -- start start 2024-09-10T10:27:08.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.589+0000 7f2358b62640 1 Processor -- start 2024-09-10T10:27:08.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.589+0000 7f9cd452c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9ccc0c72f0 0x7f9ccc0c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.589+0000 7f9cd452c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc073f80 0x7f9ccc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.589+0000 7f9cd452c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ccc15dfb0 con 0x7f9ccc0c8350 2024-09-10T10:27:08.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.589+0000 7f9cd452c640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9ccc0c6b20 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.589+0000 7f9cd452c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ccc155140 con 0x7f9ccc073f80 2024-09-10T10:27:08.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.589+0000 7f2358b62640 1 -- start start 2024-09-10T10:27:08.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.589+0000 7fc924933640 1 Processor -- start 2024-09-10T10:27:08.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f9cd1ac7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc073f80 0x7f9ccc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f9cd1ac7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc073f80 0x7f9ccc071a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:42798/0 (socket says 172.21.15.120:42798) 2024-09-10T10:27:08.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f9cd1ac7640 1 -- 172.21.15.120:0/1628569811 learned_addr learned my addr 172.21.15.120:0/1628569811 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f3ab0ea6640 1 Processor -- start 2024-09-10T10:27:08.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7fc924933640 1 -- start start 2024-09-10T10:27:08.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f658c111640 1 Processor -- start 2024-09-10T10:27:08.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f9cd22c8640 1 --2- 172.21.15.120:0/1628569811 >> v2:172.21.15.120:3300/0 conn(0x7f9ccc0c72f0 0x7f9ccc0c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f9cd1ac7640 1 -- 172.21.15.120:0/1628569811 >> v2:172.21.15.120:3300/0 conn(0x7f9ccc0c72f0 msgr2=0x7f9ccc0c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f9cd1ac7640 1 --2- 172.21.15.120:0/1628569811 >> v2:172.21.15.120:3300/0 conn(0x7f9ccc0c72f0 0x7f9ccc0c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f9cd1ac7640 1 -- 172.21.15.120:0/1628569811 >> v1:172.21.15.120:6789/0 conn(0x7f9ccc0c8350 legacy=0x7f9ccc15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f9cd1ac7640 1 -- 172.21.15.120:0/1628569811 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc15e290 con 0x7f9ccc073f80 2024-09-10T10:27:08.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f2358b62640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234c0dcd90 0x7f234c045d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2358b62640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f234c0e6640 0x7f234c04aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2358b62640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f234c0e6d30 con 0x7f234c04a680 2024-09-10T10:27:08.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2358b62640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f234c044d40 con 0x7f234c0e6640 2024-09-10T10:27:08.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2358b62640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f234c0dd170 con 0x7f234c0dcd90 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2352ffd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f234c04a680 0x7f234c0e5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49144/0 (socket says 172.21.15.120:49144) 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2352ffd640 1 -- 172.21.15.120:0/3204537316 learned_addr learned my addr 172.21.15.120:0/3204537316 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2353fff640 1 --2- 172.21.15.120:0/3204537316 >> v2:172.21.15.134:3300/0 conn(0x7f234c0e6640 0x7f234c04aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2353fff640 1 -- 172.21.15.120:0/3204537316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234c0dcd90 msgr2=0x7f234c045d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2353fff640 1 --2- 172.21.15.120:0/3204537316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234c0dcd90 0x7f234c045d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2353fff640 1 -- 172.21.15.120:0/3204537316 >> v1:172.21.15.120:6789/0 conn(0x7f234c04a680 legacy=0x7f234c0e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7fc924933640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc920158f40 0x7fc92015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7fc924933640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc92015f430 con 0x7fc920159320 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7fc924933640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc92015f710 con 0x7fc9200c8b10 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7fc924933640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc920162770 con 0x7fc920158f40 2024-09-10T10:27:08.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7fc91e59c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc920158f40 0x7fc92015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.590+0000 7f3ab0ea6640 1 -- start start 2024-09-10T10:27:08.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f3ab0ea6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac158f40 0x7f3aac15ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f3ab0ea6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3aac159320 0x7f3aac0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f3ab0ea6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aac15f430 con 0x7f3aac162c00 2024-09-10T10:27:08.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f3ab0ea6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3aac162770 con 0x7f3aac159320 2024-09-10T10:27:08.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f3ab0ea6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3aac15f710 con 0x7f3aac158f40 2024-09-10T10:27:08.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7fc91e59c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc920158f40 0x7fc92015ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34402/0 (socket says 172.21.15.120:34402) 2024-09-10T10:27:08.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7fc91dd9b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fc920159320 0x7fc920162c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49156/0 (socket says 172.21.15.120:49156) 2024-09-10T10:27:08.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7fc91e59c640 1 -- 172.21.15.120:0/3491228728 learned_addr learned my addr 172.21.15.120:0/3491228728 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f658c111640 1 -- start start 2024-09-10T10:27:08.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f3aaad9d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f3aac162c00 0x7f3aac0c7430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49172/0 (socket says 172.21.15.120:49172) 2024-09-10T10:27:08.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7fdbacd98640 1 Processor -- start 2024-09-10T10:27:08.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f3aaad9d640 1 -- 172.21.15.120:0/863001356 learned_addr learned my addr 172.21.15.120:0/863001356 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:08.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f9cd1ac7640 1 --2- 172.21.15.120:0/1628569811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc073f80 0x7f9ccc071a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009870 tx=0x7f9cbc02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5642b96294285d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7fdbacd98640 1 -- start start 2024-09-10T10:27:08.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7fc91e59c640 1 -- 172.21.15.120:0/3491228728 >> v1:172.21.15.134:6789/0 conn(0x7fc9200c8b10 legacy=0x7fc9200c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f3aaa59c640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac158f40 0x7f3aac15ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.591+0000 7f2353fff640 1 -- 172.21.15.120:0/3204537316 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c044f90 con 0x7f234c0e6640 2024-09-10T10:27:08.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7fc91d59a640 1 -- 172.21.15.120:0/3491228728 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1283073411 0 0) 0x7fc92015f430 con 0x7fc920159320 2024-09-10T10:27:08.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f9cd12c6640 1 -- 172.21.15.120:0/1628569811 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc037070 con 0x7f9ccc073f80 2024-09-10T10:27:08.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7fc91e59c640 1 -- 172.21.15.120:0/3491228728 >> v1:172.21.15.120:6789/0 conn(0x7fc920159320 legacy=0x7fc920162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f658c111640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65840cc0c0 0x7f658415aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f658c111640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65840cc4a0 0x7f6584165430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f658c111640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f658415b1d0 con 0x7f65840cb2f0 2024-09-10T10:27:08.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f658c111640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f65840cbd00 con 0x7f65840cc4a0 2024-09-10T10:27:08.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f658c111640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f658415ba70 con 0x7f65840cc0c0 2024-09-10T10:27:08.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f9cd12c6640 1 -- 172.21.15.120:0/1628569811 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cbc02ae50 con 0x7f9ccc073f80 2024-09-10T10:27:08.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f01f9bd6640 1 Processor -- start 2024-09-10T10:27:08.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f01f9bd6640 1 -- start start 2024-09-10T10:27:08.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f2353fff640 1 --2- 172.21.15.120:0/3204537316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f234c0e6640 0x7f234c04aa80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2354069460 tx=0x7f23540639f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e454a8ab52e147a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7fc91e59c640 1 -- 172.21.15.120:0/3491228728 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92015f9f0 con 0x7fc920158f40 2024-09-10T10:27:08.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.592+0000 7f3aa9d9b640 1 --2- 172.21.15.120:0/863001356 >> v2:172.21.15.134:3300/0 conn(0x7f3aac159320 0x7f3aac0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f6589ead640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65840cc0c0 0x7f658415aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f658a6ae640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65840cc4a0 0x7f6584165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f658a6ae640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65840cc4a0 0x7f6584165430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41780/0 (socket says 172.21.15.120:41780) 2024-09-10T10:27:08.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f658a6ae640 1 -- 172.21.15.120:0/1868161647 learned_addr learned my addr 172.21.15.120:0/1868161647 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f658a6ae640 1 -- 172.21.15.120:0/1868161647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65840cc0c0 msgr2=0x7f658415aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f658a6ae640 1 --2- 172.21.15.120:0/1868161647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65840cc0c0 0x7f658415aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f658a6ae640 1 -- 172.21.15.120:0/1868161647 >> v1:172.21.15.120:6789/0 conn(0x7f65840cb2f0 legacy=0x7f658415e9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:27:08.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f658a6ae640 1 -- 172.21.15.120:0/1868161647 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658415b4b0 con 0x7f65840cc4a0 2024-09-10T10:27:08.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f23527fc640 1 -- 172.21.15.120:0/3204537316 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23540a8990 con 0x7f234c0e6640 2024-09-10T10:27:08.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f23527fc640 1 -- 172.21.15.120:0/3204537316 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23540a8b70 con 0x7f234c0e6640 2024-09-10T10:27:08.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f23527fc640 1 -- 172.21.15.120:0/3204537316 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23540a8990 con 0x7f234c0e6640 2024-09-10T10:27:08.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f9cd452c640 1 -- 172.21.15.120:0/1628569811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc073f80 msgr2=0x7f9ccc071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f9cd12c6640 1 -- 172.21.15.120:0/1628569811 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc037070 con 0x7f9ccc073f80 2024-09-10T10:27:08.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308044661 0 0) 0x7f3aac15f430 con 0x7f3aac162c00 2024-09-10T10:27:08.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a94003520 con 0x7f3aac162c00 2024-09-10T10:27:08.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f9cd452c640 1 --2- 172.21.15.120:0/1628569811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc073f80 0x7f9ccc071a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009870 tx=0x7f9cbc02a300 comp rx=0 tx=0).stop 2024-09-10T10:27:08.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f9cd452c640 1 -- 172.21.15.120:0/1628569811 shutdown_connections 2024-09-10T10:27:08.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f9cd452c640 1 --2- 172.21.15.120:0/1628569811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc073f80 0x7f9ccc071a60 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009870 tx=0x7f9cbc02a300 comp rx=0 tx=0).stop 2024-09-10T10:27:08.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f9cd452c640 1 --2- 172.21.15.120:0/1628569811 >> v2:172.21.15.120:3300/0 conn(0x7f9ccc0c72f0 0x7f9ccc0c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f9cd452c640 1 -- 172.21.15.120:0/1628569811 >> 172.21.15.120:0/1628569811 conn(0x7f9ccc0cbf90 msgr2=0x7f9ccc0cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1391455855 0 0) 0x7f3a94003520 con 0x7f3aac162c00 2024-09-10T10:27:08.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac158f40 msgr2=0x7f3aac15ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac158f40 0x7f3aac15ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 >> v2:172.21.15.134:3300/0 conn(0x7f3aac159320 msgr2=0x7f3aac0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 --2- 172.21.15.120:0/863001356 >> v2:172.21.15.134:3300/0 conn(0x7f3aac159320 0x7f3aac0c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac15f9f0 con 0x7f3aac162c00 2024-09-10T10:27:08.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f3aa40019f0 con 0x7f3aac162c00 2024-09-10T10:27:08.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.593+0000 7fdbacd98640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdba00e5fb0 0x7fdba00501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7fdbacd98640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 0x7fdba00e6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7fdbacd98640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdba00e6d40 con 0x7fdba004f0a0 2024-09-10T10:27:08.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7fdba5d9b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fdba004f0a0 0x7fdba004f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49174/0 (socket says 172.21.15.120:49174) 2024-09-10T10:27:08.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7fdba5d9b640 1 -- 172.21.15.120:0/4227931140 learned_addr learned my addr 172.21.15.120:0/4227931140 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:08.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7fdbacd98640 1 -- 172.21.15.120:0/4227931140 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdba0049020 con 0x7fdba00e5fb0 2024-09-10T10:27:08.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7fdbacd98640 1 -- 172.21.15.120:0/4227931140 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdba0039520 con 0x7fdba00e6390 2024-09-10T10:27:08.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f01f9bd6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f01f4159200 0x7f01f40c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f01f9bd6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f4163370 con 0x7f01f40cc480 2024-09-10T10:27:08.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f01f9bd6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f415fe20 con 0x7f01f40c8ff0 2024-09-10T10:27:08.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f01f9bd6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f01f415f990 con 0x7f01f4159200 2024-09-10T10:27:08.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 >> v1:172.21.15.120:6789/0 conn(0x7f3aac162c00 legacy=0x7f3aac0c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f01f37fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f01f4159200 0x7f01f40c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f9cd452c640 1 -- 172.21.15.120:0/1628569811 shutdown_connections 2024-09-10T10:27:08.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a940039b0 0x7f3a94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f01f3fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f01f40cc480 0x7f01f4162450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49182/0 (socket says 172.21.15.120:49182) 2024-09-10T10:27:08.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f01f3fff640 1 -- 172.21.15.120:0/1046333871 learned_addr learned my addr 172.21.15.120:0/1046333871 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:08.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7ff6c9f8d640 1 Processor -- start 2024-09-10T10:27:08.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac158f40 0x7f3a94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7fdba6d9d640 1 --2- 172.21.15.120:0/4227931140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 0x7fdba00e6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94004d40 0x7f3a940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac162770 con 0x7f3a940039b0 2024-09-10T10:27:08.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f2358b62640 1 -- 172.21.15.120:0/3204537316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f234c0e6640 msgr2=0x7f234c04aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f2358b62640 1 --2- 172.21.15.120:0/3204537316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f234c0e6640 0x7f234c04aa80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2354069460 tx=0x7f23540639f0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac15f710 con 0x7f3a94004d40 2024-09-10T10:27:08.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7fdba659c640 1 --2- 172.21.15.120:0/4227931140 >> v2:172.21.15.134:3300/0 conn(0x7fdba00e5fb0 0x7fdba00501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9400bad0 con 0x7f3aac158f40 2024-09-10T10:27:08.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7ff6c9f8d640 1 -- start start 2024-09-10T10:27:08.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f01f37fe640 1 -- 172.21.15.120:0/1046333871 >> v1:172.21.15.134:6789/0 conn(0x7f01f40c8ff0 legacy=0x7f01f40d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f3aaa59c640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a940039b0 0x7f3a94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f01f37fe640 1 -- 172.21.15.120:0/1046333871 >> v1:172.21.15.120:6789/0 conn(0x7f01f40cc480 legacy=0x7f01f4162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f01f37fe640 1 -- 172.21.15.120:0/1046333871 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f4163090 con 0x7f01f4159200 2024-09-10T10:27:08.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f658a6ae640 1 --2- 172.21.15.120:0/1868161647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cc4a0 0x7f6584165430 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6580009870 tx=0x7f658002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56761f657063c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7fc91e59c640 1 --2- 172.21.15.120:0/3491228728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc920158f40 0x7fc92015ee10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc910009a50 tx=0x7fc91002a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848c0cc5fb0756bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7fdba559a640 1 -- 172.21.15.120:0/4227931140 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1755915172 0 0) 0x7fdba00e6d40 con 0x7fdba004f0a0 2024-09-10T10:27:08.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7fdba559a640 1 -- 172.21.15.120:0/4227931140 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb90003520 con 0x7fdba004f0a0 2024-09-10T10:27:08.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.594+0000 7f9cd452c640 1 -- 172.21.15.120:0/1628569811 wait complete. 2024-09-10T10:27:08.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f2358b62640 1 -- 172.21.15.120:0/3204537316 shutdown_connections 2024-09-10T10:27:08.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f2358b62640 1 --2- 172.21.15.120:0/3204537316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f234c0e6640 0x7f234c04aa80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f2358b62640 1 --2- 172.21.15.120:0/3204537316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234c0dcd90 0x7f234c045d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f2358b62640 1 -- 172.21.15.120:0/3204537316 >> 172.21.15.120:0/3204537316 conn(0x7f234c04fc50 msgr2=0x7f234c050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f2358b62640 1 -- 172.21.15.120:0/3204537316 shutdown_connections 2024-09-10T10:27:08.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f2358b62640 1 -- 172.21.15.120:0/3204537316 wait complete. 2024-09-10T10:27:08.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f6588eab640 1 -- 172.21.15.120:0/1868161647 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580037070 con 0x7f65840cc4a0 2024-09-10T10:27:08.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7fc91d59a640 1 -- 172.21.15.120:0/3491228728 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910036070 con 0x7fc920158f40 2024-09-10T10:27:08.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f3aaad9d640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94004d40 0x7f3a940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7fc91d59a640 1 -- 172.21.15.120:0/3491228728 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc910031600 con 0x7fc920158f40 2024-09-10T10:27:08.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f3aaa59c640 1 -- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac158f40 msgr2=0x7f3a94004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:27:08.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f3aaa59c640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac158f40 0x7f3a94004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7fc91d59a640 1 -- 172.21.15.120:0/3491228728 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910036070 con 0x7fc920158f40 2024-09-10T10:27:08.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7fdba6d9d640 1 -- 172.21.15.120:0/4227931140 >> v2:172.21.15.134:3300/0 conn(0x7fdba00e5fb0 msgr2=0x7fdba00501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7fdba6d9d640 1 --2- 172.21.15.120:0/4227931140 >> v2:172.21.15.134:3300/0 conn(0x7fdba00e5fb0 0x7fdba00501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7fdba6d9d640 1 -- 172.21.15.120:0/4227931140 >> v1:172.21.15.120:6789/0 conn(0x7fdba004f0a0 legacy=0x7fdba004f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7fdba6d9d640 1 -- 172.21.15.120:0/4227931140 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba0049f30 con 0x7fdba00e6390 2024-09-10T10:27:08.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.595+0000 7f3aaa59c640 1 -- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94004d40 msgr2=0x7f3a940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f01f37fe640 1 --2- 172.21.15.120:0/1046333871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4159200 0x7f01f40c8ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01e40097a0 tx=0x7f01e402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15ef8d3e6c8627b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f3aaa59c640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94004d40 0x7f3a940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f6588eab640 1 -- 172.21.15.120:0/1868161647 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6580032440 con 0x7f65840cc4a0 2024-09-10T10:27:08.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f6588eab640 1 -- 172.21.15.120:0/1868161647 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580032810 con 0x7f65840cc4a0 2024-09-10T10:27:08.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f3aaa59c640 1 -- 172.21.15.120:0/863001356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a94003520 con 0x7f3a940039b0 2024-09-10T10:27:08.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f3aaad9d640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a94004d40 0x7f3a940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:08.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7ff6c9f8d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff6c4158ea0 0x7ff6c40c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7ff6c9f8d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff6c4159280 0x7ff6c415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7ff6c9f8d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 0x7ff6c415f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7ff6c9f8d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c4072540 con 0x7ff6c4159280 2024-09-10T10:27:08.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7ff6c9f8d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c4072de0 con 0x7ff6c4158ea0 2024-09-10T10:27:08.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7ff6c9f8d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c415f820 con 0x7ff6c4073270 2024-09-10T10:27:08.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f9cd452c640 1 Processor -- start 2024-09-10T10:27:08.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f01f27fc640 1 -- 172.21.15.120:0/1046333871 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4037070 con 0x7f01f4159200 2024-09-10T10:27:08.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f9cd452c640 1 -- start start 2024-09-10T10:27:08.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f01f27fc640 1 -- 172.21.15.120:0/1046333871 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e4032440 con 0x7f01f4159200 2024-09-10T10:27:08.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7ff6c8f8b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff6c4158ea0 0x7ff6c40c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7ff6c8f8b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff6c4158ea0 0x7ff6c40c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41816/0 (socket says 172.21.15.120:41816) 2024-09-10T10:27:08.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7f01f27fc640 1 -- 172.21.15.120:0/1046333871 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4032810 con 0x7f01f4159200 2024-09-10T10:27:08.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7ff6c8f8b640 1 -- 172.21.15.120:0/4053353004 learned_addr learned my addr 172.21.15.120:0/4053353004 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7fc924933640 1 -- 172.21.15.120:0/3491228728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc920158f40 msgr2=0x7fc92015ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7fc924933640 1 --2- 172.21.15.120:0/3491228728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc920158f40 0x7fc92015ee10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc910009a50 tx=0x7fc91002a490 comp rx=0 tx=0).stop 2024-09-10T10:27:08.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7ff6c978c640 1 --2- 172.21.15.120:0/4053353004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 0x7ff6c415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7fc924933640 1 -- 172.21.15.120:0/3491228728 shutdown_connections 2024-09-10T10:27:08.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7fc924933640 1 --2- 172.21.15.120:0/3491228728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc920158f40 0x7fc92015ee10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7ff6c8f8b640 1 -- 172.21.15.120:0/4053353004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 msgr2=0x7ff6c415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7ff6c8f8b640 1 --2- 172.21.15.120:0/4053353004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 0x7ff6c415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f658c111640 1 -- 172.21.15.120:0/1868161647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cc4a0 msgr2=0x7f6584165430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f658c111640 1 --2- 172.21.15.120:0/1868161647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cc4a0 0x7f6584165430 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6580009870 tx=0x7f658002a210 comp rx=0 tx=0).stop 2024-09-10T10:27:08.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7ff6c8f8b640 1 -- 172.21.15.120:0/4053353004 >> v2:172.21.15.120:3300/0 conn(0x7ff6c4159280 msgr2=0x7ff6c415ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:08.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7ff6c3fff640 1 --2- 172.21.15.120:0/4053353004 >> v2:172.21.15.120:3300/0 conn(0x7ff6c4159280 0x7ff6c415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7fdba6d9d640 1 --2- 172.21.15.120:0/4227931140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 0x7fdba00e6800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdba80518b0 tx=0x7fdba80639f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c516447b5d7e1734 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f2358b62640 1 Processor -- start 2024-09-10T10:27:08.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7ff6c8f8b640 1 --2- 172.21.15.120:0/4053353004 >> v2:172.21.15.120:3300/0 conn(0x7ff6c4159280 0x7ff6c415ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f658c111640 1 -- 172.21.15.120:0/1868161647 shutdown_connections 2024-09-10T10:27:08.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f658c111640 1 --2- 172.21.15.120:0/1868161647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cc4a0 0x7f6584165430 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f658c111640 1 --2- 172.21.15.120:0/1868161647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65840cc0c0 0x7f658415aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f658c111640 1 -- 172.21.15.120:0/1868161647 >> 172.21.15.120:0/1868161647 conn(0x7f658406d560 msgr2=0x7f658406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.596+0000 7fc924933640 1 -- 172.21.15.120:0/3491228728 >> 172.21.15.120:0/3491228728 conn(0x7fc92006f3c0 msgr2=0x7fc92006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7ff6c8f8b640 1 -- 172.21.15.120:0/4053353004 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c415fb00 con 0x7ff6c4158ea0 2024-09-10T10:27:08.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f3aaa59c640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a940039b0 0x7f3a94003dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000ba30 tx=0x7f3a94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2102e5be069907f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7fc924933640 1 -- 172.21.15.120:0/3491228728 shutdown_connections 2024-09-10T10:27:08.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f01f9bd6640 1 -- 172.21.15.120:0/1046333871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4159200 msgr2=0x7f01f40c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f01f9bd6640 1 --2- 172.21.15.120:0/1046333871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4159200 0x7f01f40c8ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01e40097a0 tx=0x7f01e402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f2358b62640 1 -- start start 2024-09-10T10:27:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7ff6c3fff640 1 --2- 172.21.15.120:0/4053353004 >> v2:172.21.15.120:3300/0 conn(0x7ff6c4159280 0x7ff6c415ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f01f9bd6640 1 -- 172.21.15.120:0/1046333871 shutdown_connections 2024-09-10T10:27:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f01f9bd6640 1 --2- 172.21.15.120:0/1046333871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4159200 0x7f01f40c8ab0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f01f9bd6640 1 -- 172.21.15.120:0/1046333871 >> 172.21.15.120:0/1046333871 conn(0x7f01f406d7a0 msgr2=0x7f01f406e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f01f9bd6640 1 -- 172.21.15.120:0/1046333871 shutdown_connections 2024-09-10T10:27:08.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0030070 con 0x7f3a940039b0 2024-09-10T10:27:08.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f9cd452c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ccc0c72f0 0x7f9ccc0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f3aa959a640 1 -- 172.21.15.120:0/863001356 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa002ae00 con 0x7f3a940039b0 2024-09-10T10:27:08.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7f9cd452c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc0c8350 0x7f9ccc0c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f9cd22c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ccc0c72f0 0x7f9ccc0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f9cd22c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ccc0c72f0 0x7f9ccc0c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41832/0 (socket says 172.21.15.120:41832) 2024-09-10T10:27:08.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f9cd452c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ccc0c52c0 0x7f9ccc15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f9cd1ac7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc0c8350 0x7f9ccc0c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7fc924933640 1 -- 172.21.15.120:0/3491228728 wait complete. 2024-09-10T10:27:08.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f9cd1ac7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc0c8350 0x7f9ccc0c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:42864/0 (socket says 172.21.15.120:42864) 2024-09-10T10:27:08.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7ff6c8f8b640 1 --2- 172.21.15.120:0/4053353004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6c4158ea0 0x7ff6c40c7030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff6c406e750 tx=0x7ff6bc02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf7b7fc2ec3df67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f2358b62640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f234c04a680 0x7f234c17c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7fdba559a640 1 -- 172.21.15.120:0/4227931140 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80a8910 con 0x7fdba00e6390 2024-09-10T10:27:08.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7fdba559a640 1 -- 172.21.15.120:0/4227931140 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba80a8ab0 con 0x7fdba00e6390 2024-09-10T10:27:08.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.597+0000 7fdba559a640 1 -- 172.21.15.120:0/4227931140 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80a9420 con 0x7fdba00e6390 2024-09-10T10:27:08.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7fdbacd98640 1 -- 172.21.15.120:0/4227931140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 msgr2=0x7fdba00e6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7fdbacd98640 1 --2- 172.21.15.120:0/4227931140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 0x7fdba00e6800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdba80518b0 tx=0x7fdba80639f0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f2358b62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f234c0dcd90 0x7f234c049c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f2358b62640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234c0e6640 0x7f234c16d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f23537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f234c04a680 0x7f234c17c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f2358b62640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f234c045550 con 0x7f234c04a680 2024-09-10T10:27:08.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f23537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f234c04a680 0x7f234c17c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34434/0 (socket says 172.21.15.120:34434) 2024-09-10T10:27:08.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f2358b62640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0e6ad0 con 0x7f234c0dcd90 2024-09-10T10:27:08.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f2358b62640 1 -- 172.21.15.120:0/257194926 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f234c045830 con 0x7f234c0e6640 2024-09-10T10:27:08.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f23537fe640 1 -- 172.21.15.120:0/257194926 learned_addr learned my addr 172.21.15.120:0/257194926 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f2353fff640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234c0e6640 0x7f234c16d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7fdbacd98640 1 -- 172.21.15.120:0/4227931140 shutdown_connections 2024-09-10T10:27:08.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7fdbacd98640 1 --2- 172.21.15.120:0/4227931140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 0x7fdba00e6800 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7fdbacd98640 1 --2- 172.21.15.120:0/4227931140 >> v2:172.21.15.134:3300/0 conn(0x7fdba00e5fb0 0x7fdba00501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7fdbacd98640 1 -- 172.21.15.120:0/4227931140 >> 172.21.15.120:0/4227931140 conn(0x7fdba0050ae0 msgr2=0x7fdba0050ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f2352ffd640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f234c0dcd90 0x7f234c049c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7ff6c37fe640 1 -- 172.21.15.120:0/4053353004 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc036070 con 0x7ff6c4158ea0 2024-09-10T10:27:08.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7ff6c37fe640 1 -- 172.21.15.120:0/4053353004 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6bc031600 con 0x7ff6c4158ea0 2024-09-10T10:27:08.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7ff6c37fe640 1 -- 172.21.15.120:0/4053353004 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0319d0 con 0x7ff6c4158ea0 2024-09-10T10:27:08.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7fdbacd98640 1 -- 172.21.15.120:0/4227931140 shutdown_connections 2024-09-10T10:27:08.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f658c111640 1 -- 172.21.15.120:0/1868161647 shutdown_connections 2024-09-10T10:27:08.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/863001356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a940039b0 msgr2=0x7f3a94003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f3ab0ea6640 1 --2- 172.21.15.120:0/863001356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a940039b0 0x7f3a94003dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000ba30 tx=0x7f3a94005830 comp rx=0 tx=0).stop 2024-09-10T10:27:08.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f658c111640 1 -- 172.21.15.120:0/1868161647 wait complete. 2024-09-10T10:27:08.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f9cd452c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc15eb30 con 0x7f9ccc0c52c0 2024-09-10T10:27:08.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f9cd452c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc15e850 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f9cd452c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ccc0b1290 con 0x7f9ccc0c8350 2024-09-10T10:27:08.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f9cd22c8640 1 -- 172.21.15.120:0/3201776588 learned_addr learned my addr 172.21.15.120:0/3201776588 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f23537fe640 1 -- 172.21.15.120:0/257194926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234c0e6640 msgr2=0x7f234c16d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f23537fe640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234c0e6640 0x7f234c16d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f23537fe640 1 -- 172.21.15.120:0/257194926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f234c0dcd90 msgr2=0x7f234c049c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f23537fe640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f234c0dcd90 0x7f234c049c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f23537fe640 1 -- 172.21.15.120:0/257194926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c17cb40 con 0x7f234c04a680 2024-09-10T10:27:08.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f2352ffd640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f234c0dcd90 0x7f234c049c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:08.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f3aaad9d640 1 -- 172.21.15.120:0/863001356 reap_dead start 2024-09-10T10:27:08.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f01f9bd6640 1 -- 172.21.15.120:0/1046333871 wait complete. 2024-09-10T10:27:08.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/863001356 shutdown_connections 2024-09-10T10:27:08.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/863001356 >> 172.21.15.120:0/863001356 conn(0x7f3aac06f3c0 msgr2=0x7f3aac06f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/863001356 shutdown_connections 2024-09-10T10:27:08.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/4053353004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6c4158ea0 msgr2=0x7ff6c40c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7ff6c9f8d640 1 --2- 172.21.15.120:0/4053353004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6c4158ea0 0x7ff6c40c7030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff6c406e750 tx=0x7ff6bc02a730 comp rx=0 tx=0).stop 2024-09-10T10:27:08.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7fc924933640 1 Processor -- start 2024-09-10T10:27:08.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f23537fe640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f234c04a680 0x7f234c17c600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f234000b450 tx=0x7f234000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f9cd22c8640 1 -- 172.21.15.120:0/3201776588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc0c8350 msgr2=0x7f9ccc0c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f9cd22c8640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc0c8350 0x7f9ccc0c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f9cd22c8640 1 -- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ccc0c52c0 msgr2=0x7f9ccc15a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/863001356 wait complete. 2024-09-10T10:27:08.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7f9cd2ac9640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ccc0c52c0 0x7f9ccc15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7f9cd22c8640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ccc0c52c0 0x7f9ccc15a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7f9cd22c8640 1 -- 172.21.15.120:0/3201776588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc15b6a0 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7f9cd2ac9640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ccc0c52c0 0x7f9ccc15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:08.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7f9cd1ac7640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc0c8350 0x7f9ccc0c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T10:27:08.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7f233ffff640 1 -- 172.21.15.120:0/257194926 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340015020 con 0x7f234c04a680 2024-09-10T10:27:08.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7f233ffff640 1 -- 172.21.15.120:0/257194926 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23400118b0 con 0x7f234c04a680 2024-09-10T10:27:08.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.598+0000 7fdbacd98640 1 -- 172.21.15.120:0/4227931140 wait complete. 2024-09-10T10:27:08.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7f658c111640 1 Processor -- start 2024-09-10T10:27:08.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7f658c111640 1 -- start start 2024-09-10T10:27:08.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/4053353004 shutdown_connections 2024-09-10T10:27:08.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7ff6c9f8d640 1 --2- 172.21.15.120:0/4053353004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 0x7ff6c415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7ff6c9f8d640 1 --2- 172.21.15.120:0/4053353004 >> v2:172.21.15.120:3300/0 conn(0x7ff6c4159280 0x7ff6c415ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7ff6c9f8d640 1 --2- 172.21.15.120:0/4053353004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6c4158ea0 0x7ff6c40c7030 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/4053353004 >> 172.21.15.120:0/4053353004 conn(0x7ff6c406c5f0 msgr2=0x7ff6c406f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:08.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7fdbacd98640 1 Processor -- start 2024-09-10T10:27:08.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.599+0000 7fc924933640 1 -- start start 2024-09-10T10:27:08.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7fc924933640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9200c8b10 0x7fc920161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7fc924933640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc920158f40 0x7fc9201569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7fc924933640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc920159320 0x7fc92015d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7f01f9bd6640 1 Processor -- start 2024-09-10T10:27:08.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/4053353004 shutdown_connections 2024-09-10T10:27:08.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7fc924933640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc920163310 con 0x7fc920159320 2024-09-10T10:27:08.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/4053353004 wait complete. 2024-09-10T10:27:08.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7fc924933640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc920162400 con 0x7fc9200c8b10 2024-09-10T10:27:08.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9200c8b10 0x7fc920161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7f3ab0ea6640 1 Processor -- start 2024-09-10T10:27:08.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7f233ffff640 1 -- 172.21.15.120:0/257194926 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340011ca0 con 0x7f234c04a680 2024-09-10T10:27:08.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7f3ab0ea6640 1 -- start start 2024-09-10T10:27:08.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9200c8b10 0x7fc920161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41858/0 (socket says 172.21.15.120:41858) 2024-09-10T10:27:08.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7f2358b62640 1 -- 172.21.15.120:0/257194926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c0e5280 con 0x7f234c04a680 2024-09-10T10:27:08.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7fc91e59c640 1 -- 172.21.15.120:0/2788553720 learned_addr learned my addr 172.21.15.120:0/2788553720 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7f01f9bd6640 1 -- start start 2024-09-10T10:27:08.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7fc91ed9d640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc920159320 0x7fc92015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc920072070 con 0x7fc920158f40 2024-09-10T10:27:08.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.600+0000 7fdbacd98640 1 -- start start 2024-09-10T10:27:08.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f658c111640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cb2f0 0x7f65840b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f658c111640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65840cc0c0 0x7f6584155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f658c111640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65840cc4a0 0x7f6584156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f9cd22c8640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ccc0c72f0 0x7f9ccc0c5ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800e990 tx=0x7f9cc800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fc91dd9b640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc920158f40 0x7fc9201569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fc91ed9d640 1 -- 172.21.15.120:0/2788553720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc920158f40 msgr2=0x7fc9201569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fc91ed9d640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc920158f40 0x7fc9201569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fc91ed9d640 1 -- 172.21.15.120:0/2788553720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9200c8b10 msgr2=0x7fc920161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f6589ead640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cb2f0 0x7f65840b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f6589ead640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cb2f0 0x7f65840b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41874/0 (socket says 172.21.15.120:41874) 2024-09-10T10:27:08.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f6589ead640 1 -- 172.21.15.120:0/3253340335 learned_addr learned my addr 172.21.15.120:0/3253340335 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fc91ed9d640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9200c8b10 0x7fc920161970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fc91ed9d640 1 -- 172.21.15.120:0/2788553720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc920160aa0 con 0x7fc920159320 2024-09-10T10:27:08.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f01f9bd6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01f40c8ff0 0x7f01f41f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f658415e1d0 con 0x7f65840cc4a0 2024-09-10T10:27:08.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f658415e540 con 0x7f65840cb2f0 2024-09-10T10:27:08.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f65840cc8b0 con 0x7f65840cc0c0 2024-09-10T10:27:08.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f01f9bd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f40cc480 0x7f01f41f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f01f9bd6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4159200 0x7f01f40b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f01f9bd6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01f415f620 con 0x7f01f40cc480 2024-09-10T10:27:08.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f01f9bd6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01f415f340 con 0x7f01f4159200 2024-09-10T10:27:08.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f01f9bd6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01f41595e0 con 0x7f01f40c8ff0 2024-09-10T10:27:08.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fc91dd9b640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc920158f40 0x7fc9201569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:08.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f65896ac640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65840cc0c0 0x7f6584155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f2358b62640 1 -- 172.21.15.120:0/257194926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c0e57e0 con 0x7f234c04a680 2024-09-10T10:27:08.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f658a6ae640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65840cc4a0 0x7f6584156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f9cc2ffd640 1 -- 172.21.15.120:0/3201776588 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8017070 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7ff6c9f8d640 1 Processor -- start 2024-09-10T10:27:08.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f01f37fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01f40c8ff0 0x7f01f41f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7ff6c9f8d640 1 -- start start 2024-09-10T10:27:08.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7f01f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4159200 0x7f01f40b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f9cd452c640 1 -- 172.21.15.120:0/3201776588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc0c3fb0 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f01f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4159200 0x7f01f40b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41892/0 (socket says 172.21.15.120:41892) 2024-09-10T10:27:08.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f01f3fff640 1 -- 172.21.15.120:0/2452548353 learned_addr learned my addr 172.21.15.120:0/2452548353 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.601+0000 7fdbacd98640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba004f0a0 0x7fdba00d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fdbacd98640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdba00e5fb0 0x7fdba01848c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fdbacd98640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 0x7fdba0182700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fdbacd98640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdba004a4f0 con 0x7fdba00e5fb0 2024-09-10T10:27:08.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fdbacd98640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0050930 con 0x7fdba004f0a0 2024-09-10T10:27:08.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fdbacd98640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdba004eb80 con 0x7fdba00e6390 2024-09-10T10:27:08.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3ab0ea6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac13a3b0 0x7f3aac15df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fdba659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba004f0a0 0x7fdba00d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fdba5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdba00e5fb0 0x7fdba01848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fdba5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdba00e5fb0 0x7fdba01848c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34450/0 (socket says 172.21.15.120:34450) 2024-09-10T10:27:08.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fdba5d9b640 1 -- 172.21.15.120:0/506146538 learned_addr learned my addr 172.21.15.120:0/506146538 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7fdba6d9d640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 0x7fdba0182700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3ab0ea6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac1557c0 0x7f3aac15e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3ab0ea6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aac0cb560 0x7f3aac0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.602+0000 7fc91ed9d640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc920159320 0x7fc92015d670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc91800c7b0 tx=0x7fc91800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f9cd452c640 1 -- 172.21.15.120:0/3201776588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc0c4570 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3ab0ea6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac072070 con 0x7f3aac0cb560 2024-09-10T10:27:08.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3ab0ea6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac162400 con 0x7f3aac13a3b0 2024-09-10T10:27:08.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f01f2ffd640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f40cc480 0x7f01f41f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3ab0ea6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3aac071930 con 0x7f3aac1557c0 2024-09-10T10:27:08.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f6589ead640 1 -- 172.21.15.120:0/3253340335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65840cc0c0 msgr2=0x7f6584155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f6589ead640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65840cc0c0 0x7f6584155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f6589ead640 1 -- 172.21.15.120:0/3253340335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65840cc4a0 msgr2=0x7f6584156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f6589ead640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65840cc4a0 0x7f6584156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f6589ead640 1 -- 172.21.15.120:0/3253340335 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65841f42d0 con 0x7f65840cb2f0 2024-09-10T10:27:08.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f6589ead640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cb2f0 0x7f65840b4b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f657400c7f0 tx=0x7f657400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3aaa59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac13a3b0 0x7f3aac15df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3aa9d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac1557c0 0x7f3aac15e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f01f2ffd640 1 -- 172.21.15.120:0/2452548353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01f40c8ff0 msgr2=0x7f01f41f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f01f2ffd640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01f40c8ff0 0x7f01f41f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f01f2ffd640 1 -- 172.21.15.120:0/2452548353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4159200 msgr2=0x7f01f40b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f01f2ffd640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4159200 0x7f01f40b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7fdba5d9b640 1 -- 172.21.15.120:0/506146538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 msgr2=0x7fdba0182700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7fdba5d9b640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 0x7fdba0182700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7fdba5d9b640 1 -- 172.21.15.120:0/506146538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba004f0a0 msgr2=0x7fdba00d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7fdba5d9b640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba004f0a0 0x7fdba00d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7fdba5d9b640 1 -- 172.21.15.120:0/506146538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba00d3560 con 0x7fdba00e5fb0 2024-09-10T10:27:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f01f2ffd640 1 -- 172.21.15.120:0/2452548353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f415dd70 con 0x7f01f40cc480 2024-09-10T10:27:08.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3aa9d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac1557c0 0x7f3aac15e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:42900/0 (socket says 172.21.15.120:42900) 2024-09-10T10:27:08.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3aaa59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac13a3b0 0x7f3aac15df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41904/0 (socket says 172.21.15.120:41904) 2024-09-10T10:27:08.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f657a7fc640 1 -- 172.21.15.120:0/3253340335 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574015070 con 0x7f65840cb2f0 2024-09-10T10:27:08.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f657a7fc640 1 -- 172.21.15.120:0/3253340335 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6574004220 con 0x7f65840cb2f0 2024-09-10T10:27:08.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f657a7fc640 1 -- 172.21.15.120:0/3253340335 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574007620 con 0x7f65840cb2f0 2024-09-10T10:27:08.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f3aaa59c640 1 -- 172.21.15.120:0/4079199418 learned_addr learned my addr 172.21.15.120:0/4079199418 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65841550d0 con 0x7f65840cb2f0 2024-09-10T10:27:08.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65840d0390 con 0x7f65840cb2f0 2024-09-10T10:27:08.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7f3aaad9d640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aac0cb560 0x7f3aac0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7ff6c9f8d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 0x7ff6c415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7ff6c9f8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6c4158ea0 0x7ff6c415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7f2358b62640 1 -- 172.21.15.120:0/257194926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234c0e2be0 con 0x7f234c04a680 2024-09-10T10:27:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7f3aa9d9b640 1 -- 172.21.15.120:0/4079199418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac13a3b0 msgr2=0x7f3aac15df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7f3aa9d9b640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac13a3b0 0x7f3aac15df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f9cc2ffd640 1 -- 172.21.15.120:0/3201776588 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc800c9e0 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.603+0000 7f9cc2ffd640 1 -- 172.21.15.120:0/3201776588 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc800cdd0 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7fc90effd640 1 -- 172.21.15.120:0/2788553720 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918017070 con 0x7fc920159320 2024-09-10T10:27:08.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7ff6c9f8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6c415af20 0x7ff6c41f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7ff6c9f8d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c40c7570 con 0x7ff6c415af20 2024-09-10T10:27:08.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7ff6c9f8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4072a70 con 0x7ff6c4158ea0 2024-09-10T10:27:08.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7ff6c9f8d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c4162270 con 0x7ff6c4073270 2024-09-10T10:27:08.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7ff6c8f8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 0x7ff6c415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7ff6c8f8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 0x7ff6c415b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:42906/0 (socket says 172.21.15.120:42906) 2024-09-10T10:27:08.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7ff6c8f8b640 1 -- 172.21.15.120:0/723611132 learned_addr learned my addr 172.21.15.120:0/723611132 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:08.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7ff6c978c640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6c415af20 0x7ff6c41f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7f3aa9d9b640 1 -- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aac0cb560 msgr2=0x7f3aac0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7f9cd452c640 1 -- 172.21.15.120:0/3201776588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ccc0c7e00 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7ff6c3fff640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6c4158ea0 0x7ff6c415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7fdba5d9b640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdba00e5fb0 0x7fdba01848c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c001eb0 tx=0x7fdb9c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7fc90effd640 1 -- 172.21.15.120:0/2788553720 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc918003da0 con 0x7fc920159320 2024-09-10T10:27:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9201f8220 con 0x7fc920159320 2024-09-10T10:27:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7ff6c8f8b640 1 -- 172.21.15.120:0/723611132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6c4158ea0 msgr2=0x7ff6c415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7ff6c8f8b640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6c4158ea0 0x7ff6c415be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7ff6c8f8b640 1 -- 172.21.15.120:0/723611132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6c415af20 msgr2=0x7ff6c41f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7ff6c8f8b640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6c415af20 0x7ff6c41f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7ff6c8f8b640 1 -- 172.21.15.120:0/723611132 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c4159710 con 0x7ff6c4073270 2024-09-10T10:27:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7fc90effd640 1 -- 172.21.15.120:0/2788553720 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918005270 con 0x7fc920159320 2024-09-10T10:27:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9201f87e0 con 0x7fc920159320 2024-09-10T10:27:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7f3aa9d9b640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aac0cb560 0x7f3aac0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7f3aa9d9b640 1 -- 172.21.15.120:0/4079199418 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac155330 con 0x7f3aac1557c0 2024-09-10T10:27:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7f3aaad9d640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aac0cb560 0x7f3aac0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7ff6c8f8b640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 0x7ff6c415b950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc02ad20 tx=0x7ff6bc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.605+0000 7f01f2ffd640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f40cc480 0x7f01f41f4960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f01f40c93f0 tx=0x7f01e800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65840c5560 con 0x7f65840cb2f0 2024-09-10T10:27:08.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7ff6c17fa640 1 -- 172.21.15.120:0/723611132 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc041020 con 0x7ff6c4073270 2024-09-10T10:27:08.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7f9cc2ffd640 1 -- 172.21.15.120:0/3201776588 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9cc801a070 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/723611132 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c415e8c0 con 0x7ff6c4073270 2024-09-10T10:27:08.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7f657a7fc640 1 -- 172.21.15.120:0/3253340335 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65740077c0 con 0x7f65840cb2f0 2024-09-10T10:27:08.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/723611132 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c415eba0 con 0x7ff6c4073270 2024-09-10T10:27:08.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7ff6c17fa640 1 -- 172.21.15.120:0/723611132 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6bc0087b0 con 0x7ff6c4073270 2024-09-10T10:27:08.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7f657a7fc640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f656806bf00 0x7f656806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc920073110 con 0x7fc920159320 2024-09-10T10:27:08.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7ff6c17fa640 1 -- 172.21.15.120:0/723611132 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc036040 con 0x7ff6c4073270 2024-09-10T10:27:08.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7f657a7fc640 1 -- 172.21.15.120:0/3253340335 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f657408b030 con 0x7f65840cb2f0 2024-09-10T10:27:08.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7f65896ac640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f656806bf00 0x7f656806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.607+0000 7f9cc2ffd640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9cb006bf00 0x7f9cb006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.604+0000 7f233ffff640 1 -- 172.21.15.120:0/257194926 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2340018020 con 0x7f234c04a680 2024-09-10T10:27:08.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7f233ffff640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233806bf00 0x7f233806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.606+0000 7f233ffff640 1 -- 172.21.15.120:0/257194926 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f234008b140 con 0x7f234c04a680 2024-09-10T10:27:08.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.607+0000 7f2352ffd640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233806bf00 0x7f233806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.607+0000 7f65896ac640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f656806bf00 0x7f656806e3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f65840b6550 tx=0x7f657c0094d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.607+0000 7f9cd1ac7640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9cb006bf00 0x7f9cb006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.607+0000 7f9cc2ffd640 1 -- 172.21.15.120:0/3201776588 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9cc808b5e0 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.607+0000 7f3aa9d9b640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac1557c0 0x7f3aac15e440 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00e7f0 tx=0x7f3a9c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.607+0000 7f2352ffd640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233806bf00 0x7f233806e3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f234c0051e0 tx=0x7f2348009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.607+0000 7f3a9affd640 1 -- 172.21.15.120:0/4079199418 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c017070 con 0x7f3aac1557c0 2024-09-10T10:27:08.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.607+0000 7f9cd1ac7640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9cb006bf00 0x7f9cb006e3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc034000 tx=0x7f9cbc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.608+0000 7f3a9affd640 1 -- 172.21.15.120:0/4079199418 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9c00c940 con 0x7f3aac1557c0 2024-09-10T10:27:08.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.608+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/4079199418 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aac0cbf20 con 0x7f3aac1557c0 2024-09-10T10:27:08.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.608+0000 7f3a9affd640 1 -- 172.21.15.120:0/4079199418 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00cd10 con 0x7f3aac1557c0 2024-09-10T10:27:08.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.608+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/4079199418 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aac0cad10 con 0x7f3aac1557c0 2024-09-10T10:27:08.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.608+0000 7ff6c17fa640 1 -- 172.21.15.120:0/723611132 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6bc044020 con 0x7ff6c4073270 2024-09-10T10:27:08.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.609+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/723611132 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a4005290 con 0x7ff6c4073270 2024-09-10T10:27:08.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.609+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/4079199418 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aac1630b0 con 0x7f3aac1557c0 2024-09-10T10:27:08.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.609+0000 7ff6c17fa640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff6b006bf00 0x7ff6b006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.609+0000 7ff6c17fa640 1 -- 172.21.15.120:0/723611132 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff6bc0ac000 con 0x7ff6c4073270 2024-09-10T10:27:08.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.609+0000 7ff6c3fff640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff6b006bf00 0x7ff6b006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.609+0000 7ff6c3fff640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff6b006bf00 0x7ff6b006e3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40c69f0 tx=0x7ff6b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.609+0000 7f3a9affd640 1 -- 172.21.15.120:0/4079199418 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a9c01a070 con 0x7f3aac1557c0 2024-09-10T10:27:08.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.610+0000 7f3a9affd640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3a8c06be60 0x7f3a8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.610+0000 7f3a9affd640 1 -- 172.21.15.120:0/4079199418 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3a9c08b1f0 con 0x7f3aac1557c0 2024-09-10T10:27:08.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.610+0000 7f3aaa59c640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3a8c06be60 0x7f3a8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.610+0000 7f3aaa59c640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3a8c06be60 0x7f3a8c06e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000ba30 tx=0x7f3aa0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.612+0000 7f233ffff640 1 -- 172.21.15.120:0/257194926 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f234001f020 con 0x7f234c04a680 2024-09-10T10:27:08.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.612+0000 7fdb96ffd640 1 -- 172.21.15.120:0/506146538 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c017070 con 0x7fdba00e5fb0 2024-09-10T10:27:08.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.612+0000 7f01dbfff640 1 -- 172.21.15.120:0/2452548353 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8017070 con 0x7f01f40cc480 2024-09-10T10:27:08.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.612+0000 7f01f9bd6640 1 -- 172.21.15.120:0/2452548353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f415ee00 con 0x7f01f40cc480 2024-09-10T10:27:08.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.612+0000 7fdbacd98640 1 -- 172.21.15.120:0/506146538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba0185330 con 0x7fdba00e5fb0 2024-09-10T10:27:08.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.612+0000 7f01f9bd6640 1 -- 172.21.15.120:0/2452548353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f415e010 con 0x7f01f40cc480 2024-09-10T10:27:08.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.612+0000 7fdbacd98640 1 -- 172.21.15.120:0/506146538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba00d5090 con 0x7fdba00e5fb0 2024-09-10T10:27:08.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.612+0000 7f01dbfff640 1 -- 172.21.15.120:0/2452548353 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e80052d0 con 0x7f01f40cc480 2024-09-10T10:27:08.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.612+0000 7f01dbfff640 1 -- 172.21.15.120:0/2452548353 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8021060 con 0x7f01f40cc480 2024-09-10T10:27:08.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.613+0000 7fdb96ffd640 1 -- 172.21.15.120:0/506146538 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9c005da0 con 0x7fdba00e5fb0 2024-09-10T10:27:08.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.613+0000 7fdbacd98640 1 -- 172.21.15.120:0/506146538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba0053890 con 0x7fdba00e5fb0 2024-09-10T10:27:08.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.613+0000 7f01f9bd6640 1 -- 172.21.15.120:0/2452548353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d4005290 con 0x7f01f40cc480 2024-09-10T10:27:08.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.613+0000 7fdb96ffd640 1 -- 172.21.15.120:0/506146538 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c007270 con 0x7fdba00e5fb0 2024-09-10T10:27:08.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.614+0000 7fc90effd640 1 -- 172.21.15.120:0/2788553720 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc91801a070 con 0x7fc920159320 2024-09-10T10:27:08.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.614+0000 7fdb96ffd640 1 -- 172.21.15.120:0/506146538 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb9c01a070 con 0x7fdba00e5fb0 2024-09-10T10:27:08.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.614+0000 7f9cc2ffd640 1 -- 172.21.15.120:0/3201776588 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9cc8059c30 con 0x7f9ccc0c72f0 2024-09-10T10:27:08.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.615+0000 7fc90effd640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc90006be60 0x7fc90006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.615+0000 7fc90effd640 1 -- 172.21.15.120:0/2788553720 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc91808ab60 con 0x7fc920159320 2024-09-10T10:27:08.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.615+0000 7f01dbfff640 1 -- 172.21.15.120:0/2452548353 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01e801a070 con 0x7f01f40cc480 2024-09-10T10:27:08.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.615+0000 7fc91e59c640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc90006be60 0x7fc90006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.615+0000 7f01dbfff640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01e006be30 0x7f01e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.615+0000 7fc91e59c640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc90006be60 0x7fc90006e320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc91002a4d0 tx=0x7fc910008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.616+0000 7f01f37fe640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01e006be30 0x7f01e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.616+0000 7fdb96ffd640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb8806be30 0x7fdb8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:08.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.616+0000 7fdba659c640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb8806be30 0x7fdb8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:08.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.616+0000 7f01f37fe640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01e006be30 0x7f01e006e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01e4034000 tx=0x7f01e402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.616+0000 7fdba659c640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb8806be30 0x7fdb8806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009b40 tx=0x7fdb98009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:08.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.620+0000 7ff6c17fa640 1 -- 172.21.15.120:0/723611132 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff6bc04b020 con 0x7ff6c4073270 2024-09-10T10:27:08.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.623+0000 7fdb96ffd640 1 -- 172.21.15.120:0/506146538 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdb9c056530 con 0x7fdba00e5fb0 2024-09-10T10:27:08.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.623+0000 7fc90effd640 1 -- 172.21.15.120:0/2788553720 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc920073110 con 0x7fc920159320 2024-09-10T10:27:08.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.623+0000 7f01dbfff640 1 -- 172.21.15.120:0/2452548353 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f01e8056d80 con 0x7f01f40cc480 2024-09-10T10:27:08.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.624+0000 7f657a7fc640 1 -- 172.21.15.120:0/3253340335 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6574058fa0 con 0x7f65840cb2f0 2024-09-10T10:27:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.628+0000 7f3a9affd640 1 -- 172.21.15.120:0/4079199418 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3aac0cbf20 con 0x7f3aac1557c0 2024-09-10T10:27:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.632+0000 7fdb96ffd640 1 -- 172.21.15.120:0/506146538 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdba0053890 con 0x7fdba00e5fb0 2024-09-10T10:27:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.641+0000 7f01dbfff640 1 -- 172.21.15.120:0/2452548353 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f01e8058ce0 con 0x7f01f40cc480 2024-09-10T10:27:08.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.995+0000 7f2358b62640 1 -- 172.21.15.120:0/257194926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f234c04b1c0 con 0x7f234c04a680 2024-09-10T10:27:08.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:08.996+0000 7f233ffff640 1 -- 172.21.15.120:0/257194926 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f234000d040 con 0x7f234c04a680 2024-09-10T10:27:08.997 INFO:teuthology.orchestra.run.smithi120.stdout:55834574856 2024-09-10T10:27:09.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.000+0000 7f233dffb640 1 -- 172.21.15.120:0/257194926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233806bf00 msgr2=0x7f233806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.000+0000 7f233dffb640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233806bf00 0x7f233806e3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f234c0051e0 tx=0x7f2348009040 comp rx=0 tx=0).stop 2024-09-10T10:27:09.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.000+0000 7f233dffb640 1 -- 172.21.15.120:0/257194926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f234c04a680 msgr2=0x7f234c17c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.000+0000 7f233dffb640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f234c04a680 0x7f234c17c600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f234000b450 tx=0x7f234000b910 comp rx=0 tx=0).stop 2024-09-10T10:27:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.000+0000 7f233dffb640 1 -- 172.21.15.120:0/257194926 shutdown_connections 2024-09-10T10:27:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.001+0000 7f233dffb640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233806bf00 0x7f233806e3c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.001+0000 7f233dffb640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234c0e6640 0x7f234c16d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.001+0000 7f233dffb640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f234c0dcd90 0x7f234c049c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.001+0000 7f233dffb640 1 --2- 172.21.15.120:0/257194926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f234c04a680 0x7f234c17c600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.001+0000 7f233dffb640 1 -- 172.21.15.120:0/257194926 >> 172.21.15.120:0/257194926 conn(0x7f234c04fc50 msgr2=0x7f234c050250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:09.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.001+0000 7f233dffb640 1 -- 172.21.15.120:0/257194926 shutdown_connections 2024-09-10T10:27:09.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.001+0000 7f233dffb640 1 -- 172.21.15.120:0/257194926 wait complete. 2024-09-10T10:27:09.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-09-10T10:27:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.018+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/723611132 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff6a4005580 con 0x7ff6c4073270 2024-09-10T10:27:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.019+0000 7ff6c17fa640 1 -- 172.21.15.120:0/723611132 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff6bc03ae20 con 0x7ff6c4073270 2024-09-10T10:27:09.021 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:09.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.022+0000 7f9cd452c640 1 -- 172.21.15.120:0/3201776588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9ccc0c6ea0 con 0x7f9ccc0c72f0 2024-09-10T10:27:09.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.022+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/4079199418 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3aac0c9b60 con 0x7f3aac1557c0 2024-09-10T10:27:09.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/723611132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff6b006bf00 msgr2=0x7ff6b006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff6b006bf00 0x7ff6b006e3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40c69f0 tx=0x7ff6b4008040 comp rx=0 tx=0).stop 2024-09-10T10:27:09.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/723611132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 msgr2=0x7ff6c415b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 0x7ff6c415b950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc02ad20 tx=0x7ff6bc004a40 comp rx=0 tx=0).stop 2024-09-10T10:27:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/723611132 shutdown_connections 2024-09-10T10:27:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff6b006bf00 0x7ff6b006e3c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6c415af20 0x7ff6c41f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6c4158ea0 0x7ff6c415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 --2- 172.21.15.120:0/723611132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6c4073270 0x7ff6c415b950 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/723611132 >> 172.21.15.120:0/723611132 conn(0x7ff6c406c5f0 msgr2=0x7ff6c4154970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:09.025 INFO:teuthology.orchestra.run.smithi120.stdout:55834574856 2024-09-10T10:27:09.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7f9cc2ffd640 1 -- 172.21.15.120:0/3201776588 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9cc8059530 con 0x7f9ccc0c72f0 2024-09-10T10:27:09.026 INFO:teuthology.orchestra.run.smithi120.stdout:55834574856 2024-09-10T10:27:09.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7f3a9affd640 1 -- 172.21.15.120:0/4079199418 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a9c059210 con 0x7f3aac1557c0 2024-09-10T10:27:09.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/723611132 shutdown_connections 2024-09-10T10:27:09.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.023+0000 7ff6c9f8d640 1 -- 172.21.15.120:0/723611132 wait complete. 2024-09-10T10:27:09.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3a8c06be60 msgr2=0x7f3a8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3a8c06be60 0x7f3a8c06e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000ba30 tx=0x7f3aa0030000 comp rx=0 tx=0).stop 2024-09-10T10:27:09.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/4079199418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac1557c0 msgr2=0x7f3aac15e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac1557c0 0x7f3aac15e440 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00e7f0 tx=0x7f3a9c00ecb0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/4079199418 shutdown_connections 2024-09-10T10:27:09.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3a8c06be60 0x7f3a8c06e320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aac0cb560 0x7f3aac0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3aac1557c0 0x7f3aac15e440 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00e7f0 tx=0x7f3a9c00ecb0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 --2- 172.21.15.120:0/4079199418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac13a3b0 0x7f3aac15df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/4079199418 >> 172.21.15.120:0/4079199418 conn(0x7f3aac06f3c0 msgr2=0x7f3aac156760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:09.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/4079199418 shutdown_connections 2024-09-10T10:27:09.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f3ab0ea6640 1 -- 172.21.15.120:0/4079199418 wait complete. 2024-09-10T10:27:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f9cd452c640 1 -- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9cb006bf00 msgr2=0x7f9cb006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f9cd452c640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9cb006bf00 0x7f9cb006e3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc034000 tx=0x7f9cbc02e000 comp rx=0 tx=0).stop 2024-09-10T10:27:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f9cd452c640 1 -- 172.21.15.120:0/3201776588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ccc0c72f0 msgr2=0x7f9ccc0c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f9cd452c640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ccc0c72f0 0x7f9ccc0c5ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800e990 tx=0x7f9cc800ee50 comp rx=0 tx=0).stop 2024-09-10T10:27:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f9cd452c640 1 -- 172.21.15.120:0/3201776588 shutdown_connections 2024-09-10T10:27:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f9cd452c640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9cb006bf00 0x7f9cb006e3c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f9cd452c640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ccc0c52c0 0x7f9ccc15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f9cd452c640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ccc0c8350 0x7f9ccc0c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f9cd452c640 1 --2- 172.21.15.120:0/3201776588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ccc0c72f0 0x7f9ccc0c5ef0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.026+0000 7f9cd452c640 1 -- 172.21.15.120:0/3201776588 >> 172.21.15.120:0/3201776588 conn(0x7f9ccc0cbf90 msgr2=0x7f9ccc0c20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.027+0000 7f9cd452c640 1 -- 172.21.15.120:0/3201776588 shutdown_connections 2024-09-10T10:27:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.027+0000 7f9cd452c640 1 -- 172.21.15.120:0/3201776588 wait complete. 2024-09-10T10:27:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.031+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6584149bb0 con 0x7f65840cb2f0 2024-09-10T10:27:09.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.032+0000 7f657a7fc640 1 -- 172.21.15.120:0/3253340335 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f657405d2a0 con 0x7f65840cb2f0 2024-09-10T10:27:09.033 INFO:teuthology.orchestra.run.smithi120.stdout:55834574856 2024-09-10T10:27:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.033+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc920071930 con 0x7fc920159320 2024-09-10T10:27:09.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.034+0000 7fc90effd640 1 -- 172.21.15.120:0/2788553720 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc918058b60 con 0x7fc920159320 2024-09-10T10:27:09.035 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:09.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f656806bf00 msgr2=0x7f656806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f656806bf00 0x7f656806e3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f65840b6550 tx=0x7f657c0094d0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cb2f0 msgr2=0x7f65840b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cb2f0 0x7f65840b4b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f657400c7f0 tx=0x7f657400ccb0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 shutdown_connections 2024-09-10T10:27:09.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f656806bf00 0x7f656806e3c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65840cc4a0 0x7f6584156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65840cc0c0 0x7f6584155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 --2- 172.21.15.120:0/3253340335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65840cb2f0 0x7f65840b4b70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 >> 172.21.15.120:0/3253340335 conn(0x7f658406d560 msgr2=0x7f6584153a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:09.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 shutdown_connections 2024-09-10T10:27:09.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7f658c111640 1 -- 172.21.15.120:0/3253340335 wait complete. 2024-09-10T10:27:09.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc90006be60 msgr2=0x7fc90006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7fc924933640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc90006be60 0x7fc90006e320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc91002a4d0 tx=0x7fc910008030 comp rx=0 tx=0).stop 2024-09-10T10:27:09.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc920159320 msgr2=0x7fc92015d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.036+0000 7fc924933640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc920159320 0x7fc92015d670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc91800c7b0 tx=0x7fc91800cc70 comp rx=0 tx=0).stop 2024-09-10T10:27:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.037+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 shutdown_connections 2024-09-10T10:27:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.037+0000 7fc924933640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc90006be60 0x7fc90006e320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.037+0000 7fc924933640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc920159320 0x7fc92015d670 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.037+0000 7fc924933640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc920158f40 0x7fc9201569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.037+0000 7fc924933640 1 --2- 172.21.15.120:0/2788553720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9200c8b10 0x7fc920161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.037+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 >> 172.21.15.120:0/2788553720 conn(0x7fc92006f3c0 msgr2=0x7fc92006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.037+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 shutdown_connections 2024-09-10T10:27:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.037+0000 7fc924933640 1 -- 172.21.15.120:0/2788553720 wait complete. 2024-09-10T10:27:09.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-09-10T10:27:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.038+0000 7fdbacd98640 1 -- 172.21.15.120:0/506146538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdba0185500 con 0x7fdba00e5fb0 2024-09-10T10:27:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.040+0000 7fdb96ffd640 1 -- 172.21.15.120:0/506146538 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb9c058490 con 0x7fdba00e5fb0 2024-09-10T10:27:09.042 INFO:teuthology.orchestra.run.smithi120.stdout:55834574856 2024-09-10T10:27:09.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.043+0000 7fdbacd98640 1 -- 172.21.15.120:0/506146538 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb8806be30 msgr2=0x7fdb8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.043+0000 7fdbacd98640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb8806be30 0x7fdb8806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009b40 tx=0x7fdb98009190 comp rx=0 tx=0).stop 2024-09-10T10:27:09.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.043+0000 7fdbacd98640 1 -- 172.21.15.120:0/506146538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdba00e5fb0 msgr2=0x7fdba01848c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.043+0000 7fdbacd98640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdba00e5fb0 0x7fdba01848c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c001eb0 tx=0x7fdb9c00c960 comp rx=0 tx=0).stop 2024-09-10T10:27:09.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.044+0000 7fdbacd98640 1 -- 172.21.15.120:0/506146538 shutdown_connections 2024-09-10T10:27:09.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.044+0000 7fdbacd98640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb8806be30 0x7fdb8806e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.044+0000 7fdbacd98640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdba00e6390 0x7fdba0182700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.044+0000 7fdbacd98640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdba00e5fb0 0x7fdba01848c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.044+0000 7fdbacd98640 1 --2- 172.21.15.120:0/506146538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba004f0a0 0x7fdba00d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.044+0000 7fdbacd98640 1 -- 172.21.15.120:0/506146538 >> 172.21.15.120:0/506146538 conn(0x7fdba0050ae0 msgr2=0x7fdba0037b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:09.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.044+0000 7fdbacd98640 1 -- 172.21.15.120:0/506146538 shutdown_connections 2024-09-10T10:27:09.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.045+0000 7fdbacd98640 1 -- 172.21.15.120:0/506146538 wait complete. 2024-09-10T10:27:09.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-09-10T10:27:09.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2024-09-10T10:27:09.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.049+0000 7f01f9bd6640 1 -- 172.21.15.120:0/2452548353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f01d4005850 con 0x7f01f40cc480 2024-09-10T10:27:09.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.049+0000 7f01dbfff640 1 -- 172.21.15.120:0/2452548353 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f01e805cfe0 con 0x7f01f40cc480 2024-09-10T10:27:09.051 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:09.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-09-10T10:27:09.054 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 -- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01e006be30 msgr2=0x7f01e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01e006be30 0x7f01e006e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01e4034000 tx=0x7f01e402e000 comp rx=0 tx=0).stop 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 -- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f40cc480 msgr2=0x7f01f41f4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f40cc480 0x7f01f41f4960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f01f40c93f0 tx=0x7f01e800c820 comp rx=0 tx=0).stop 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 -- 172.21.15.120:0/2452548353 shutdown_connections 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01e006be30 0x7f01e006e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f4159200 0x7f01f40b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f40cc480 0x7f01f41f4960 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 --2- 172.21.15.120:0/2452548353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01f40c8ff0 0x7f01f41f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 -- 172.21.15.120:0/2452548353 >> 172.21.15.120:0/2452548353 conn(0x7f01f406d7a0 msgr2=0x7f01f40cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 -- 172.21.15.120:0/2452548353 shutdown_connections 2024-09-10T10:27:09.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:09.053+0000 7f01f9bd6640 1 -- 172.21.15.120:0/2452548353 wait complete. 2024-09-10T10:27:09.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-09-10T10:27:09.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-09-10T10:27:10.019 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T10:27:10.042 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T10:27:10.048 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T10:27:10.049 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T10:27:10.056 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T10:27:10.057 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T10:27:10.063 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T10:27:10.069 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T10:27:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.169+0000 7f9ac128f640 1 Processor -- start 2024-09-10T10:27:10.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.170+0000 7f9ac128f640 1 -- start start 2024-09-10T10:27:10.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.171+0000 7f9ac128f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9ab4005a00 0x7f9ab40da540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.171+0000 7f9ac128f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9ab404ae70 0x7f9ab404b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.171+0000 7f9ac128f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab40d49d0 con 0x7f9ab40d45f0 2024-09-10T10:27:10.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.171+0000 7f9ac128f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab4032c10 con 0x7f9ab4005a00 2024-09-10T10:27:10.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.171+0000 7f9ac128f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab4005fd0 con 0x7f9ab404ae70 2024-09-10T10:27:10.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.172+0000 7f9abb7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9ab4005a00 0x7f9ab40da540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.172+0000 7f9abb7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9ab4005a00 0x7f9ab40da540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34480/0 (socket says 172.21.15.120:34480) 2024-09-10T10:27:10.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.172+0000 7f9abb7fe640 1 -- 172.21.15.120:0/1270075943 learned_addr learned my addr 172.21.15.120:0/1270075943 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.173+0000 7f9abb7fe640 1 -- 172.21.15.120:0/1270075943 >> v2:172.21.15.134:3300/0 conn(0x7f9ab404ae70 msgr2=0x7f9ab404b280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:27:10.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.173+0000 7f9abb7fe640 1 --2- 172.21.15.120:0/1270075943 >> v2:172.21.15.134:3300/0 conn(0x7f9ab404ae70 0x7f9ab404b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.173+0000 7f9abb7fe640 1 -- 172.21.15.120:0/1270075943 >> v1:172.21.15.120:6789/0 conn(0x7f9ab40d45f0 legacy=0x7f9ab4034250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:27:10.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.173+0000 7f9abb7fe640 1 -- 172.21.15.120:0/1270075943 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab40062b0 con 0x7f9ab4005a00 2024-09-10T10:27:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.174+0000 7f9abb7fe640 1 --2- 172.21.15.120:0/1270075943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ab4005a00 0x7f9ab40da540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ab404ab90 tx=0x7f9ab002a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219f658fc90c6fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.175+0000 7f9abaffd640 1 -- 172.21.15.120:0/1270075943 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0036070 con 0x7f9ab4005a00 2024-09-10T10:27:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.175+0000 7f9abaffd640 1 -- 172.21.15.120:0/1270075943 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0031600 con 0x7f9ab4005a00 2024-09-10T10:27:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.176+0000 7f9ac128f640 1 -- 172.21.15.120:0/1270075943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ab4005a00 msgr2=0x7f9ab40da540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.176+0000 7f9ac128f640 1 --2- 172.21.15.120:0/1270075943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ab4005a00 0x7f9ab40da540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ab404ab90 tx=0x7f9ab002a540 comp rx=0 tx=0).stop 2024-09-10T10:27:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.177+0000 7f9ac128f640 1 -- 172.21.15.120:0/1270075943 shutdown_connections 2024-09-10T10:27:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.177+0000 7f9ac128f640 1 --2- 172.21.15.120:0/1270075943 >> v2:172.21.15.134:3300/0 conn(0x7f9ab404ae70 0x7f9ab404b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.177+0000 7f9ac128f640 1 --2- 172.21.15.120:0/1270075943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ab4005a00 0x7f9ab40da540 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.177+0000 7f9ac128f640 1 -- 172.21.15.120:0/1270075943 >> 172.21.15.120:0/1270075943 conn(0x7f9ab404fc50 msgr2=0x7f9ab4050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.177+0000 7f9ac128f640 1 -- 172.21.15.120:0/1270075943 shutdown_connections 2024-09-10T10:27:10.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.177+0000 7f9ac128f640 1 -- 172.21.15.120:0/1270075943 wait complete. 2024-09-10T10:27:10.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.179+0000 7f9ac128f640 1 Processor -- start 2024-09-10T10:27:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.180+0000 7f9ac128f640 1 -- start start 2024-09-10T10:27:10.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.181+0000 7f9ac128f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ab4005a00 0x7f9ab4049800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.181+0000 7f9ac128f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ab404ae70 0x7f9ab40471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.181+0000 7f9ac128f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ab40d45f0 0x7f9ab40ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.181+0000 7f9ac128f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4006b50 con 0x7f9ab40d45f0 2024-09-10T10:27:10.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.181+0000 7f9ac128f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4006870 con 0x7f9ab404ae70 2024-09-10T10:27:10.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.181+0000 7f9ac128f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab4006e30 con 0x7f9ab4005a00 2024-09-10T10:27:10.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.182+0000 7f9abb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ab404ae70 0x7f9ab40471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.182+0000 7f9abb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ab404ae70 0x7f9ab40471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41924/0 (socket says 172.21.15.120:41924) 2024-09-10T10:27:10.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.182+0000 7f9abb7fe640 1 -- 172.21.15.120:0/3937278999 learned_addr learned my addr 172.21.15.120:0/3937278999 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.182+0000 7f9ac0a8e640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ab40d45f0 0x7f9ab40ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.182+0000 7f9abbfff640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ab4005a00 0x7f9ab4049800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.182+0000 7f9abb7fe640 1 -- 172.21.15.120:0/3937278999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ab4005a00 msgr2=0x7f9ab4049800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.182+0000 7f9abb7fe640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ab4005a00 0x7f9ab4049800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.183+0000 7f9abb7fe640 1 -- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ab40d45f0 msgr2=0x7f9ab40ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.183+0000 7f9abb7fe640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ab40d45f0 0x7f9ab40ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.183+0000 7f9abb7fe640 1 -- 172.21.15.120:0/3937278999 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab4047cb0 con 0x7f9ab404ae70 2024-09-10T10:27:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.183+0000 7f9abbfff640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ab4005a00 0x7f9ab4049800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.183+0000 7f9ac0a8e640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ab40d45f0 0x7f9ab40ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.183+0000 7f9abb7fe640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ab404ae70 0x7f9ab40471c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009bc0 tx=0x7f9ab00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.184+0000 7f9ab8ff9640 1 -- 172.21.15.120:0/3937278999 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0041020 con 0x7f9ab404ae70 2024-09-10T10:27:10.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.184+0000 7f9ac128f640 1 -- 172.21.15.120:0/3937278999 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab40487a0 con 0x7f9ab404ae70 2024-09-10T10:27:10.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.184+0000 7f9ac128f640 1 -- 172.21.15.120:0/3937278999 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab4048200 con 0x7f9ab404ae70 2024-09-10T10:27:10.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.186+0000 7f9ab8ff9640 1 -- 172.21.15.120:0/3937278999 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0003810 con 0x7f9ab404ae70 2024-09-10T10:27:10.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.187+0000 7f9ab8ff9640 1 -- 172.21.15.120:0/3937278999 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0036040 con 0x7f9ab404ae70 2024-09-10T10:27:10.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.188+0000 7f9ab8ff9640 1 -- 172.21.15.120:0/3937278999 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ab0044020 con 0x7f9ab404ae70 2024-09-10T10:27:10.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.190+0000 7f9aa67fc640 1 -- 172.21.15.120:0/3937278999 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a9c005290 con 0x7f9ab404ae70 2024-09-10T10:27:10.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.190+0000 7f9ab8ff9640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aac06c040 0x7f9aac06e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.190+0000 7f9ab8ff9640 1 -- 172.21.15.120:0/3937278999 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9ab00abf40 con 0x7f9ab404ae70 2024-09-10T10:27:10.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.191+0000 7f9abbfff640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aac06c040 0x7f9aac06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.191+0000 7f9abbfff640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aac06c040 0x7f9aac06e500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800bbd0 tx=0x7f9aa8006d20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.197+0000 7f9ab8ff9640 1 -- 172.21.15.120:0/3937278999 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9ab004b030 con 0x7f9ab404ae70 2024-09-10T10:27:10.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.221+0000 7f3450cd0640 1 Processor -- start 2024-09-10T10:27:10.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.221+0000 7f3450cd0640 1 -- start start 2024-09-10T10:27:10.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.221+0000 7ff31490a640 1 Processor -- start 2024-09-10T10:27:10.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.222+0000 7f3450cd0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f344c0c8fa0 0x7f344c0c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.223+0000 7f3450cd0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c06f040 0x7f344c157110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.223+0000 7f3450cd0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f344c06f5a0 con 0x7f344c0c7cc0 2024-09-10T10:27:10.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.221+0000 7ff31490a640 1 -- start start 2024-09-10T10:27:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.223+0000 7ff30dd9b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff30804d070 0x7ff30804dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49192/0 (socket says 172.21.15.120:49192) 2024-09-10T10:27:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.223+0000 7ff31490a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff308046d70 0x7ff3080e0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.223+0000 7f3449d9b640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f344c0c7cc0 0x7f344c073080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:35800/0 (socket says 172.21.15.120:35800) 2024-09-10T10:27:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.223+0000 7ff30dd9b640 1 -- 172.21.15.120:0/3237416318 learned_addr learned my addr 172.21.15.120:0/3237416318 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.223+0000 7ff30ed9d640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff308046d70 0x7ff3080e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.224+0000 7ff31490a640 1 -- 172.21.15.120:0/3237416318 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3080dcd60 con 0x7ff30804d070 2024-09-10T10:27:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.224+0000 7ff31490a640 1 -- 172.21.15.120:0/3237416318 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3080dd040 con 0x7ff3080d66a0 2024-09-10T10:27:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.224+0000 7ff31490a640 1 -- 172.21.15.120:0/3237416318 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff3080dd320 con 0x7ff308046d70 2024-09-10T10:27:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.223+0000 7f3449d9b640 1 -- 172.21.15.120:0/1939127705 learned_addr learned my addr 172.21.15.120:0/1939127705 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.224+0000 7f3450cd0640 1 -- 172.21.15.120:0/1939127705 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f344c0c98c0 con 0x7f344c0c8fa0 2024-09-10T10:27:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.224+0000 7f3450cd0640 1 -- 172.21.15.120:0/1939127705 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f344c06f880 con 0x7f344c06f040 2024-09-10T10:27:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.224+0000 7f344ad9d640 1 --2- 172.21.15.120:0/1939127705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c06f040 0x7f344c157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.224+0000 7f344a59c640 1 --2- 172.21.15.120:0/1939127705 >> v2:172.21.15.120:3300/0 conn(0x7f344c0c8fa0 0x7f344c0c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.225+0000 7f344ad9d640 1 -- 172.21.15.120:0/1939127705 >> v2:172.21.15.120:3300/0 conn(0x7f344c0c8fa0 msgr2=0x7f344c0c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.225+0000 7f344ad9d640 1 --2- 172.21.15.120:0/1939127705 >> v2:172.21.15.120:3300/0 conn(0x7f344c0c8fa0 0x7f344c0c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.225+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4282923051 0 0) 0x7ff3080dcd60 con 0x7ff30804d070 2024-09-10T10:27:10.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.225+0000 7f344ad9d640 1 -- 172.21.15.120:0/1939127705 >> v1:172.21.15.134:6789/0 conn(0x7f344c0c7cc0 legacy=0x7f344c073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.225+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2f8003520 con 0x7ff30804d070 2024-09-10T10:27:10.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.225+0000 7fae466c1640 1 Processor -- start 2024-09-10T10:27:10.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.225+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2307176108 0 0) 0x7ff3080dd040 con 0x7ff3080d66a0 2024-09-10T10:27:10.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.225+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3080dcd60 con 0x7ff3080d66a0 2024-09-10T10:27:10.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3789108086 0 0) 0x7ff3080dcd60 con 0x7ff3080d66a0 2024-09-10T10:27:10.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff308046d70 msgr2=0x7ff3080e0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff308046d70 0x7ff3080e0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 >> v1:172.21.15.120:6789/0 conn(0x7ff30804d070 legacy=0x7ff30804dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3080dfa20 con 0x7ff3080d66a0 2024-09-10T10:27:10.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7ff304001670 con 0x7ff3080d66a0 2024-09-10T10:27:10.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7f344ad9d640 1 -- 172.21.15.120:0/1939127705 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344c06fb60 con 0x7f344c06f040 2024-09-10T10:27:10.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 >> v1:172.21.15.134:6789/0 conn(0x7ff3080d66a0 legacy=0x7ff3080dc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f8003b40 0x7ff2f8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7f344a59c640 1 --2- 172.21.15.120:0/1939127705 >> v2:172.21.15.120:3300/0 conn(0x7f344c0c8fa0 0x7f344c0c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f8004800 0x7ff2f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 0x7ff2f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3080dd320 con 0x7ff2f8003b40 2024-09-10T10:27:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f800c0c0 con 0x7ff2f8004800 2024-09-10T10:27:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f800c2c0 con 0x7ff2f8005210 2024-09-10T10:27:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.226+0000 7ff30ed9d640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff308046d70 0x7ff3080e0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.225+0000 7fae466c1640 1 -- start start 2024-09-10T10:27:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7fae466c1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fae3804a680 0x7fae3804aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7fae466c1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae380e6a70 con 0x7fae380dcd90 2024-09-10T10:27:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7fae466c1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae380e6d30 con 0x7fae380e5f30 2024-09-10T10:27:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7fae466c1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fae38044d40 con 0x7fae3804a680 2024-09-10T10:27:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7ff30e59c640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f8003b40 0x7ff2f8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7ff30ed9d640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 0x7ff2f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7fae44ebe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fae3804a680 0x7fae3804aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7fae44ebe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fae3804a680 0x7fae3804aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34516/0 (socket says 172.21.15.120:34516) 2024-09-10T10:27:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7fae44ebe640 1 -- 172.21.15.120:0/2793717410 learned_addr learned my addr 172.21.15.120:0/2793717410 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7ff30dd9b640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f8004800 0x7ff2f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.228+0000 7fae44ebe640 1 -- 172.21.15.120:0/2793717410 >> v1:172.21.15.134:6789/0 conn(0x7fae380e5f30 legacy=0x7fae380e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.228+0000 7fae44ebe640 1 -- 172.21.15.120:0/2793717410 >> v1:172.21.15.120:6789/0 conn(0x7fae380dcd90 legacy=0x7fae38045d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.228+0000 7fae44ebe640 1 -- 172.21.15.120:0/2793717410 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae38044f90 con 0x7fae3804a680 2024-09-10T10:27:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7ff30ed9d640 1 -- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f8004800 msgr2=0x7ff2f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7ff30ed9d640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f8004800 0x7ff2f8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7ff30ed9d640 1 -- 172.21.15.120:0/3237416318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f8003b40 msgr2=0x7ff2f8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.227+0000 7ff30ed9d640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f8003b40 0x7ff2f8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.228+0000 7ff30ed9d640 1 -- 172.21.15.120:0/3237416318 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3080dcd60 con 0x7ff2f8005210 2024-09-10T10:27:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.228+0000 7f344ad9d640 1 --2- 172.21.15.120:0/1939127705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c06f040 0x7f344c157110 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f34400097a0 tx=0x7f344002a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d51effbe77fbfd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.228+0000 7ff30ed9d640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 0x7ff2f800b9c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff310000b60 tx=0x7ff2f8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14be098b3dd17bc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.229+0000 7f344959a640 1 -- 172.21.15.120:0/1939127705 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440037070 con 0x7f344c06f040 2024-09-10T10:27:10.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.229+0000 7f344959a640 1 -- 172.21.15.120:0/1939127705 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344002ac90 con 0x7f344c06f040 2024-09-10T10:27:10.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.229+0000 7f344959a640 1 -- 172.21.15.120:0/1939127705 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440032660 con 0x7f344c06f040 2024-09-10T10:27:10.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.229+0000 7fae44ebe640 1 --2- 172.21.15.120:0/2793717410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3804a680 0x7fae3804aab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae30009870 tx=0x7fae3002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85aef4a1abc3afda server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.229+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310063640 con 0x7ff2f8005210 2024-09-10T10:27:10.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.229+0000 7ff30d59a640 1 -- 172.21.15.120:0/3237416318 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3100a87d0 con 0x7ff2f8005210 2024-09-10T10:27:10.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7ff31490a640 1 -- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 msgr2=0x7ff2f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7ff31490a640 1 --2- 172.21.15.120:0/3237416318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 0x7ff2f800b9c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff310000b60 tx=0x7ff2f8005830 comp rx=0 tx=0).stop 2024-09-10T10:27:10.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7ff30ed9d640 1 -- 172.21.15.120:0/3237416318 reap_dead start 2024-09-10T10:27:10.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7f3450cd0640 1 -- 172.21.15.120:0/1939127705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c06f040 msgr2=0x7f344c157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7f3450cd0640 1 --2- 172.21.15.120:0/1939127705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c06f040 0x7f344c157110 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f34400097a0 tx=0x7f344002a000 comp rx=0 tx=0).stop 2024-09-10T10:27:10.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7ff31490a640 1 -- 172.21.15.120:0/3237416318 shutdown_connections 2024-09-10T10:27:10.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7ff31490a640 1 -- 172.21.15.120:0/3237416318 >> 172.21.15.120:0/3237416318 conn(0x7ff30800dbc0 msgr2=0x7ff30800dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7ff31490a640 1 -- 172.21.15.120:0/3237416318 shutdown_connections 2024-09-10T10:27:10.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7ff31490a640 1 -- 172.21.15.120:0/3237416318 wait complete. 2024-09-10T10:27:10.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7fae37fff640 1 -- 172.21.15.120:0/2793717410 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30037070 con 0x7fae3804a680 2024-09-10T10:27:10.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7fae37fff640 1 -- 172.21.15.120:0/2793717410 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae30032640 con 0x7fae3804a680 2024-09-10T10:27:10.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.230+0000 7fae37fff640 1 -- 172.21.15.120:0/2793717410 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30037070 con 0x7fae3804a680 2024-09-10T10:27:10.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.231+0000 7f3450cd0640 1 -- 172.21.15.120:0/1939127705 shutdown_connections 2024-09-10T10:27:10.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.231+0000 7f3450cd0640 1 --2- 172.21.15.120:0/1939127705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c06f040 0x7f344c157110 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.231+0000 7f3450cd0640 1 --2- 172.21.15.120:0/1939127705 >> v2:172.21.15.120:3300/0 conn(0x7f344c0c8fa0 0x7f344c0c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.231+0000 7f3450cd0640 1 -- 172.21.15.120:0/1939127705 >> 172.21.15.120:0/1939127705 conn(0x7f344c06bc10 msgr2=0x7f344c06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.231+0000 7f3450cd0640 1 -- 172.21.15.120:0/1939127705 shutdown_connections 2024-09-10T10:27:10.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.231+0000 7f3450cd0640 1 -- 172.21.15.120:0/1939127705 wait complete. 2024-09-10T10:27:10.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.231+0000 7fae466c1640 1 -- 172.21.15.120:0/2793717410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3804a680 msgr2=0x7fae3804aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.231+0000 7fae466c1640 1 --2- 172.21.15.120:0/2793717410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3804a680 0x7fae3804aab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae30009870 tx=0x7fae3002a300 comp rx=0 tx=0).stop 2024-09-10T10:27:10.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.232+0000 7fae466c1640 1 -- 172.21.15.120:0/2793717410 shutdown_connections 2024-09-10T10:27:10.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.232+0000 7fae466c1640 1 --2- 172.21.15.120:0/2793717410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3804a680 0x7fae3804aab0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.232+0000 7fae466c1640 1 -- 172.21.15.120:0/2793717410 >> 172.21.15.120:0/2793717410 conn(0x7fae3804fc50 msgr2=0x7fae38050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.232+0000 7fae466c1640 1 -- 172.21.15.120:0/2793717410 shutdown_connections 2024-09-10T10:27:10.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.232+0000 7ff31490a640 1 Processor -- start 2024-09-10T10:27:10.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.232+0000 7fae466c1640 1 -- 172.21.15.120:0/2793717410 wait complete. 2024-09-10T10:27:10.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.232+0000 7ff31490a640 1 -- start start 2024-09-10T10:27:10.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.233+0000 7ff31490a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 0x7ff308049b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.233+0000 7ff31490a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3080cf9e0 0x7ff30804b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7f3450cd0640 1 Processor -- start 2024-09-10T10:27:10.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7fae466c1640 1 Processor -- start 2024-09-10T10:27:10.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7ff30dd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3080cf9e0 0x7ff30804b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7fae466c1640 1 -- start start 2024-09-10T10:27:10.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7f3450cd0640 1 -- start start 2024-09-10T10:27:10.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7f9335349640 1 Processor -- start 2024-09-10T10:27:10.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 0x7ff308049b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 0x7ff308049b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:42948/0 (socket says 172.21.15.120:42948) 2024-09-10T10:27:10.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7ff30dd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3080cf9e0 0x7ff30804b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41934/0 (socket says 172.21.15.120:41934) 2024-09-10T10:27:10.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7ff31490a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30804b760 0x7ff3081753f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7ff30e59c640 1 -- 172.21.15.120:0/3715553897 learned_addr learned my addr 172.21.15.120:0/3715553897 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7ff30ed9d640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30804b760 0x7ff3081753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff308047180 con 0x7ff30804b760 2024-09-10T10:27:10.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3080dcb00 con 0x7ff3080cf9e0 2024-09-10T10:27:10.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff3080dffe0 con 0x7ff2f8005210 2024-09-10T10:27:10.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.234+0000 7f9335349640 1 -- start start 2024-09-10T10:27:10.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7fae466c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3804a680 0x7fae3816fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7fae466c1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae380dcd90 0x7fae3816ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7fae466c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae380e5f30 0x7fae3816e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7fae466c1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae38045550 con 0x7fae3804a680 2024-09-10T10:27:10.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7f9335349640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280d8870 0x7f9328049950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7f9335349640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93280502d0 0x7f9328049e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7fae466c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae380dd170 con 0x7fae380e5f30 2024-09-10T10:27:10.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7fae466c1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fae38045830 con 0x7fae380dcd90 2024-09-10T10:27:10.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7f9335349640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9328050850 con 0x7f93280491c0 2024-09-10T10:27:10.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7f9335349640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f93280e1c60 con 0x7f93280502d0 2024-09-10T10:27:10.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.235+0000 7f9335349640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93280e1f40 con 0x7f93280d8870 2024-09-10T10:27:10.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7f932f7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f93280491c0 0x7f93280de730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:49200/0 (socket says 172.21.15.120:49200) 2024-09-10T10:27:10.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7f932f7fe640 1 -- 172.21.15.120:0/3892880257 learned_addr learned my addr 172.21.15.120:0/3892880257 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:10.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae45ec0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae380e5f30 0x7fae3816e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae45ec0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae380e5f30 0x7fae3816e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41944/0 (socket says 172.21.15.120:41944) 2024-09-10T10:27:10.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae45ec0640 1 -- 172.21.15.120:0/2467180661 learned_addr learned my addr 172.21.15.120:0/2467180661 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7f932e7fc640 1 --2- 172.21.15.120:0/3892880257 >> v2:172.21.15.134:3300/0 conn(0x7f93280502d0 0x7f9328049e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7f932effd640 1 --2- 172.21.15.120:0/3892880257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280d8870 0x7f9328049950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae456bf640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3804a680 0x7fae3816fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae44ebe640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae380dcd90 0x7fae3816ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7f3450cd0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f344c06f040 0x7f344c0bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae45ec0640 1 -- 172.21.15.120:0/2467180661 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae380dcd90 msgr2=0x7fae3816ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae45ec0640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae380dcd90 0x7fae3816ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae45ec0640 1 -- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3804a680 msgr2=0x7fae3816fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae45ec0640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3804a680 0x7fae3816fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae45ec0640 1 -- 172.21.15.120:0/2467180661 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae38006fc0 con 0x7fae380e5f30 2024-09-10T10:27:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae44ebe640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae380dcd90 0x7fae3816ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7f932dffb640 1 -- 172.21.15.120:0/3892880257 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1283420812 0 0) 0x7f9328050850 con 0x7f93280491c0 2024-09-10T10:27:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae456bf640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3804a680 0x7fae3816fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:10.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7f932dffb640 1 -- 172.21.15.120:0/3892880257 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9318003520 con 0x7f93280491c0 2024-09-10T10:27:10.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7fae45ec0640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae380e5f30 0x7fae3816e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae4006b850 tx=0x7fae40069890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7ff30dd9b640 1 -- 172.21.15.120:0/3715553897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 msgr2=0x7ff308049b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7ff30dd9b640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 0x7ff308049b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7f932e7fc640 1 -- 172.21.15.120:0/3892880257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280d8870 msgr2=0x7f9328049950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7f932e7fc640 1 --2- 172.21.15.120:0/3892880257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280d8870 0x7f9328049950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7f932e7fc640 1 -- 172.21.15.120:0/3892880257 >> v1:172.21.15.120:6789/0 conn(0x7f93280491c0 legacy=0x7f93280de730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7ff30dd9b640 1 -- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30804b760 msgr2=0x7ff3081753f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7fae35ffb640 1 -- 172.21.15.120:0/2467180661 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40091070 con 0x7fae380e5f30 2024-09-10T10:27:10.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7ff30dd9b640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30804b760 0x7ff3081753f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7ff30dd9b640 1 -- 172.21.15.120:0/3715553897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3080513a0 con 0x7ff3080cf9e0 2024-09-10T10:27:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.236+0000 7ff30ed9d640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30804b760 0x7ff3081753f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7f932e7fc640 1 -- 172.21.15.120:0/3892880257 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93280e2220 con 0x7f93280502d0 2024-09-10T10:27:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7f344a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f344c06f040 0x7f344c0bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7f344a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f344c06f040 0x7f344c0bcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34542/0 (socket says 172.21.15.120:34542) 2024-09-10T10:27:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7f344a59c640 1 -- 172.21.15.120:0/3822968454 learned_addr learned my addr 172.21.15.120:0/3822968454 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7fae466c1640 1 -- 172.21.15.120:0/2467180661 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae38047930 con 0x7fae380e5f30 2024-09-10T10:27:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7fae466c1640 1 -- 172.21.15.120:0/2467180661 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae38047ef0 con 0x7fae380e5f30 2024-09-10T10:27:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7fae35ffb640 1 -- 172.21.15.120:0/2467180661 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae40071840 con 0x7fae380e5f30 2024-09-10T10:27:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7f3450cd0640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c0c7cc0 0x7f344c0bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7f3450cd0640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f344c0c8fa0 0x7f344c1ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7f3449d9b640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c0c7cc0 0x7f344c0bd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7ff30dd9b640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3080cf9e0 0x7ff30804b220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff30000d7a0 tx=0x7ff30000dba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7f932e7fc640 1 --2- 172.21.15.120:0/3892880257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280502d0 0x7f9328049e90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f93300518b0 tx=0x7f93300639f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0cbb605390ae52a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f344c072b60 con 0x7f344c06f040 2024-09-10T10:27:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f344c072880 con 0x7f344c0c8fa0 2024-09-10T10:27:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f344c073790 con 0x7f344c0c7cc0 2024-09-10T10:27:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7f344ad9d640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f344c0c8fa0 0x7f344c1ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7f932dffb640 1 -- 172.21.15.120:0/3892880257 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93300aa930 con 0x7f93280502d0 2024-09-10T10:27:10.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7ff2feffd640 1 -- 172.21.15.120:0/3715553897 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300005ad0 con 0x7ff3080cf9e0 2024-09-10T10:27:10.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7fae35ffb640 1 -- 172.21.15.120:0/2467180661 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40071cd0 con 0x7fae380e5f30 2024-09-10T10:27:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7ff2feffd640 1 -- 172.21.15.120:0/3715553897 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff300017070 con 0x7ff3080cf9e0 2024-09-10T10:27:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7f932dffb640 1 -- 172.21.15.120:0/3892880257 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93300aaad0 con 0x7f93280502d0 2024-09-10T10:27:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7f932dffb640 1 -- 172.21.15.120:0/3892880257 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93300aa930 con 0x7f93280502d0 2024-09-10T10:27:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7ff2feffd640 1 -- 172.21.15.120:0/3715553897 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300007070 con 0x7ff3080cf9e0 2024-09-10T10:27:10.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7efcb988f640 1 Processor -- start 2024-09-10T10:27:10.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.237+0000 7efcb988f640 1 -- start start 2024-09-10T10:27:10.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7efcb988f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0dce90 0x7efcac005a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7efcb988f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efcac0e6030 0x7efcac0e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7f344a59c640 1 -- 172.21.15.120:0/3822968454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c0c7cc0 msgr2=0x7f344c0bd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7f344a59c640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c0c7cc0 0x7f344c0bd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7f344a59c640 1 -- 172.21.15.120:0/3822968454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f344c0c8fa0 msgr2=0x7f344c1ee3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7f344a59c640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f344c0c8fa0 0x7f344c1ee3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7f344a59c640 1 -- 172.21.15.120:0/3822968454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344c1edea0 con 0x7f344c06f040 2024-09-10T10:27:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7f9335349640 1 -- 172.21.15.120:0/3892880257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280502d0 msgr2=0x7f9328049e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7f9335349640 1 --2- 172.21.15.120:0/3892880257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280502d0 0x7f9328049e90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f93300518b0 tx=0x7f93300639f0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7fae35ffb640 1 -- 172.21.15.120:0/2467180661 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae40094070 con 0x7fae380e5f30 2024-09-10T10:27:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7fae466c1640 1 -- 172.21.15.120:0/2467180661 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae20005290 con 0x7fae380e5f30 2024-09-10T10:27:10.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30804a7b0 con 0x7ff3080cf9e0 2024-09-10T10:27:10.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff308051610 con 0x7ff3080cf9e0 2024-09-10T10:27:10.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7f9335349640 1 -- 172.21.15.120:0/3892880257 shutdown_connections 2024-09-10T10:27:10.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7f9335349640 1 --2- 172.21.15.120:0/3892880257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280502d0 0x7f9328049e90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7f9335349640 1 --2- 172.21.15.120:0/3892880257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280d8870 0x7f9328049950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7f9335349640 1 -- 172.21.15.120:0/3892880257 >> 172.21.15.120:0/3892880257 conn(0x7f9328054450 msgr2=0x7f9328054860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7efcb988f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcac0e6ae0 con 0x7efcac04aa40 2024-09-10T10:27:10.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7efcb988f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7efcac045660 con 0x7efcac0e6030 2024-09-10T10:27:10.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.238+0000 7efcb988f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efcac045f00 con 0x7efcac0dce90 2024-09-10T10:27:10.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7efcb888d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0dce90 0x7efcac005a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7efcb888d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0dce90 0x7efcac005a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:42984/0 (socket says 172.21.15.120:42984) 2024-09-10T10:27:10.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7f344a59c640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f344c06f040 0x7f344c0bcca0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f343c00e7e0 tx=0x7f343c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7efcb888d640 1 -- 172.21.15.120:0/1547570418 learned_addr learned my addr 172.21.15.120:0/1547570418 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7efcb908e640 1 --2- 172.21.15.120:0/1547570418 >> v2:172.21.15.120:3300/0 conn(0x7efcac0e6030 0x7efcac0e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7efcb888d640 1 -- 172.21.15.120:0/1547570418 >> v2:172.21.15.120:3300/0 conn(0x7efcac0e6030 msgr2=0x7efcac0e64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7efcb888d640 1 --2- 172.21.15.120:0/1547570418 >> v2:172.21.15.120:3300/0 conn(0x7efcac0e6030 0x7efcac0e64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7efcb888d640 1 -- 172.21.15.120:0/1547570418 >> v1:172.21.15.134:6789/0 conn(0x7efcac04aa40 legacy=0x7efcac046390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:27:10.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.239+0000 7efcb888d640 1 -- 172.21.15.120:0/1547570418 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac0e6dc0 con 0x7efcac0dce90 2024-09-10T10:27:10.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.241+0000 7fae35ffb640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fae2406bf30 0x7fae2406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7f9335349640 1 -- 172.21.15.120:0/3892880257 shutdown_connections 2024-09-10T10:27:10.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7f9335349640 1 -- 172.21.15.120:0/3892880257 wait complete. 2024-09-10T10:27:10.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7efcb888d640 1 --2- 172.21.15.120:0/1547570418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0dce90 0x7efcac005a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efcb4051500 tx=0x7efcb40639f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bab877ba5eab06a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7efcb37fe640 1 -- 172.21.15.120:0/1547570418 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40a8950 con 0x7efcac0dce90 2024-09-10T10:27:10.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.240+0000 7efcb37fe640 1 -- 172.21.15.120:0/1547570418 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb40a8af0 con 0x7efcac0dce90 2024-09-10T10:27:10.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.241+0000 7fae456bf640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fae2406bf30 0x7fae2406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7fae456bf640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fae2406bf30 0x7fae2406e3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae3c003f00 tx=0x7fae3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7fae35ffb640 1 -- 172.21.15.120:0/2467180661 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fae401073b0 con 0x7fae380e5f30 2024-09-10T10:27:10.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7f343affd640 1 -- 172.21.15.120:0/3822968454 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c00f040 con 0x7f344c06f040 2024-09-10T10:27:10.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.241+0000 7efcb988f640 1 -- 172.21.15.120:0/1547570418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0dce90 msgr2=0x7efcac005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.241+0000 7efcb988f640 1 --2- 172.21.15.120:0/1547570418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0dce90 0x7efcac005a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efcb4051500 tx=0x7efcb40639f0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7f343affd640 1 -- 172.21.15.120:0/3822968454 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c004030 con 0x7f344c06f040 2024-09-10T10:27:10.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f344c14e240 con 0x7f344c06f040 2024-09-10T10:27:10.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30804cdd0 con 0x7ff3080cf9e0 2024-09-10T10:27:10.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344c14e800 con 0x7f344c06f040 2024-09-10T10:27:10.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7efcb988f640 1 -- 172.21.15.120:0/1547570418 shutdown_connections 2024-09-10T10:27:10.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7efcb988f640 1 --2- 172.21.15.120:0/1547570418 >> v2:172.21.15.120:3300/0 conn(0x7efcac0e6030 0x7efcac0e64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7efcb988f640 1 --2- 172.21.15.120:0/1547570418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0dce90 0x7efcac005a00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7efcb988f640 1 -- 172.21.15.120:0/1547570418 >> 172.21.15.120:0/1547570418 conn(0x7efcac04fc50 msgr2=0x7efcac050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7f343affd640 1 -- 172.21.15.120:0/3822968454 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c004400 con 0x7f344c06f040 2024-09-10T10:27:10.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7f9335349640 1 Processor -- start 2024-09-10T10:27:10.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.243+0000 7f9335349640 1 -- start start 2024-09-10T10:27:10.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.242+0000 7efcb988f640 1 -- 172.21.15.120:0/1547570418 shutdown_connections 2024-09-10T10:27:10.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.243+0000 7efcb988f640 1 -- 172.21.15.120:0/1547570418 wait complete. 2024-09-10T10:27:10.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.243+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3430005290 con 0x7f344c06f040 2024-09-10T10:27:10.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f9335349640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280491c0 0x7f9328053200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f9335349640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93280502d0 0x7f9328177a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f9335349640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280d8870 0x7f9328177f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f932effd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280491c0 0x7f9328053200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f932effd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280491c0 0x7f9328053200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:42998/0 (socket says 172.21.15.120:42998) 2024-09-10T10:27:10.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f932e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93280502d0 0x7f9328177a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f932e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93280502d0 0x7f9328177a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34556/0 (socket says 172.21.15.120:34556) 2024-09-10T10:27:10.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f932effd640 1 -- 172.21.15.120:0/1648158484 learned_addr learned my addr 172.21.15.120:0/1648158484 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93280d8c50 con 0x7f93280502d0 2024-09-10T10:27:10.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f932800f720 con 0x7f93280d8870 2024-09-10T10:27:10.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93280d8dd0 con 0x7f93280491c0 2024-09-10T10:27:10.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.244+0000 7f932f7fe640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280d8870 0x7f9328177f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7f932effd640 1 -- 172.21.15.120:0/1648158484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280d8870 msgr2=0x7f9328177f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7f932effd640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280d8870 0x7f9328177f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7f932effd640 1 -- 172.21.15.120:0/1648158484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93280502d0 msgr2=0x7f9328177a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7fae35ffb640 1 -- 172.21.15.120:0/2467180661 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fae400d5a00 con 0x7fae380e5f30 2024-09-10T10:27:10.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7f932effd640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93280502d0 0x7f9328177a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7efcb988f640 1 Processor -- start 2024-09-10T10:27:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7f932effd640 1 -- 172.21.15.120:0/1648158484 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93280d62a0 con 0x7f93280491c0 2024-09-10T10:27:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7f932e7fc640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93280502d0 0x7f9328177a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7f932effd640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280491c0 0x7f9328053200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f932000e9b0 tx=0x7f932000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7efcb988f640 1 -- start start 2024-09-10T10:27:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7f93177fe640 1 -- 172.21.15.120:0/1648158484 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320017070 con 0x7f93280491c0 2024-09-10T10:27:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7f93177fe640 1 -- 172.21.15.120:0/1648158484 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932000c9a0 con 0x7f93280491c0 2024-09-10T10:27:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.245+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93280d6580 con 0x7f93280491c0 2024-09-10T10:27:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.246+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f93280d6b40 con 0x7f93280491c0 2024-09-10T10:27:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.246+0000 7efcb988f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac04aa40 0x7efcac047440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb988f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac0dce90 0x7efcac0db700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.246+0000 7f93177fe640 1 -- 172.21.15.120:0/1648158484 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932000cd90 con 0x7f93280491c0 2024-09-10T10:27:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb888d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac04aa40 0x7efcac047440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb888d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac04aa40 0x7efcac047440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41982/0 (socket says 172.21.15.120:41982) 2024-09-10T10:27:10.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac0dce90 0x7efcac0db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb888d640 1 -- 172.21.15.120:0/3771709346 learned_addr learned my addr 172.21.15.120:0/3771709346 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb988f640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0e6030 0x7efcac0dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9328036e20 con 0x7f93280491c0 2024-09-10T10:27:10.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efcac045b90 con 0x7efcac0dce90 2024-09-10T10:27:10.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efcac0dd270 con 0x7efcac04aa40 2024-09-10T10:27:10.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efcac044b30 con 0x7efcac0e6030 2024-09-10T10:27:10.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7f343affd640 1 -- 172.21.15.120:0/3822968454 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f343c00c6d0 con 0x7f344c06f040 2024-09-10T10:27:10.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb888d640 1 -- 172.21.15.120:0/3771709346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0e6030 msgr2=0x7efcac0dbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb888d640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0e6030 0x7efcac0dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb888d640 1 -- 172.21.15.120:0/3771709346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac0dce90 msgr2=0x7efcac0db700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.247+0000 7efcb888d640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac0dce90 0x7efcac0db700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.248+0000 7efcb888d640 1 -- 172.21.15.120:0/3771709346 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac0dd4d0 con 0x7efcac04aa40 2024-09-10T10:27:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.248+0000 7efcb888d640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac04aa40 0x7efcac047440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efcb4069460 tx=0x7efcb4074750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.248+0000 7f343affd640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f342c06be30 0x7f342c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.248+0000 7f3449d9b640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f342c06be30 0x7f342c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.249+0000 7f3449d9b640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f342c06be30 0x7f342c06e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f344c0753b0 tx=0x7f3444001890 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.249+0000 7f93177fe640 1 -- 172.21.15.120:0/1648158484 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f932001a070 con 0x7f93280491c0 2024-09-10T10:27:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.249+0000 7ff2feffd640 1 -- 172.21.15.120:0/3715553897 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff300007210 con 0x7ff3080cf9e0 2024-09-10T10:27:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.250+0000 7f93177fe640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f931c06beb0 0x7f931c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.250+0000 7f93177fe640 1 -- 172.21.15.120:0/1648158484 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f932008b9c0 con 0x7f93280491c0 2024-09-10T10:27:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.250+0000 7efcb17fa640 1 -- 172.21.15.120:0/3771709346 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40a9e40 con 0x7efcac04aa40 2024-09-10T10:27:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.250+0000 7efcb17fa640 1 -- 172.21.15.120:0/3771709346 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb40b2610 con 0x7efcac04aa40 2024-09-10T10:27:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.250+0000 7efcb17fa640 1 -- 172.21.15.120:0/3771709346 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40b29c0 con 0x7efcac04aa40 2024-09-10T10:27:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.250+0000 7f932e7fc640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f931c06beb0 0x7f931c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.250+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcac0dd790 con 0x7efcac04aa40 2024-09-10T10:27:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.251+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcac0de1d0 con 0x7efcac04aa40 2024-09-10T10:27:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.250+0000 7f343affd640 1 -- 172.21.15.120:0/3822968454 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f343c08b010 con 0x7f344c06f040 2024-09-10T10:27:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.251+0000 7f932e7fc640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f931c06beb0 0x7f931c06e370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f932800f560 tx=0x7f933006e030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.250+0000 7ff2feffd640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff2f006bf00 0x7ff2f006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.251+0000 7ff2feffd640 1 -- 172.21.15.120:0/3715553897 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff30008b950 con 0x7ff3080cf9e0 2024-09-10T10:27:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.251+0000 7ff30e59c640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff2f006bf00 0x7ff2f006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.251+0000 7ff2feffd640 1 -- 172.21.15.120:0/3715553897 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff30005a030 con 0x7ff3080cf9e0 2024-09-10T10:27:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.252+0000 7ff30e59c640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff2f006bf00 0x7ff2f006e3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff304001050 tx=0x7ff304009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.254+0000 7f93177fe640 1 -- 172.21.15.120:0/1648158484 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9320059930 con 0x7f93280491c0 2024-09-10T10:27:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.253+0000 7efcb17fa640 1 -- 172.21.15.120:0/3771709346 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcb40c4030 con 0x7efcac04aa40 2024-09-10T10:27:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.253+0000 7efcb17fa640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc9806bf00 0x7efc9806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.254+0000 7efcb17fa640 1 -- 172.21.15.120:0/3771709346 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efcb4128490 con 0x7efcac04aa40 2024-09-10T10:27:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.254+0000 7f343affd640 1 -- 172.21.15.120:0/3822968454 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f343c017020 con 0x7f344c06f040 2024-09-10T10:27:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.254+0000 7efcb3fff640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc9806bf00 0x7efc9806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.255+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcac045450 con 0x7efcac04aa40 2024-09-10T10:27:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.255+0000 7efcb3fff640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc9806bf00 0x7efc9806e3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efcac04a1e0 tx=0x7efca00090e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.262+0000 7efcb17fa640 1 -- 172.21.15.120:0/3771709346 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efcb40cb060 con 0x7efcac04aa40 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.273+0000 7f8ad0b78640 1 Processor -- start 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.273+0000 7f8ad0b78640 1 -- start start 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.274+0000 7f8ad0b78640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8acc0c89b0 0x7f8acc1626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.274+0000 7f8ad0b78640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8acc162bf0 0x7f8acc169670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.274+0000 7f8ad0b78640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8acc163080 con 0x7f8acc1592b0 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.274+0000 7f8ad0b78640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f8acc06d010 con 0x7f8acc162bf0 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.274+0000 7f8ad0b78640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc0c7f20 con 0x7f8acc0c89b0 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.275+0000 7f8acad9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8acc162bf0 0x7f8acc169670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.275+0000 7f8acad9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8acc162bf0 0x7f8acc169670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34576/0 (socket says 172.21.15.120:34576) 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.275+0000 7f8acad9d640 1 -- 172.21.15.120:0/1855914422 learned_addr learned my addr 172.21.15.120:0/1855914422 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.275+0000 7f8acad9d640 1 -- 172.21.15.120:0/1855914422 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8acc0c89b0 msgr2=0x7f8acc1626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.275+0000 7f8acad9d640 1 --2- 172.21.15.120:0/1855914422 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8acc0c89b0 0x7f8acc1626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.275+0000 7f8acad9d640 1 -- 172.21.15.120:0/1855914422 >> v1:172.21.15.120:6789/0 conn(0x7f8acc1592b0 legacy=0x7f8acc0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.275+0000 7f8acad9d640 1 -- 172.21.15.120:0/1855914422 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc0c81e0 con 0x7f8acc162bf0 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.276+0000 7f8acad9d640 1 --2- 172.21.15.120:0/1855914422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8acc162bf0 0x7f8acc169670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009e50 tx=0x7f8ac402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de9caf0a8701821 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.277+0000 7f8ac959a640 1 -- 172.21.15.120:0/1855914422 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4036070 con 0x7f8acc162bf0 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.277+0000 7f8ac959a640 1 -- 172.21.15.120:0/1855914422 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac4031600 con 0x7f8acc162bf0 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.278+0000 7f8ad0b78640 1 -- 172.21.15.120:0/1855914422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8acc162bf0 msgr2=0x7f8acc169670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.278+0000 7f8ad0b78640 1 --2- 172.21.15.120:0/1855914422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8acc162bf0 0x7f8acc169670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009e50 tx=0x7f8ac402a750 comp rx=0 tx=0).stop 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.278+0000 7f8ad0b78640 1 -- 172.21.15.120:0/1855914422 shutdown_connections 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.278+0000 7f8ad0b78640 1 --2- 172.21.15.120:0/1855914422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8acc162bf0 0x7f8acc169670 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.278+0000 7f8ad0b78640 1 --2- 172.21.15.120:0/1855914422 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8acc0c89b0 0x7f8acc1626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.278+0000 7f8ad0b78640 1 -- 172.21.15.120:0/1855914422 >> 172.21.15.120:0/1855914422 conn(0x7f8acc067d40 msgr2=0x7f8acc070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.279+0000 7f8ad0b78640 1 -- 172.21.15.120:0/1855914422 shutdown_connections 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.279+0000 7f8ad0b78640 1 -- 172.21.15.120:0/1855914422 wait complete. 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.281+0000 7f8ad0b78640 1 Processor -- start 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.281+0000 7f8ad0b78640 1 -- start start 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8ad0b78640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8acc0c89b0 0x7f8acc164c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8ad0b78640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8acc1592b0 0x7f8acc165a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8ad0b78640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8acc162bf0 0x7f8acc163430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8ad0b78640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc06ca50 con 0x7f8acc162bf0 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8ad0b78640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc06c770 con 0x7f8acc1592b0 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8ad0b78640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc06cd30 con 0x7f8acc0c89b0 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8ac9d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8acc1592b0 0x7f8acc165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8ac9d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8acc1592b0 0x7f8acc165a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:41990/0 (socket says 172.21.15.120:41990) 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8ac9d9b640 1 -- 172.21.15.120:0/623861662 learned_addr learned my addr 172.21.15.120:0/623861662 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8aca59c640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8acc0c89b0 0x7f8acc164c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7f8acad9d640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8acc162bf0 0x7f8acc163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.283+0000 7f8aca59c640 1 -- 172.21.15.120:0/623861662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8acc1592b0 msgr2=0x7f8acc165a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.283+0000 7f8aca59c640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8acc1592b0 0x7f8acc165a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.283+0000 7f8aca59c640 1 -- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8acc162bf0 msgr2=0x7f8acc163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.283+0000 7f8aca59c640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8acc162bf0 0x7f8acc163430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.283+0000 7f8aca59c640 1 -- 172.21.15.120:0/623861662 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc1530b0 con 0x7f8acc0c89b0 2024-09-10T10:27:10.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.283+0000 7f8acad9d640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8acc162bf0 0x7f8acc163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:10.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.283+0000 7f8aca59c640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8acc0c89b0 0x7f8acc164c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00c390 tx=0x7f8abc00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7fdc5ca1e640 1 Processor -- start 2024-09-10T10:27:10.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.282+0000 7fdc5ca1e640 1 -- start start 2024-09-10T10:27:10.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.284+0000 7fdc5ca1e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500e67f0 0x7fdc500ee480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.284+0000 7fdc5ca1e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc50046bf0 con 0x7fdc500dce30 2024-09-10T10:27:10.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.284+0000 7fdc5ca1e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc50046e90 con 0x7fdc50046230 2024-09-10T10:27:10.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.284+0000 7fdc5ca1e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdc50047730 con 0x7fdc500e67f0 2024-09-10T10:27:10.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.284+0000 7fdc56d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500e67f0 0x7fdc500ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.284+0000 7fdc56d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500e67f0 0x7fdc500ee480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:43040/0 (socket says 172.21.15.120:43040) 2024-09-10T10:27:10.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.284+0000 7fdc56d9d640 1 -- 172.21.15.120:0/1463580803 learned_addr learned my addr 172.21.15.120:0/1463580803 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.284+0000 7f8abaffd640 1 -- 172.21.15.120:0/623861662 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc017070 con 0x7f8acc0c89b0 2024-09-10T10:27:10.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.285+0000 7fdc56d9d640 1 -- 172.21.15.120:0/1463580803 >> v1:172.21.15.134:6789/0 conn(0x7fdc50046230 legacy=0x7fdc500e60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.285+0000 7fdc56d9d640 1 -- 172.21.15.120:0/1463580803 >> v1:172.21.15.120:6789/0 conn(0x7fdc500dce30 legacy=0x7fdc50007570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:10.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.285+0000 7f8ad0b78640 1 -- 172.21.15.120:0/623861662 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc152090 con 0x7f8acc0c89b0 2024-09-10T10:27:10.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.285+0000 7f8ad0b78640 1 -- 172.21.15.120:0/623861662 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc164420 con 0x7f8acc0c89b0 2024-09-10T10:27:10.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.285+0000 7f8abaffd640 1 -- 172.21.15.120:0/623861662 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8abc0052d0 con 0x7f8acc0c89b0 2024-09-10T10:27:10.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.285+0000 7f8abaffd640 1 -- 172.21.15.120:0/623861662 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc003810 con 0x7f8acc0c89b0 2024-09-10T10:27:10.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.285+0000 7fdc56d9d640 1 -- 172.21.15.120:0/1463580803 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc50047150 con 0x7fdc500e67f0 2024-09-10T10:27:10.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.287+0000 7fdc56d9d640 1 --2- 172.21.15.120:0/1463580803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500e67f0 0x7fdc500ee480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc500478b0 tx=0x7fdc580639f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b343fc589bcb0027 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.287+0000 7fdc5559a640 1 -- 172.21.15.120:0/1463580803 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580aa920 con 0x7fdc500e67f0 2024-09-10T10:27:10.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.287+0000 7fdc5559a640 1 -- 172.21.15.120:0/1463580803 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc580aaac0 con 0x7fdc500e67f0 2024-09-10T10:27:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.288+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/1463580803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500e67f0 msgr2=0x7fdc500ee480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.288+0000 7fdc5ca1e640 1 --2- 172.21.15.120:0/1463580803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500e67f0 0x7fdc500ee480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc500478b0 tx=0x7fdc580639f0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.288+0000 7f8abaffd640 1 -- 172.21.15.120:0/623861662 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8abc01a070 con 0x7f8acc0c89b0 2024-09-10T10:27:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.288+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/1463580803 shutdown_connections 2024-09-10T10:27:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.288+0000 7fdc5ca1e640 1 --2- 172.21.15.120:0/1463580803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500e67f0 0x7fdc500ee480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.288+0000 7f8abaffd640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8aac06bf00 0x7f8aac06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.288+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/1463580803 >> 172.21.15.120:0/1463580803 conn(0x7fdc5004d830 msgr2=0x7fdc5004dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.289+0000 7f8ac9d9b640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8aac06bf00 0x7f8aac06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.289+0000 7f8abaffd640 1 -- 172.21.15.120:0/623861662 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8abc08a520 con 0x7f8acc0c89b0 2024-09-10T10:27:10.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.289+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/1463580803 shutdown_connections 2024-09-10T10:27:10.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.289+0000 7f8ac9d9b640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8aac06bf00 0x7f8aac06e3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8acc156020 tx=0x7f8ac00091c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.290+0000 7f8ad0b78640 1 -- 172.21.15.120:0/623861662 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab0005290 con 0x7f8acc0c89b0 2024-09-10T10:27:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.289+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/1463580803 wait complete. 2024-09-10T10:27:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.291+0000 7fdc5ca1e640 1 Processor -- start 2024-09-10T10:27:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.291+0000 7fdc5ca1e640 1 -- start start 2024-09-10T10:27:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.292+0000 7fdc5ca1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc50046230 0x7fdc500d2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.293+0000 7fdc5659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc50046230 0x7fdc500d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.293+0000 7fdc5659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc50046230 0x7fdc500d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34598/0 (socket says 172.21.15.120:34598) 2024-09-10T10:27:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.293+0000 7fdc5ca1e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500dce30 0x7fdc500cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.293+0000 7fdc5659c640 1 -- 172.21.15.120:0/3598427810 learned_addr learned my addr 172.21.15.120:0/3598427810 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.293+0000 7fdc5ca1e640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc500e67f0 0x7fdc50034250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.293+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc50007c80 con 0x7fdc50046230 2024-09-10T10:27:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.293+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc50047c60 con 0x7fdc500e67f0 2024-09-10T10:27:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.293+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdc500dd210 con 0x7fdc500dce30 2024-09-10T10:27:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.293+0000 7fdc55d9b640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500dce30 0x7fdc500cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.293+0000 7fdc56d9d640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc500e67f0 0x7fdc50034250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.294+0000 7fdc55d9b640 1 -- 172.21.15.120:0/3598427810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc500e67f0 msgr2=0x7fdc50034250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.294+0000 7fdc55d9b640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc500e67f0 0x7fdc50034250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.294+0000 7fdc55d9b640 1 -- 172.21.15.120:0/3598427810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc50046230 msgr2=0x7fdc500d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.294+0000 7fdc55d9b640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc50046230 0x7fdc500d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.294+0000 7fdc55d9b640 1 -- 172.21.15.120:0/3598427810 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc500365c0 con 0x7fdc500dce30 2024-09-10T10:27:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.294+0000 7fdc5659c640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc50046230 0x7fdc500d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.294+0000 7fdc55d9b640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500dce30 0x7fdc500cb620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800b3e0 tx=0x7fdc4800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.295+0000 7f8abaffd640 1 -- 172.21.15.120:0/623861662 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8abc0584d0 con 0x7f8acc0c89b0 2024-09-10T10:27:10.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.295+0000 7fdc46ffd640 1 -- 172.21.15.120:0/3598427810 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48015020 con 0x7fdc500dce30 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.295+0000 7fdc46ffd640 1 -- 172.21.15.120:0/3598427810 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc48011830 con 0x7fdc500dce30 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.295+0000 7fdc46ffd640 1 -- 172.21.15.120:0/3598427810 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48011c40 con 0x7fdc500dce30 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.295+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc50036000 con 0x7fdc500dce30 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.295+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc50033770 con 0x7fdc500dce30 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.296+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5004b730 con 0x7fdc500dce30 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.298+0000 7fdc46ffd640 1 -- 172.21.15.120:0/3598427810 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc48018020 con 0x7fdc500dce30 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.298+0000 7fdc46ffd640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc3806bf00 0x7fdc3806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.299+0000 7fdc46ffd640 1 -- 172.21.15.120:0/3598427810 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdc4808b2b0 con 0x7fdc500dce30 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.299+0000 7fdc5659c640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc3806bf00 0x7fdc3806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.300+0000 7fdc5659c640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc3806bf00 0x7fdc3806e3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009d80 tx=0x7fdc4c009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:10.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.301+0000 7fdc46ffd640 1 -- 172.21.15.120:0/3598427810 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc4801f020 con 0x7fdc500dce30 2024-09-10T10:27:10.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.569+0000 7f9aa67fc640 1 -- 172.21.15.120:0/3937278999 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9a9c005580 con 0x7f9ab404ae70 2024-09-10T10:27:10.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.570+0000 7f9ab8ff9640 1 -- 172.21.15.120:0/3937278999 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9ab007a5a0 con 0x7f9ab404ae70 2024-09-10T10:27:10.574 INFO:teuthology.orchestra.run.smithi120.stdout:55834574856 2024-09-10T10:27:10.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.573+0000 7f9aa67fc640 1 -- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aac06c040 msgr2=0x7f9aac06e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.573+0000 7f9aa67fc640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aac06c040 0x7f9aac06e500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800bbd0 tx=0x7f9aa8006d20 comp rx=0 tx=0).stop 2024-09-10T10:27:10.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.573+0000 7f9aa67fc640 1 -- 172.21.15.120:0/3937278999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ab404ae70 msgr2=0x7f9ab40471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.573+0000 7f9aa67fc640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ab404ae70 0x7f9ab40471c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009bc0 tx=0x7f9ab00046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.573+0000 7f9aa67fc640 1 -- 172.21.15.120:0/3937278999 shutdown_connections 2024-09-10T10:27:10.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.573+0000 7f9aa67fc640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aac06c040 0x7f9aac06e500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.573+0000 7f9aa67fc640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ab40d45f0 0x7f9ab40ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.573+0000 7f9aa67fc640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ab404ae70 0x7f9ab40471c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.573+0000 7f9aa67fc640 1 --2- 172.21.15.120:0/3937278999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ab4005a00 0x7f9ab4049800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.573+0000 7f9aa67fc640 1 -- 172.21.15.120:0/3937278999 >> 172.21.15.120:0/3937278999 conn(0x7f9ab404fc50 msgr2=0x7f9ab40d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.577+0000 7f9aa67fc640 1 -- 172.21.15.120:0/3937278999 shutdown_connections 2024-09-10T10:27:10.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.577+0000 7f9aa67fc640 1 -- 172.21.15.120:0/3937278999 wait complete. 2024-09-10T10:27:10.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-09-10T10:27:10.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.619+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efcac005a00 con 0x7efcac04aa40 2024-09-10T10:27:10.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.620+0000 7efcb17fa640 1 -- 172.21.15.120:0/3771709346 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efcb40ae030 con 0x7efcac04aa40 2024-09-10T10:27:10.621 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:10.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc9806bf00 msgr2=0x7efc9806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc9806bf00 0x7efc9806e3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efcac04a1e0 tx=0x7efca00090e0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac04aa40 msgr2=0x7efcac047440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac04aa40 0x7efcac047440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efcb4069460 tx=0x7efcb4074750 comp rx=0 tx=0).stop 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 shutdown_connections 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc9806bf00 0x7efc9806e3c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcac0e6030 0x7efcac0dbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcac0dce90 0x7efcac0db700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 --2- 172.21.15.120:0/3771709346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcac04aa40 0x7efcac047440 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 >> 172.21.15.120:0/3771709346 conn(0x7efcac04fc50 msgr2=0x7efcac0349c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 shutdown_connections 2024-09-10T10:27:10.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.622+0000 7efcb988f640 1 -- 172.21.15.120:0/3771709346 wait complete. 2024-09-10T10:27:10.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.632+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3430005580 con 0x7f344c06f040 2024-09-10T10:27:10.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.633+0000 7f343affd640 1 -- 172.21.15.120:0/3822968454 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f343c058f10 con 0x7f344c06f040 2024-09-10T10:27:10.635 INFO:teuthology.orchestra.run.smithi120.stdout:55834574856 2024-09-10T10:27:10.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-09-10T10:27:10.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f342c06be30 msgr2=0x7f342c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f342c06be30 0x7f342c06e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f344c0753b0 tx=0x7f3444001890 comp rx=0 tx=0).stop 2024-09-10T10:27:10.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f344c06f040 msgr2=0x7f344c0bcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f344c06f040 0x7f344c0bcca0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f343c00e7e0 tx=0x7f343c00eca0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 shutdown_connections 2024-09-10T10:27:10.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f342c06be30 0x7f342c06e2f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f344c0c8fa0 0x7f344c1ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f344c0c7cc0 0x7f344c0bd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 --2- 172.21.15.120:0/3822968454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f344c06f040 0x7f344c0bcca0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 >> 172.21.15.120:0/3822968454 conn(0x7f344c06bc10 msgr2=0x7f344c152360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 shutdown_connections 2024-09-10T10:27:10.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.636+0000 7f3450cd0640 1 -- 172.21.15.120:0/3822968454 wait complete. 2024-09-10T10:27:10.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.637+0000 7fae466c1640 1 -- 172.21.15.120:0/2467180661 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fae20005580 con 0x7fae380e5f30 2024-09-10T10:27:10.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.638+0000 7fae35ffb640 1 -- 172.21.15.120:0/2467180661 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fae400d5300 con 0x7fae380e5f30 2024-09-10T10:27:10.639 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:10.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.641+0000 7fae466c1640 1 -- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fae2406bf30 msgr2=0x7fae2406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.641+0000 7fae466c1640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fae2406bf30 0x7fae2406e3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae3c003f00 tx=0x7fae3c009040 comp rx=0 tx=0).stop 2024-09-10T10:27:10.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.641+0000 7fae466c1640 1 -- 172.21.15.120:0/2467180661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae380e5f30 msgr2=0x7fae3816e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.641+0000 7fae466c1640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae380e5f30 0x7fae3816e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae4006b850 tx=0x7fae40069890 comp rx=0 tx=0).stop 2024-09-10T10:27:10.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.642+0000 7fae466c1640 1 -- 172.21.15.120:0/2467180661 shutdown_connections 2024-09-10T10:27:10.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.642+0000 7fae466c1640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fae2406bf30 0x7fae2406e3f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.642+0000 7fae466c1640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae380e5f30 0x7fae3816e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.642+0000 7fae466c1640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae380dcd90 0x7fae3816ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.642+0000 7fae466c1640 1 --2- 172.21.15.120:0/2467180661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3804a680 0x7fae3816fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.642+0000 7fae466c1640 1 -- 172.21.15.120:0/2467180661 >> 172.21.15.120:0/2467180661 conn(0x7fae3804fc50 msgr2=0x7fae38005940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.642+0000 7fae466c1640 1 -- 172.21.15.120:0/2467180661 shutdown_connections 2024-09-10T10:27:10.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.642+0000 7fae466c1640 1 -- 172.21.15.120:0/2467180661 wait complete. 2024-09-10T10:27:10.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2024-09-10T10:27:10.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.653+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f932804e490 con 0x7f93280491c0 2024-09-10T10:27:10.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.653+0000 7f93177fe640 1 -- 172.21.15.120:0/1648158484 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f932005dc30 con 0x7f93280491c0 2024-09-10T10:27:10.654 INFO:teuthology.orchestra.run.smithi120.stdout:55834574856 2024-09-10T10:27:10.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.655+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff308005a30 con 0x7ff3080cf9e0 2024-09-10T10:27:10.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7ff2feffd640 1 -- 172.21.15.120:0/3715553897 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff300059930 con 0x7ff3080cf9e0 2024-09-10T10:27:10.657 INFO:teuthology.orchestra.run.smithi120.stdout:55834574856 2024-09-10T10:27:10.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-09-10T10:27:10.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f931c06beb0 msgr2=0x7f931c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7f9335349640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f931c06beb0 0x7f931c06e370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f932800f560 tx=0x7f933006e030 comp rx=0 tx=0).stop 2024-09-10T10:27:10.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280491c0 msgr2=0x7f9328053200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7f9335349640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280491c0 0x7f9328053200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f932000e9b0 tx=0x7f932000ee70 comp rx=0 tx=0).stop 2024-09-10T10:27:10.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 shutdown_connections 2024-09-10T10:27:10.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7f9335349640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f931c06beb0 0x7f931c06e370 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7f9335349640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280d8870 0x7f9328177f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7f9335349640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93280502d0 0x7f9328177a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7f9335349640 1 --2- 172.21.15.120:0/1648158484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280491c0 0x7f9328053200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.656+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 >> 172.21.15.120:0/1648158484 conn(0x7f9328054450 msgr2=0x7f9328005c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.657+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 shutdown_connections 2024-09-10T10:27:10.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.657+0000 7f9335349640 1 -- 172.21.15.120:0/1648158484 wait complete. 2024-09-10T10:27:10.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff2f006bf00 msgr2=0x7ff2f006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff2f006bf00 0x7ff2f006e3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff304001050 tx=0x7ff304009040 comp rx=0 tx=0).stop 2024-09-10T10:27:10.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3080cf9e0 msgr2=0x7ff30804b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3080cf9e0 0x7ff30804b220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff30000d7a0 tx=0x7ff30000dba0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 shutdown_connections 2024-09-10T10:27:10.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff2f006bf00 0x7ff2f006e3c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff30804b760 0x7ff3081753f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3080cf9e0 0x7ff30804b220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 --2- 172.21.15.120:0/3715553897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f8005210 0x7ff308049b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 >> 172.21.15.120:0/3715553897 conn(0x7ff30800dbc0 msgr2=0x7ff308051ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 shutdown_connections 2024-09-10T10:27:10.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.658+0000 7ff31490a640 1 -- 172.21.15.120:0/3715553897 wait complete. 2024-09-10T10:27:10.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.663+0000 7f8ad0b78640 1 -- 172.21.15.120:0/623861662 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8ab0005580 con 0x7f8acc0c89b0 2024-09-10T10:27:10.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.663+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fdc50036f00 con 0x7fdc500dce30 2024-09-10T10:27:10.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.664+0000 7fdc46ffd640 1 -- 172.21.15.120:0/3598427810 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc48059870 con 0x7fdc500dce30 2024-09-10T10:27:10.668 INFO:teuthology.orchestra.run.smithi120.stdout:55834574857 2024-09-10T10:27:10.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.664+0000 7f8abaffd640 1 -- 172.21.15.120:0/623861662 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8abc05c7d0 con 0x7f8acc0c89b0 2024-09-10T10:27:10.669 INFO:teuthology.orchestra.run.smithi120.stdout:55834574856 2024-09-10T10:27:10.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 -- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8aac06bf00 msgr2=0x7f8aac06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8aac06bf00 0x7f8aac06e3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8acc156020 tx=0x7f8ac00091c0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 -- 172.21.15.120:0/623861662 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8acc0c89b0 msgr2=0x7f8acc164c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8acc0c89b0 0x7f8acc164c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00c390 tx=0x7f8abc00c850 comp rx=0 tx=0).stop 2024-09-10T10:27:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 -- 172.21.15.120:0/623861662 shutdown_connections 2024-09-10T10:27:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc3806bf00 msgr2=0x7fdc3806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7fdc5ca1e640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc3806bf00 0x7fdc3806e3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009d80 tx=0x7fdc4c009110 comp rx=0 tx=0).stop 2024-09-10T10:27:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f8aac06bf00 0x7f8aac06e3c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8acc162bf0 0x7f8acc163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8acc1592b0 0x7f8acc165a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 --2- 172.21.15.120:0/623861662 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8acc0c89b0 0x7f8acc164c20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500dce30 msgr2=0x7fdc500cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7fdc5ca1e640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500dce30 0x7fdc500cb620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800b3e0 tx=0x7fdc4800b8a0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 -- 172.21.15.120:0/623861662 >> 172.21.15.120:0/623861662 conn(0x7f8acc067d40 msgr2=0x7f8acc0c9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 shutdown_connections 2024-09-10T10:27:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 -- 172.21.15.120:0/623861662 shutdown_connections 2024-09-10T10:27:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7fdc5ca1e640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc3806bf00 0x7fdc3806e3c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7fdc5ca1e640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc500e67f0 0x7fdc50034250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7fdc5ca1e640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc500dce30 0x7fdc500cb620 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7fdc5ca1e640 1 --2- 172.21.15.120:0/3598427810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc50046230 0x7fdc500d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 >> 172.21.15.120:0/3598427810 conn(0x7fdc5004d830 msgr2=0x7fdc500da9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.666+0000 7f8ad0b78640 1 -- 172.21.15.120:0/623861662 wait complete. 2024-09-10T10:27:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.667+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 shutdown_connections 2024-09-10T10:27:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:10.667+0000 7fdc5ca1e640 1 -- 172.21.15.120:0/3598427810 wait complete. 2024-09-10T10:27:10.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-09-10T10:27:10.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-09-10T10:27:10.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-09-10T10:27:10.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-09-10T10:27:11.274 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:27:11.276 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:27:11.596 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T10:27:11.637 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T10:27:11.654 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T10:27:11.658 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T10:27:11.675 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T10:27:11.675 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T10:27:11.681 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T10:27:11.681 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T10:27:11.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.713+0000 7f14bffff640 1 Processor -- start 2024-09-10T10:27:11.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.713+0000 7f14bffff640 1 -- start start 2024-09-10T10:27:11.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.714+0000 7f14bffff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f14c0154c90 0x7f14c00ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.714+0000 7f14bffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c00cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.714+0000 7f14bffff640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c00cc930 con 0x7f14c006d950 2024-09-10T10:27:11.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.714+0000 7f14bffff640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f14c015ab60 con 0x7f14c0154c90 2024-09-10T10:27:11.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.714+0000 7f14bffff640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f14c015ae40 con 0x7f14c00b3020 2024-09-10T10:27:11.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.715+0000 7f14beffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f14c0154c90 0x7f14c00ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.715+0000 7f14be7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c00cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.715+0000 7f14beffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f14c0154c90 0x7f14c00ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34600/0 (socket says 172.21.15.120:34600) 2024-09-10T10:27:11.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.715+0000 7f14beffd640 1 -- 172.21.15.120:0/2995602825 learned_addr learned my addr 172.21.15.120:0/2995602825 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.716+0000 7f14be7fc640 1 -- 172.21.15.120:0/2995602825 >> v2:172.21.15.120:3300/0 conn(0x7f14c0154c90 msgr2=0x7f14c00ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.716+0000 7f14be7fc640 1 --2- 172.21.15.120:0/2995602825 >> v2:172.21.15.120:3300/0 conn(0x7f14c0154c90 0x7f14c00ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.716+0000 7f14be7fc640 1 -- 172.21.15.120:0/2995602825 >> v1:172.21.15.134:6789/0 conn(0x7f14c006d950 legacy=0x7f14c015de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.716+0000 7f14be7fc640 1 -- 172.21.15.120:0/2995602825 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c015b120 con 0x7f14c00b3020 2024-09-10T10:27:11.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.717+0000 7f14be7fc640 1 --2- 172.21.15.120:0/2995602825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c00cd190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f14b00097a0 tx=0x7f14b002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3ceeb137c3ed8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.717+0000 7f14bdffb640 1 -- 172.21.15.120:0/2995602825 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0037070 con 0x7f14c00b3020 2024-09-10T10:27:11.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.717+0000 7f14bdffb640 1 -- 172.21.15.120:0/2995602825 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b0032440 con 0x7f14c00b3020 2024-09-10T10:27:11.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.717+0000 7f14bdffb640 1 -- 172.21.15.120:0/2995602825 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0032810 con 0x7f14c00b3020 2024-09-10T10:27:11.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.719+0000 7f14bffff640 1 -- 172.21.15.120:0/2995602825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 msgr2=0x7f14c00cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.719+0000 7f14bffff640 1 --2- 172.21.15.120:0/2995602825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c00cd190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f14b00097a0 tx=0x7f14b002a210 comp rx=0 tx=0).stop 2024-09-10T10:27:11.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.719+0000 7f14bffff640 1 -- 172.21.15.120:0/2995602825 shutdown_connections 2024-09-10T10:27:11.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.719+0000 7f14bffff640 1 --2- 172.21.15.120:0/2995602825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c00cd190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.719+0000 7f14bffff640 1 --2- 172.21.15.120:0/2995602825 >> v2:172.21.15.120:3300/0 conn(0x7f14c0154c90 0x7f14c00ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.719+0000 7f14bffff640 1 -- 172.21.15.120:0/2995602825 >> 172.21.15.120:0/2995602825 conn(0x7f14c0067950 msgr2=0x7f14c0067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:11.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.719+0000 7f14bffff640 1 -- 172.21.15.120:0/2995602825 shutdown_connections 2024-09-10T10:27:11.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.720+0000 7f14bffff640 1 -- 172.21.15.120:0/2995602825 wait complete. 2024-09-10T10:27:11.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.721+0000 7f14bffff640 1 Processor -- start 2024-09-10T10:27:11.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.722+0000 7f14bffff640 1 -- start start 2024-09-10T10:27:11.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.723+0000 7f14bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c006d950 0x7f14c0152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.723+0000 7f14bffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c01510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.723+0000 7f14bffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14c0154c90 0x7f14c00ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.723+0000 7f14bffff640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00cc4f0 con 0x7f14c006d950 2024-09-10T10:27:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.723+0000 7f14bffff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14c015a830 con 0x7f14c0154c90 2024-09-10T10:27:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.723+0000 7f14bffff640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f14c00cc670 con 0x7f14c00b3020 2024-09-10T10:27:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.724+0000 7f14be7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c01510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.724+0000 7f14be7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c01510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:43070/0 (socket says 172.21.15.120:43070) 2024-09-10T10:27:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.724+0000 7f14be7fc640 1 -- 172.21.15.120:0/946649861 learned_addr learned my addr 172.21.15.120:0/946649861 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.724+0000 7f14beffd640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c006d950 0x7f14c0152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.724+0000 7f14be7fc640 1 -- 172.21.15.120:0/946649861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14c0154c90 msgr2=0x7f14c00ce220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:27:11.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.724+0000 7f14be7fc640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14c0154c90 0x7f14c00ce220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.724+0000 7f14be7fc640 1 -- 172.21.15.120:0/946649861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c006d950 msgr2=0x7f14c0152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.724+0000 7f14be7fc640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c006d950 0x7f14c0152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.724+0000 7f14be7fc640 1 -- 172.21.15.120:0/946649861 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c00d09e0 con 0x7f14c00b3020 2024-09-10T10:27:11.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.725+0000 7f14be7fc640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c01510a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f14b002a800 tx=0x7f14b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.725+0000 7f14a7fff640 1 -- 172.21.15.120:0/946649861 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0037050 con 0x7f14c00b3020 2024-09-10T10:27:11.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.725+0000 7f14a7fff640 1 -- 172.21.15.120:0/946649861 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b0043070 con 0x7f14c00b3020 2024-09-10T10:27:11.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.725+0000 7f14a7fff640 1 -- 172.21.15.120:0/946649861 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b003cb10 con 0x7f14c00b3020 2024-09-10T10:27:11.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.726+0000 7f14bffff640 1 -- 172.21.15.120:0/946649861 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c01f14a0 con 0x7f14c00b3020 2024-09-10T10:27:11.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.727+0000 7f14bffff640 1 -- 172.21.15.120:0/946649861 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c01f09c0 con 0x7f14c00b3020 2024-09-10T10:27:11.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.728+0000 7f14bffff640 1 -- 172.21.15.120:0/946649861 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c00c53c0 con 0x7f14c00b3020 2024-09-10T10:27:11.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.730+0000 7f14a7fff640 1 -- 172.21.15.120:0/946649861 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14b003ccb0 con 0x7f14c00b3020 2024-09-10T10:27:11.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.730+0000 7f14a7fff640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14ac06bf30 0x7f14ac06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.731+0000 7f14a7fff640 1 -- 172.21.15.120:0/946649861 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f14b00abe20 con 0x7f14c00b3020 2024-09-10T10:27:11.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.731+0000 7f14beffd640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14ac06bf30 0x7f14ac06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.732+0000 7f14beffd640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14ac06bf30 0x7f14ac06e3f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009b30 tx=0x7f14b4009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.735+0000 7f14a7fff640 1 -- 172.21.15.120:0/946649861 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f14b007a470 con 0x7f14c00b3020 2024-09-10T10:27:11.765 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-09-10T10:27:11.763+0000 7f890ecec640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-10T10:27:11.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.783+0000 7f9ac0d57640 1 Processor -- start 2024-09-10T10:27:11.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.783+0000 7f9ac0d57640 1 -- start start 2024-09-10T10:27:11.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.784+0000 7f9ac0d57640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9abc150e30 0x7f9abc0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.784+0000 7f9ac0d57640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9abc0c6990 0x7f9abc0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.784+0000 7f9ac0d57640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9abc0c79e0 con 0x7f9abc150a50 2024-09-10T10:27:11.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.784+0000 7f9ac0d57640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9abc0c7290 con 0x7f9abc150e30 2024-09-10T10:27:11.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.784+0000 7f9ac0d57640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9abc0c1af0 con 0x7f9abc0c6990 2024-09-10T10:27:11.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.785+0000 7f9abaffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9abc150e30 0x7f9abc0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.785+0000 7f9abaffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9abc150e30 0x7f9abc0c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34614/0 (socket says 172.21.15.120:34614) 2024-09-10T10:27:11.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.785+0000 7f9abaffd640 1 -- 172.21.15.120:0/1298625220 learned_addr learned my addr 172.21.15.120:0/1298625220 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.786+0000 7f9abaffd640 1 -- 172.21.15.120:0/1298625220 >> v2:172.21.15.134:3300/0 conn(0x7f9abc0c6990 msgr2=0x7f9abc0c9420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:27:11.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.786+0000 7f9abaffd640 1 --2- 172.21.15.120:0/1298625220 >> v2:172.21.15.134:3300/0 conn(0x7f9abc0c6990 0x7f9abc0c9420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.786+0000 7f9abaffd640 1 -- 172.21.15.120:0/1298625220 >> v1:172.21.15.134:6789/0 conn(0x7f9abc150a50 legacy=0x7f9abc1568b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:27:11.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.786+0000 7f9abaffd640 1 -- 172.21.15.120:0/1298625220 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc0c2590 con 0x7f9abc150e30 2024-09-10T10:27:11.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.787+0000 7f9abaffd640 1 --2- 172.21.15.120:0/1298625220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150e30 0x7f9abc0c8ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0740a0 tx=0x7f9ab402be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b37acca3b85aabc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.788+0000 7f9aba7fc640 1 -- 172.21.15.120:0/1298625220 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab402c780 con 0x7f9abc150e30 2024-09-10T10:27:11.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.788+0000 7f9aba7fc640 1 -- 172.21.15.120:0/1298625220 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab402c920 con 0x7f9abc150e30 2024-09-10T10:27:11.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.789+0000 7f9ac0d57640 1 -- 172.21.15.120:0/1298625220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150e30 msgr2=0x7f9abc0c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.789+0000 7f9ac0d57640 1 --2- 172.21.15.120:0/1298625220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150e30 0x7f9abc0c8ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0740a0 tx=0x7f9ab402be80 comp rx=0 tx=0).stop 2024-09-10T10:27:11.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.789+0000 7f9ac0d57640 1 -- 172.21.15.120:0/1298625220 shutdown_connections 2024-09-10T10:27:11.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.789+0000 7f9ac0d57640 1 --2- 172.21.15.120:0/1298625220 >> v2:172.21.15.134:3300/0 conn(0x7f9abc0c6990 0x7f9abc0c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.789+0000 7f9ac0d57640 1 --2- 172.21.15.120:0/1298625220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150e30 0x7f9abc0c8ee0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.789+0000 7f9ac0d57640 1 -- 172.21.15.120:0/1298625220 >> 172.21.15.120:0/1298625220 conn(0x7f9abc067950 msgr2=0x7f9abc067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:11.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.790+0000 7f9ac0d57640 1 -- 172.21.15.120:0/1298625220 shutdown_connections 2024-09-10T10:27:11.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.790+0000 7f9ac0d57640 1 -- 172.21.15.120:0/1298625220 wait complete. 2024-09-10T10:27:11.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.791+0000 7f9ac0d57640 1 Processor -- start 2024-09-10T10:27:11.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.792+0000 7f9ac0d57640 1 -- start start 2024-09-10T10:27:11.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.793+0000 7f9ac0d57640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9abc0c6990 0x7f9abc0c3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.793+0000 7f9ac0d57640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150a50 0x7f9abc0c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.794+0000 7f9abaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150a50 0x7f9abc0c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.794+0000 7f9abaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150a50 0x7f9abc0c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34622/0 (socket says 172.21.15.120:34622) 2024-09-10T10:27:11.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.794+0000 7f9abaffd640 1 -- 172.21.15.120:0/408497353 learned_addr learned my addr 172.21.15.120:0/408497353 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.794+0000 7f9ac0d57640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9abc14cf10 0x7f9abc0c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.794+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc0c2070 con 0x7f9abc150a50 2024-09-10T10:27:11.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.794+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc0c1d90 con 0x7f9abc14cf10 2024-09-10T10:27:11.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.794+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9abc0c7bf0 con 0x7f9abc0c6990 2024-09-10T10:27:11.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.795+0000 7f9abaffd640 1 -- 172.21.15.120:0/408497353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9abc0c6990 msgr2=0x7f9abc0c3760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:27:11.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.795+0000 7f9abaffd640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9abc0c6990 0x7f9abc0c3760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.795+0000 7f9abaffd640 1 -- 172.21.15.120:0/408497353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9abc14cf10 msgr2=0x7f9abc0c5f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:27:11.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.795+0000 7f9abaffd640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9abc14cf10 0x7f9abc0c5f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.795+0000 7f9abaffd640 1 -- 172.21.15.120:0/408497353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc0c4ca0 con 0x7f9abc150a50 2024-09-10T10:27:11.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.796+0000 7f9abaffd640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150a50 0x7f9abc0c5a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400d3c0 tx=0x7f9ab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.796+0000 7f9aa3fff640 1 -- 172.21.15.120:0/408497353 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4042020 con 0x7f9abc150a50 2024-09-10T10:27:11.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.796+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc1f54c0 con 0x7f9abc150a50 2024-09-10T10:27:11.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.799+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc0c5220 con 0x7f9abc150a50 2024-09-10T10:27:11.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.800+0000 7f9aa3fff640 1 -- 172.21.15.120:0/408497353 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab403d440 con 0x7f9abc150a50 2024-09-10T10:27:11.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.800+0000 7f9aa3fff640 1 -- 172.21.15.120:0/408497353 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab402c780 con 0x7f9abc150a50 2024-09-10T10:27:11.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.800+0000 7f9aa3fff640 1 -- 172.21.15.120:0/408497353 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ab4045020 con 0x7f9abc150a50 2024-09-10T10:27:11.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.801+0000 7f9aa3fff640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aa806bf30 0x7f9aa806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.801+0000 7f9aa3fff640 1 -- 172.21.15.120:0/408497353 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9ab40b1060 con 0x7f9abc150a50 2024-09-10T10:27:11.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.802+0000 7f9abb7fe640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aa806bf30 0x7f9aa806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.803+0000 7f9abb7fe640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aa806bf30 0x7f9aa806e3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0096c0 tx=0x7f9aac006bd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.803+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a9c005290 con 0x7f9abc150a50 2024-09-10T10:27:11.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.813+0000 7f9aa3fff640 1 -- 172.21.15.120:0/408497353 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9ab407ad20 con 0x7f9abc150a50 2024-09-10T10:27:11.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.848+0000 7f3ed68e4640 1 Processor -- start 2024-09-10T10:27:11.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.848+0000 7f998a5a6640 1 Processor -- start 2024-09-10T10:27:11.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.848+0000 7f3ed68e4640 1 -- start start 2024-09-10T10:27:11.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.849+0000 7f3ed68e4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3ec80d5850 0x7f3ec8007730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.849+0000 7f3b3f804640 1 Processor -- start 2024-09-10T10:27:11.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.848+0000 7f998a5a6640 1 -- start start 2024-09-10T10:27:11.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.849+0000 7f3ed68e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec804d310 0x7f3ec8045d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.849+0000 7f3ed68e4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ec80dc3a0 con 0x7f3ec8047a80 2024-09-10T10:27:11.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.849+0000 7f3ed68e4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec80dbe90 con 0x7f3ec80d5850 2024-09-10T10:27:11.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.849+0000 7f3ed68e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec80dc0c0 con 0x7f3ec804d310 2024-09-10T10:27:11.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f998a5a6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99841591d0 0x7f998415f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f3ed4e81640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f3ec8047a80 0x7f3ec80dea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:35856/0 (socket says 172.21.15.120:35856) 2024-09-10T10:27:11.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f3ed4e81640 1 -- 172.21.15.120:0/302187300 learned_addr learned my addr 172.21.15.120:0/302187300 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:11.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f998a5a6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99840c75b0 0x7f99840c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f998a5a6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9984072140 con 0x7f99840c8a50 2024-09-10T10:27:11.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f3ecffff640 1 --2- 172.21.15.120:0/302187300 >> v2:172.21.15.134:3300/0 conn(0x7f3ec80d5850 0x7f3ec8007730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f998a5a6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f99841595b0 con 0x7f99841591d0 2024-09-10T10:27:11.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f998a5a6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9984072d10 con 0x7f99840c75b0 2024-09-10T10:27:11.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f9983fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99841591d0 0x7f998415f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f9983fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99841591d0 0x7f998415f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:42036/0 (socket says 172.21.15.120:42036) 2024-09-10T10:27:11.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f9983fff640 1 -- 172.21.15.120:0/3063558941 learned_addr learned my addr 172.21.15.120:0/3063558941 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1626911700 0 0) 0x7f3ec80dc3a0 con 0x7f3ec8047a80 2024-09-10T10:27:11.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ec0005730 con 0x7f3ec8047a80 2024-09-10T10:27:11.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3ecf7fe640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec804d310 0x7f3ec8045d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 510826813 0 0) 0x7f3ec0005730 con 0x7f3ec8047a80 2024-09-10T10:27:11.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f9988b43640 1 --2- 172.21.15.120:0/3063558941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99840c75b0 0x7f99840c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f9988b43640 1 -- 172.21.15.120:0/3063558941 >> v2:172.21.15.134:3300/0 conn(0x7f99841591d0 msgr2=0x7f998415f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f9988b43640 1 --2- 172.21.15.120:0/3063558941 >> v2:172.21.15.134:3300/0 conn(0x7f99841591d0 0x7f998415f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f9988b43640 1 -- 172.21.15.120:0/3063558941 >> v1:172.21.15.120:6789/0 conn(0x7f99840c8a50 legacy=0x7f99840731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f9988b43640 1 -- 172.21.15.120:0/3063558941 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998415fe00 con 0x7f99840c75b0 2024-09-10T10:27:11.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec804d310 msgr2=0x7f3ec8045d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.849+0000 7fa9bdbf8640 1 Processor -- start 2024-09-10T10:27:11.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7f3b3f804640 1 -- start start 2024-09-10T10:27:11.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3b3f804640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3b380d0570 0x7f3b3815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3b3f804640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b3815b290 con 0x7f3b380c7770 2024-09-10T10:27:11.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3b3f804640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b3815b570 con 0x7f3b380d0950 2024-09-10T10:27:11.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3b3f804640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3b380cbc30 con 0x7f3b380d0570 2024-09-10T10:27:11.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec804d310 0x7f3ec8045d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3b3dda1640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f3b380d0950 0x7f3b380cb520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:35860/0 (socket says 172.21.15.120:35860) 2024-09-10T10:27:11.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3b3dda1640 1 -- 172.21.15.120:0/1560291131 learned_addr learned my addr 172.21.15.120:0/1560291131 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:11.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3b3d5a0640 1 --2- 172.21.15.120:0/1560291131 >> v2:172.21.15.134:3300/0 conn(0x7f3b380d0570 0x7f3b3815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 >> v2:172.21.15.134:3300/0 conn(0x7f3ec80d5850 msgr2=0x7f3ec8007730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 --2- 172.21.15.120:0/302187300 >> v2:172.21.15.134:3300/0 conn(0x7f3ec80d5850 0x7f3ec8007730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec8046cc0 con 0x7f3ec8047a80 2024-09-10T10:27:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f3ebc001670 con 0x7f3ec8047a80 2024-09-10T10:27:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 >> v1:172.21.15.134:6789/0 conn(0x7f3ec8047a80 legacy=0x7f3ec80dea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0005bc0 0x7f3ec0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ec0006a10 0x7f3ec0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec00074f0 0x7f3ec000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80dc0c0 con 0x7f3ec0005bc0 2024-09-10T10:27:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec000e350 con 0x7f3ec0006a10 2024-09-10T10:27:11.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f3b2ffff640 1 -- 172.21.15.120:0/1560291131 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1107318522 0 0) 0x7f3b3815b570 con 0x7f3b380d0950 2024-09-10T10:27:11.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec000e550 con 0x7f3ec00074f0 2024-09-10T10:27:11.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7f3ecf7fe640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec804d310 0x7f3ec8045d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:11.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f3b2ffff640 1 -- 172.21.15.120:0/1560291131 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b24003520 con 0x7f3b380d0950 2024-09-10T10:27:11.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f1671ae7640 1 Processor -- start 2024-09-10T10:27:11.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f3ecf7fe640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ec0006a10 0x7f3ec0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f128152f640 1 Processor -- start 2024-09-10T10:27:11.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f3b3d5a0640 1 -- 172.21.15.120:0/1560291131 >> v1:172.21.15.134:6789/0 conn(0x7f3b380d0950 legacy=0x7f3b380cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f3b3d5a0640 1 -- 172.21.15.120:0/1560291131 >> v1:172.21.15.120:6789/0 conn(0x7f3b380c7770 legacy=0x7f3b3815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f3b3d5a0640 1 -- 172.21.15.120:0/1560291131 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3815b850 con 0x7f3b380d0570 2024-09-10T10:27:11.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.849+0000 7fa9bdbf8640 1 -- start start 2024-09-10T10:27:11.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7fa9bdbf8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa9b00f66c0 0x7fa9b0005a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7fa9bdbf8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 0x7fa9b00463c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7fa9bdbf8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b0046900 con 0x7fa9b00ff860 2024-09-10T10:27:11.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7fa9bdbf8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b0045690 con 0x7fa9b00f66c0 2024-09-10T10:27:11.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.850+0000 7fa9bdbf8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b0045f30 con 0x7fa9b004aa70 2024-09-10T10:27:11.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7fa9b77fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa9b00f66c0 0x7fa9b0005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7fa9b6ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 0x7fa9b00463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7fa9b6ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 0x7fa9b00463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:43108/0 (socket says 172.21.15.120:43108) 2024-09-10T10:27:11.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7fa9b6ffd640 1 -- 172.21.15.120:0/1608681915 learned_addr learned my addr 172.21.15.120:0/1608681915 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f3ecffff640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0005bc0 0x7f3ec0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7fa9b6ffd640 1 -- 172.21.15.120:0/1608681915 >> v2:172.21.15.134:3300/0 conn(0x7fa9b00f66c0 msgr2=0x7fa9b0005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7fa9b6ffd640 1 --2- 172.21.15.120:0/1608681915 >> v2:172.21.15.134:3300/0 conn(0x7fa9b00f66c0 0x7fa9b0005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7fa9b6ffd640 1 -- 172.21.15.120:0/1608681915 >> v1:172.21.15.120:6789/0 conn(0x7fa9b00ff860 legacy=0x7fa9b0044b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f1671ae7640 1 -- start start 2024-09-10T10:27:11.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f3ed4e81640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec00074f0 0x7f3ec000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f3ed4e81640 1 -- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ec0006a10 msgr2=0x7f3ec0006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f3ed4e81640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ec0006a10 0x7f3ec0006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f3ed4e81640 1 -- 172.21.15.120:0/302187300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0005bc0 msgr2=0x7f3ec0005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f3ed4e81640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0005bc0 0x7f3ec0005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f3ed4e81640 1 -- 172.21.15.120:0/302187300 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec8046cc0 con 0x7f3ec00074f0 2024-09-10T10:27:11.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f3ecffff640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0005bc0 0x7f3ec0005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:11.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.851+0000 7fa9b6ffd640 1 -- 172.21.15.120:0/1608681915 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b0100310 con 0x7fa9b004aa70 2024-09-10T10:27:11.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7fa9b6ffd640 1 --2- 172.21.15.120:0/1608681915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 0x7fa9b00463c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40088f0 tx=0x7fa9a4032ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28f27d10ec188737 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.852+0000 7f128152f640 1 -- start start 2024-09-10T10:27:11.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f3b3d5a0640 1 --2- 172.21.15.120:0/1560291131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b380d0570 0x7f3b3815ab70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b340097a0 tx=0x7f3b3402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e313a608b2eb31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f9988b43640 1 --2- 172.21.15.120:0/3063558941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99840c75b0 0x7f99840c7ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9984159730 tx=0x7f997c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56549e6a4187a970 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7fa9b67fc640 1 -- 172.21.15.120:0/1608681915 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a403a070 con 0x7fa9b004aa70 2024-09-10T10:27:11.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7fa9b67fc640 1 -- 172.21.15.120:0/1608681915 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a400ae50 con 0x7fa9b004aa70 2024-09-10T10:27:11.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7fa9b67fc640 1 -- 172.21.15.120:0/1608681915 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4036ac0 con 0x7fa9b004aa70 2024-09-10T10:27:11.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f3b2ffff640 1 -- 172.21.15.120:0/1560291131 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34037070 con 0x7f3b380d0570 2024-09-10T10:27:11.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f9982ffd640 1 -- 172.21.15.120:0/3063558941 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c007740 con 0x7f99840c75b0 2024-09-10T10:27:11.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f9982ffd640 1 -- 172.21.15.120:0/3063558941 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f997c0078e0 con 0x7f99840c75b0 2024-09-10T10:27:11.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f3b2ffff640 1 -- 172.21.15.120:0/1560291131 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b34032440 con 0x7f3b380d0570 2024-09-10T10:27:11.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f1671ae7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1664044b30 0x7f1664034ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f1671ae7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16640dcc30 con 0x7f16640d62e0 2024-09-10T10:27:11.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.853+0000 7f9982ffd640 1 -- 172.21.15.120:0/3063558941 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c007cb0 con 0x7f99840c75b0 2024-09-10T10:27:11.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f128152f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1274045f80 0x7f12740e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f128152f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1274005ba0 con 0x7f12740d5590 2024-09-10T10:27:11.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f128152f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12740350e0 con 0x7f12740467b0 2024-09-10T10:27:11.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f128152f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1274034e90 con 0x7f1274045f80 2024-09-10T10:27:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f1671ae7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16640dcf10 con 0x7f16640d66c0 2024-09-10T10:27:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f1671ae7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f166404b200 con 0x7f1664044b30 2024-09-10T10:27:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/1608681915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 msgr2=0x7fa9b00463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7fa9bdbf8640 1 --2- 172.21.15.120:0/1608681915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 0x7fa9b00463c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40088f0 tx=0x7fa9a4032ae0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f127b7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f12740467b0 0x7f1274045870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:35884/0 (socket says 172.21.15.120:35884) 2024-09-10T10:27:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f127b7fe640 1 -- 172.21.15.120:0/3423935064 learned_addr learned my addr 172.21.15.120:0/3423935064 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f166bfff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f16640d66c0 0x7f166404bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:35868/0 (socket says 172.21.15.120:35868) 2024-09-10T10:27:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f166bfff640 1 -- 172.21.15.120:0/1724502475 learned_addr learned my addr 172.21.15.120:0/1724502475 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/1608681915 shutdown_connections 2024-09-10T10:27:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7fa9bdbf8640 1 --2- 172.21.15.120:0/1608681915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 0x7fa9b00463c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7fa9bdbf8640 1 --2- 172.21.15.120:0/1608681915 >> v2:172.21.15.134:3300/0 conn(0x7fa9b00f66c0 0x7fa9b0005a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/1608681915 >> 172.21.15.120:0/1608681915 conn(0x7fa9b004fc80 msgr2=0x7fa9b0050090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f3b2ffff640 1 -- 172.21.15.120:0/1560291131 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34032810 con 0x7f3b380d0570 2024-09-10T10:27:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f3ed4e81640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec00074f0 0x7f3ec000dbd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8047ed0 tx=0x7f3ec0007a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b68203efc99bbbc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f3b3f804640 1 -- 172.21.15.120:0/1560291131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b380d0570 msgr2=0x7f3b3815ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f3b3f804640 1 --2- 172.21.15.120:0/1560291131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b380d0570 0x7f3b3815ab70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b340097a0 tx=0x7f3b3402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f3b3f804640 1 -- 172.21.15.120:0/1560291131 shutdown_connections 2024-09-10T10:27:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f3b3f804640 1 --2- 172.21.15.120:0/1560291131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b380d0570 0x7f3b3815ab70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f3b3f804640 1 -- 172.21.15.120:0/1560291131 >> 172.21.15.120:0/1560291131 conn(0x7f3b3806ca90 msgr2=0x7f3b3806e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1415401718 0 0) 0x7f12740350e0 con 0x7f12740467b0 2024-09-10T10:27:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f126c005730 con 0x7f12740467b0 2024-09-10T10:27:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614480976 0 0) 0x7f1274005ba0 con 0x7f12740d5590 2024-09-10T10:27:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12740350e0 con 0x7f12740d5590 2024-09-10T10:27:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.854+0000 7f1280d2e640 1 --2- 172.21.15.120:0/3423935064 >> v2:172.21.15.120:3300/0 conn(0x7f1274045f80 0x7f12740e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f998a5a6640 1 -- 172.21.15.120:0/3063558941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99840c75b0 msgr2=0x7f99840c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f998a5a6640 1 --2- 172.21.15.120:0/3063558941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99840c75b0 0x7f99840c7ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9984159730 tx=0x7f997c02a6d0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc036070 con 0x7f3ec00074f0 2024-09-10T10:27:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ebc005ad0 con 0x7f3ec00074f0 2024-09-10T10:27:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f3eceffd640 1 -- 172.21.15.120:0/302187300 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc007dd0 con 0x7f3ec00074f0 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3223316521 0 0) 0x7f126c005730 con 0x7f12740467b0 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 >> v2:172.21.15.120:3300/0 conn(0x7f1274045f80 msgr2=0x7f12740e6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 --2- 172.21.15.120:0/3423935064 >> v2:172.21.15.120:3300/0 conn(0x7f1274045f80 0x7f12740e6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 >> v1:172.21.15.120:6789/0 conn(0x7f12740d5590 legacy=0x7f12740db340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12740353c0 con 0x7f12740467b0 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f12680015a0 con 0x7f12740467b0 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 >> v1:172.21.15.134:6789/0 conn(0x7f12740467b0 legacy=0x7f1274045870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f126c006e50 0x7f126c007250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 0x7f126c005e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f126c006370 0x7f126c00bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1274034e90 con 0x7f126c006370 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f126c00c260 con 0x7f126c0059b0 2024-09-10T10:27:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f126c00c460 con 0x7f126c006e50 2024-09-10T10:27:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f1280d2e640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f126c006370 0x7f126c00bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127b7fe640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 0x7f126c005e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f3b3f804640 1 -- 172.21.15.120:0/1560291131 shutdown_connections 2024-09-10T10:27:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220027695 0 0) 0x7f16640dcf10 con 0x7f16640d66c0 2024-09-10T10:27:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f165c003520 con 0x7f16640d66c0 2024-09-10T10:27:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f127bfff640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f126c006e50 0x7f126c007250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2101957212 0 0) 0x7f16640dcc30 con 0x7f16640d62e0 2024-09-10T10:27:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.855+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16640dcf10 con 0x7f16640d62e0 2024-09-10T10:27:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f3ed68e4640 1 -- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec00074f0 msgr2=0x7f3ec000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f3ed68e4640 1 --2- 172.21.15.120:0/302187300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec00074f0 0x7f3ec000dbd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8047ed0 tx=0x7f3ec0007a40 comp rx=0 tx=0).stop 2024-09-10T10:27:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f3ed4e81640 1 -- 172.21.15.120:0/302187300 reap_dead start 2024-09-10T10:27:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f3ed68e4640 1 -- 172.21.15.120:0/302187300 shutdown_connections 2024-09-10T10:27:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f3ed68e4640 1 -- 172.21.15.120:0/302187300 >> 172.21.15.120:0/302187300 conn(0x7f3ec804d830 msgr2=0x7f3ec804dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/1608681915 shutdown_connections 2024-09-10T10:27:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/1608681915 wait complete. 2024-09-10T10:27:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f998a5a6640 1 -- 172.21.15.120:0/3063558941 shutdown_connections 2024-09-10T10:27:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f998a5a6640 1 --2- 172.21.15.120:0/3063558941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99840c75b0 0x7f99840c7ee0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f998a5a6640 1 --2- 172.21.15.120:0/3063558941 >> v2:172.21.15.134:3300/0 conn(0x7f99841591d0 0x7f998415f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f998a5a6640 1 -- 172.21.15.120:0/3063558941 >> 172.21.15.120:0/3063558941 conn(0x7f99840ccbb0 msgr2=0x7f99840ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:11.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f127b7fe640 1 -- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f126c006e50 msgr2=0x7f126c007250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f127b7fe640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f126c006e50 0x7f126c007250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f127b7fe640 1 -- 172.21.15.120:0/3423935064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f126c006370 msgr2=0x7f126c00bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f127b7fe640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f126c006370 0x7f126c00bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f127b7fe640 1 -- 172.21.15.120:0/3423935064 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c005730 con 0x7f126c0059b0 2024-09-10T10:27:11.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f3b3f804640 1 -- 172.21.15.120:0/1560291131 wait complete. 2024-09-10T10:27:11.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f3ed68e4640 1 -- 172.21.15.120:0/302187300 shutdown_connections 2024-09-10T10:27:11.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f16712e6640 1 --2- 172.21.15.120:0/1724502475 >> v2:172.21.15.120:3300/0 conn(0x7f1664044b30 0x7f1664034ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3625885099 0 0) 0x7f165c003520 con 0x7f16640d66c0 2024-09-10T10:27:11.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 >> v2:172.21.15.120:3300/0 conn(0x7f1664044b30 msgr2=0x7f1664034ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f166b7fe640 1 --2- 172.21.15.120:0/1724502475 >> v2:172.21.15.120:3300/0 conn(0x7f1664044b30 0x7f1664034ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 >> v1:172.21.15.120:6789/0 conn(0x7f16640d62e0 legacy=0x7f16640dc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16640dd1f0 con 0x7f16640d66c0 2024-09-10T10:27:11.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f1658001670 con 0x7f16640d66c0 2024-09-10T10:27:11.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f1280d2e640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f126c006370 0x7f126c00bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:11.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f1280d2e640 1 -- 172.21.15.120:0/3423935064 reap_dead start 2024-09-10T10:27:11.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 >> v1:172.21.15.134:6789/0 conn(0x7f16640d66c0 legacy=0x7f166404bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7f166b7fe640 1 --2- 172.21.15.120:0/1724502475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f165c003b40 0x7f165c003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7f166b7fe640 1 --2- 172.21.15.120:0/1724502475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f165c004800 0x7f165c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7f166b7fe640 1 --2- 172.21.15.120:0/1724502475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f165c0053c0 0x7f165c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f166404b200 con 0x7f165c003b40 2024-09-10T10:27:11.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f165c00c280 con 0x7f165c0053c0 2024-09-10T10:27:11.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f165c00c480 con 0x7f165c004800 2024-09-10T10:27:11.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f127bfff640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f126c006e50 0x7f126c007250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:11.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.856+0000 7f3ed68e4640 1 -- 172.21.15.120:0/302187300 wait complete. 2024-09-10T10:27:11.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7f166bfff640 1 --2- 172.21.15.120:0/1724502475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f165c004800 0x7f165c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7f127b7fe640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 0x7f126c005e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1268007ef0 tx=0x7f126c007a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47632d5be2e2806b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7fa9bdbf8640 1 Processor -- start 2024-09-10T10:27:11.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7f1670ae5640 1 --2- 172.21.15.120:0/1724502475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f165c003b40 0x7f165c003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.857+0000 7f3b3f804640 1 Processor -- start 2024-09-10T10:27:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f3b3f804640 1 -- start start 2024-09-10T10:27:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7fa9bdbf8640 1 -- start start 2024-09-10T10:27:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268007b10 con 0x7f126c0059b0 2024-09-10T10:27:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f998a5a6640 1 -- 172.21.15.120:0/3063558941 shutdown_connections 2024-09-10T10:27:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f3ed68e4640 1 Processor -- start 2024-09-10T10:27:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f166bfff640 1 -- 172.21.15.120:0/1724502475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f165c0053c0 msgr2=0x7f165c00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:27:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f127affd640 1 -- 172.21.15.120:0/3423935064 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1268005ad0 con 0x7f126c0059b0 2024-09-10T10:27:11.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f166bfff640 1 --2- 172.21.15.120:0/1724502475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f165c0053c0 0x7f165c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f166bfff640 1 -- 172.21.15.120:0/1724502475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f165c003b40 msgr2=0x7f165c003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:11.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f166bfff640 1 --2- 172.21.15.120:0/1724502475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f165c003b40 0x7f165c003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f166bfff640 1 -- 172.21.15.120:0/1724502475 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16640dd1f0 con 0x7f165c004800 2024-09-10T10:27:11.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f3ed68e4640 1 -- start start 2024-09-10T10:27:11.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7fa9bdbf8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 0x7fa9b00f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f128152f640 1 -- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 msgr2=0x7f126c005e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f128152f640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 0x7f126c005e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1268007ef0 tx=0x7f126c007a40 comp rx=0 tx=0).stop 2024-09-10T10:27:11.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7fa9bdbf8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9b00f66c0 0x7fa9b004e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.858+0000 7f998a5a6640 1 -- 172.21.15.120:0/3063558941 wait complete. 2024-09-10T10:27:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f128152f640 1 -- 172.21.15.120:0/3423935064 shutdown_connections 2024-09-10T10:27:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f128152f640 1 --2- 172.21.15.120:0/3423935064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 0x7f126c005e30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f128152f640 1 -- 172.21.15.120:0/3423935064 >> 172.21.15.120:0/3423935064 conn(0x7f127404fc50 msgr2=0x7f1274050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3b3f804640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b380c7770 0x7f3b38151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7fa9b6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9b00f66c0 0x7fa9b004e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3ed68e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ec00074f0 0x7f3ec80ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7fa9b77fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 0x7fa9b00f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7fa9b6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9b00f66c0 0x7fa9b004e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34674/0 (socket says 172.21.15.120:34674) 2024-09-10T10:27:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3b3f804640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b380d0570 0x7f3b3815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7fa9b6ffd640 1 -- 172.21.15.120:0/2489505050 learned_addr learned my addr 172.21.15.120:0/2489505050 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7fa9bdbf8640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9b00ff860 0x7fa9b004e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3b3f804640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b380d0950 0x7f3b3814c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3b3d5a0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b380c7770 0x7f3b38151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3b3d5a0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b380c7770 0x7f3b38151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:43150/0 (socket says 172.21.15.120:43150) 2024-09-10T10:27:11.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f166bfff640 1 --2- 172.21.15.120:0/1724502475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f165c004800 0x7f165c004c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f165800c290 tx=0x7f165c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb95a4f7acea0e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b00458e0 con 0x7fa9b00f66c0 2024-09-10T10:27:11.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b00f6aa0 con 0x7fa9b00ff860 2024-09-10T10:27:11.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3d5a0640 1 -- 172.21.15.120:0/2579674557 learned_addr learned my addr 172.21.15.120:0/2579674557 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b0045bc0 con 0x7fa9b004aa70 2024-09-10T10:27:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b38165640 con 0x7f3b380d0570 2024-09-10T10:27:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f998a5a6640 1 Processor -- start 2024-09-10T10:27:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b381657c0 con 0x7f3b380d0950 2024-09-10T10:27:11.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3ed68e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec8048130 0x7f3ec80078e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3b38165940 con 0x7f3b380c7770 2024-09-10T10:27:11.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3ed68e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec8173b80 0x7f3ec8170100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3ed68e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8047560 con 0x7f3ec8048130 2024-09-10T10:27:11.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3ed68e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8047280 con 0x7f3ec00074f0 2024-09-10T10:27:11.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f3ed68e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec80468c0 con 0x7f3ec8173b80 2024-09-10T10:27:11.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3ecffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ec00074f0 0x7f3ec80ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3dda1640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b380d0950 0x7f3b3814c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3cd9f640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b380d0570 0x7f3b3815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658007d10 con 0x7f165c004800 2024-09-10T10:27:11.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3ecf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec8048130 0x7f3ec80078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3ecf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec8048130 0x7f3ec80078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34680/0 (socket says 172.21.15.120:34680) 2024-09-10T10:27:11.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1658005ad0 con 0x7f165c004800 2024-09-10T10:27:11.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3ecf7fe640 1 -- 172.21.15.120:0/2873775981 learned_addr learned my addr 172.21.15.120:0/2873775981 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f166b7fe640 1 -- 172.21.15.120:0/1724502475 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658007070 con 0x7f165c004800 2024-09-10T10:27:11.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.859+0000 7f128152f640 1 -- 172.21.15.120:0/3423935064 shutdown_connections 2024-09-10T10:27:11.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3d5a0640 1 -- 172.21.15.120:0/2579674557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b380d0950 msgr2=0x7f3b3814c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3d5a0640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b380d0950 0x7f3b3814c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3d5a0640 1 -- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b380d0570 msgr2=0x7f3b3815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3d5a0640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b380d0570 0x7f3b3815bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3b3d5a0640 1 -- 172.21.15.120:0/2579674557 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b38151f40 con 0x7f3b380c7770 2024-09-10T10:27:11.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3b3cd9f640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b380d0570 0x7f3b3815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:11.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f3ed4e81640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec8173b80 0x7f3ec8170100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3ecf7fe640 1 -- 172.21.15.120:0/2873775981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec8173b80 msgr2=0x7f3ec8170100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3ecf7fe640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec8173b80 0x7f3ec8170100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3ecf7fe640 1 -- 172.21.15.120:0/2873775981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ec00074f0 msgr2=0x7f3ec80ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f16712e6640 1 -- 172.21.15.120:0/1724502475 reap_dead start 2024-09-10T10:27:11.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7fa9b7fff640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9b00ff860 0x7fa9b004e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f998a5a6640 1 -- start start 2024-09-10T10:27:11.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.860+0000 7f128152f640 1 -- 172.21.15.120:0/3423935064 wait complete. 2024-09-10T10:27:11.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3b3d5a0640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b380c7770 0x7f3b38151080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b340098c0 tx=0x7f3b340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3ecf7fe640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ec00074f0 0x7f3ec80ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3ecf7fe640 1 -- 172.21.15.120:0/2873775981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec8174070 con 0x7f3ec8048130 2024-09-10T10:27:11.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f1671ae7640 1 -- 172.21.15.120:0/1724502475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f165c004800 msgr2=0x7f165c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f1671ae7640 1 --2- 172.21.15.120:0/1724502475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f165c004800 0x7f165c004c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f165800c290 tx=0x7f165c005830 comp rx=0 tx=0).stop 2024-09-10T10:27:11.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f998a5a6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99840c75b0 0x7f99841fde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7fa9b6ffd640 1 -- 172.21.15.120:0/2489505050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 msgr2=0x7fa9b00f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3b2dffb640 1 -- 172.21.15.120:0/2579674557 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34042020 con 0x7f3b380c7770 2024-09-10T10:27:11.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7fa9b6ffd640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 0x7fa9b00f19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7fa9b6ffd640 1 -- 172.21.15.120:0/2489505050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9b00ff860 msgr2=0x7fa9b004e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7fa9b6ffd640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9b00ff860 0x7fa9b004e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7fa9b6ffd640 1 -- 172.21.15.120:0/2489505050 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b00f2930 con 0x7fa9b00f66c0 2024-09-10T10:27:11.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f1671ae7640 1 -- 172.21.15.120:0/1724502475 shutdown_connections 2024-09-10T10:27:11.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f1671ae7640 1 --2- 172.21.15.120:0/1724502475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f165c004800 0x7f165c004c80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3b2dffb640 1 -- 172.21.15.120:0/2579674557 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3403d730 con 0x7f3b380c7770 2024-09-10T10:27:11.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f1671ae7640 1 -- 172.21.15.120:0/1724502475 >> 172.21.15.120:0/1724502475 conn(0x7f166404fc50 msgr2=0x7f1664050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:11.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b38152220 con 0x7f3b380c7770 2024-09-10T10:27:11.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3b2dffb640 1 -- 172.21.15.120:0/2579674557 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34037040 con 0x7f3b380c7770 2024-09-10T10:27:11.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f3ecf7fe640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec8048130 0x7f3ec80078e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e5c0 tx=0x7f3ec400ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f1671ae7640 1 -- 172.21.15.120:0/1724502475 shutdown_connections 2024-09-10T10:27:11.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f1671ae7640 1 -- 172.21.15.120:0/1724502475 wait complete. 2024-09-10T10:27:11.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.861+0000 7f998a5a6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99840c8a50 0x7f99841e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f998a5a6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99841591d0 0x7f998406c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b381527e0 con 0x7f3b380c7770 2024-09-10T10:27:11.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f128152f640 1 Processor -- start 2024-09-10T10:27:11.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f3eccff9640 1 -- 172.21.15.120:0/2873775981 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4019020 con 0x7f3ec8048130 2024-09-10T10:27:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f3ed68e4640 1 -- 172.21.15.120:0/2873775981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec80dac30 con 0x7f3ec8048130 2024-09-10T10:27:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f3ed68e4640 1 -- 172.21.15.120:0/2873775981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec80daed0 con 0x7f3ec8048130 2024-09-10T10:27:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f3eccff9640 1 -- 172.21.15.120:0/2873775981 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec4015690 con 0x7f3ec8048130 2024-09-10T10:27:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f998a5a6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99840726c0 con 0x7f99840c8a50 2024-09-10T10:27:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f998a5a6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99840723e0 con 0x7f99840c75b0 2024-09-10T10:27:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f998a5a6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f99840729a0 con 0x7f99841591d0 2024-09-10T10:27:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f9983fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99840c75b0 0x7f99841fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f99837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99840c8a50 0x7f99841e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f99837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99840c8a50 0x7f99841e82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34702/0 (socket says 172.21.15.120:34702) 2024-09-10T10:27:11.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f99837fe640 1 -- 172.21.15.120:0/2885573689 learned_addr learned my addr 172.21.15.120:0/2885573689 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f9988b43640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99841591d0 0x7f998406c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f3eccff9640 1 -- 172.21.15.120:0/2873775981 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4019020 con 0x7f3ec8048130 2024-09-10T10:27:11.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.862+0000 7f128152f640 1 -- start start 2024-09-10T10:27:11.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7fa9b6ffd640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9b00f66c0 0x7fa9b004e020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4000c00 tx=0x7fa9a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f128152f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 0x7f1274048810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3815ab70 con 0x7f3b380c7770 2024-09-10T10:27:11.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f128152f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740497b0 0x7f1274049bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f128152f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1274047100 0x7f1274047580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f128152f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12740d5970 con 0x7f1274047100 2024-09-10T10:27:11.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f128152f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1274035730 con 0x7f126c0059b0 2024-09-10T10:27:11.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f9983fff640 1 -- 172.21.15.120:0/2885573689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99841591d0 msgr2=0x7f998406c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f9983fff640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99841591d0 0x7f998406c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f9983fff640 1 -- 172.21.15.120:0/2885573689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99840c8a50 msgr2=0x7f99841e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.863+0000 7f9983fff640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99840c8a50 0x7f99841e82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f9983fff640 1 -- 172.21.15.120:0/2885573689 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9984157970 con 0x7f99840c75b0 2024-09-10T10:27:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f128152f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f12740335b0 con 0x7f12740497b0 2024-09-10T10:27:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f1671ae7640 1 Processor -- start 2024-09-10T10:27:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f1671ae7640 1 -- start start 2024-09-10T10:27:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f1280d2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1274047100 0x7f1274047580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f127bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 0x7f1274048810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f1280d2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1274047100 0x7f1274047580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34716/0 (socket says 172.21.15.120:34716) 2024-09-10T10:27:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f1280d2e640 1 -- 172.21.15.120:0/3645098095 learned_addr learned my addr 172.21.15.120:0/3645098095 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f9983fff640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99840c75b0 0x7f99841fde10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f997400e8b0 tx=0x7f997400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f127b7fe640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740497b0 0x7f1274049bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f9980ff9640 1 -- 172.21.15.120:0/2885573689 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974017070 con 0x7f99840c75b0 2024-09-10T10:27:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f9980ff9640 1 -- 172.21.15.120:0/2885573689 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f997400c940 con 0x7f99840c75b0 2024-09-10T10:27:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f9980ff9640 1 -- 172.21.15.120:0/2885573689 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997400cd10 con 0x7f99840c75b0 2024-09-10T10:27:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f998a5a6640 1 -- 172.21.15.120:0/2885573689 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9984201510 con 0x7f99840c75b0 2024-09-10T10:27:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.864+0000 7f998a5a6640 1 -- 172.21.15.120:0/2885573689 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9984152a40 con 0x7f99840c75b0 2024-09-10T10:27:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.865+0000 7f3ed68e4640 1 -- 172.21.15.120:0/2873775981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb0005290 con 0x7f3ec8048130 2024-09-10T10:27:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.865+0000 7f1671ae7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f165c004800 0x7f1664175ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.865+0000 7f3b2dffb640 1 -- 172.21.15.120:0/2579674557 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b34045020 con 0x7f3b380c7770 2024-09-10T10:27:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.865+0000 7f1671ae7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16640d8e60 0x7f16640068c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1280d2e640 1 -- 172.21.15.120:0/3645098095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740497b0 msgr2=0x7f1274049bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1280d2e640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740497b0 0x7f1274049bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1280d2e640 1 -- 172.21.15.120:0/3645098095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 msgr2=0x7f1274048810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1280d2e640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 0x7f1274048810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1280d2e640 1 -- 172.21.15.120:0/3645098095 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1274047ac0 con 0x7f1274047100 2024-09-10T10:27:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f3eccff9640 1 -- 172.21.15.120:0/2873775981 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ec401c020 con 0x7f3ec8048130 2024-09-10T10:27:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1670ae5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f165c004800 0x7f1664175ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1670ae5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f165c004800 0x7f1664175ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:42130/0 (socket says 172.21.15.120:42130) 2024-09-10T10:27:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1670ae5640 1 -- 172.21.15.120:0/841034522 learned_addr learned my addr 172.21.15.120:0/841034522 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f166bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16640d8e60 0x7f16640068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1671ae7640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16640d4750 0x7f16640d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1280d2e640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1274047100 0x7f1274047580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f127c06a930 tx=0x7f127c069890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f3b2dffb640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b1c06be30 0x7f3b1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1664035cf0 con 0x7f16640d8e60 2024-09-10T10:27:11.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f166404b730 con 0x7f165c004800 2024-09-10T10:27:11.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1664035e70 con 0x7f16640d4750 2024-09-10T10:27:11.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f3b2dffb640 1 -- 172.21.15.120:0/2579674557 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3b340ab990 con 0x7f3b380c7770 2024-09-10T10:27:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f16712e6640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16640d4750 0x7f16640d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.867+0000 7f9980ff9640 1 -- 172.21.15.120:0/2885573689 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f997401a070 con 0x7f99840c75b0 2024-09-10T10:27:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.866+0000 7f3eccff9640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3eb406c230 0x7f3eb406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.867+0000 7f3eccff9640 1 -- 172.21.15.120:0/2873775981 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3ec400f040 con 0x7f3ec8048130 2024-09-10T10:27:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.867+0000 7f3ecffff640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3eb406c230 0x7f3eb406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.867+0000 7fa9a3fff640 1 -- 172.21.15.120:0/2489505050 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4050070 con 0x7fa9b00f66c0 2024-09-10T10:27:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.867+0000 7f998a5a6640 1 -- 172.21.15.120:0/2885573689 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99840c6c30 con 0x7f99840c75b0 2024-09-10T10:27:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.867+0000 7f3b3cd9f640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b1c06be30 0x7f3b1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.867+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b0187300 con 0x7fa9b00f66c0 2024-09-10T10:27:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.867+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b00f2650 con 0x7fa9b00f66c0 2024-09-10T10:27:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.867+0000 7f9980ff9640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f997006be60 0x7f997006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.867+0000 7f9980ff9640 1 -- 172.21.15.120:0/2885573689 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f997408b0b0 con 0x7f99840c75b0 2024-09-10T10:27:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.868+0000 7f99837fe640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f997006be60 0x7f997006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.868+0000 7fa9a3fff640 1 -- 172.21.15.120:0/2489505050 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a40405a0 con 0x7fa9b00f66c0 2024-09-10T10:27:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.868+0000 7fa9a3fff640 1 -- 172.21.15.120:0/2489505050 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a403a040 con 0x7fa9b00f66c0 2024-09-10T10:27:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.868+0000 7f16712e6640 1 -- 172.21.15.120:0/841034522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f165c004800 msgr2=0x7f1664175ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.868+0000 7f16712e6640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f165c004800 0x7f1664175ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.868+0000 7f16712e6640 1 -- 172.21.15.120:0/841034522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16640d8e60 msgr2=0x7f16640068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.868+0000 7f16712e6640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16640d8e60 0x7f16640068c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.868+0000 7f16712e6640 1 -- 172.21.15.120:0/841034522 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16640d4e50 con 0x7f16640d4750 2024-09-10T10:27:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.868+0000 7f3ecffff640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3eb406c230 0x7f3eb406e6f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ed002dd50 tx=0x7f3ed008f040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.869+0000 7f3b3cd9f640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b1c06be30 0x7f3b1c06e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3b380ce830 tx=0x7f3b30003d90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.869+0000 7f99837fe640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f997006be60 0x7f997006e320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f998406be30 tx=0x7f9978009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.868+0000 7f16712e6640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16640d4750 0x7f16640d7d30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f166c000b60 tx=0x7f166c069890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.869+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa998005290 con 0x7fa9b00f66c0 2024-09-10T10:27:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.871+0000 7f16697fa640 1 -- 172.21.15.120:0/841034522 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c04ffb0 con 0x7f16640d4750 2024-09-10T10:27:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.872+0000 7f16697fa640 1 -- 172.21.15.120:0/841034522 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166c0736b0 con 0x7f16640d4750 2024-09-10T10:27:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.872+0000 7f3b2dffb640 1 -- 172.21.15.120:0/2579674557 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3b3404c070 con 0x7f3b380c7770 2024-09-10T10:27:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.872+0000 7f16697fa640 1 -- 172.21.15.120:0/841034522 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c073aa0 con 0x7f16640d4750 2024-09-10T10:27:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.873+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1664047960 con 0x7f16640d4750 2024-09-10T10:27:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.873+0000 7f1278ff9640 1 -- 172.21.15.120:0/3645098095 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c0a3070 con 0x7f1274047100 2024-09-10T10:27:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.873+0000 7f128152f640 1 -- 172.21.15.120:0/3645098095 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127416fe40 con 0x7f1274047100 2024-09-10T10:27:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.873+0000 7f3eccff9640 1 -- 172.21.15.120:0/2873775981 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3ec4023020 con 0x7f3ec8048130 2024-09-10T10:27:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.873+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1664047ee0 con 0x7f16640d4750 2024-09-10T10:27:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.873+0000 7f128152f640 1 -- 172.21.15.120:0/3645098095 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1274170400 con 0x7f1274047100 2024-09-10T10:27:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.874+0000 7f128152f640 1 -- 172.21.15.120:0/3645098095 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12740463b0 con 0x7f1274047100 2024-09-10T10:27:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.875+0000 7f1278ff9640 1 -- 172.21.15.120:0/3645098095 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c09e5f0 con 0x7f1274047100 2024-09-10T10:27:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.875+0000 7f1278ff9640 1 -- 172.21.15.120:0/3645098095 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c09e9a0 con 0x7f1274047100 2024-09-10T10:27:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.875+0000 7fa9a3fff640 1 -- 172.21.15.120:0/2489505050 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9a4053070 con 0x7fa9b00f66c0 2024-09-10T10:27:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.876+0000 7fa9a3fff640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa99c06be80 0x7fa99c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.876+0000 7fa9a3fff640 1 -- 172.21.15.120:0/2489505050 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa9a40b4330 con 0x7fa9b00f66c0 2024-09-10T10:27:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.876+0000 7fa9b77fe640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa99c06be80 0x7fa99c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.877+0000 7f9980ff9640 1 -- 172.21.15.120:0/2885573689 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9974059030 con 0x7f99840c75b0 2024-09-10T10:27:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.877+0000 7fa9b77fe640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa99c06be80 0x7fa99c06e340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00bba0 tx=0x7fa9ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.875+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16640da030 con 0x7f16640d4750 2024-09-10T10:27:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.877+0000 7f16697fa640 1 -- 172.21.15.120:0/841034522 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f166c073c40 con 0x7f16640d4750 2024-09-10T10:27:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.878+0000 7f16697fa640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f165006bf30 0x7f165006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.878+0000 7f1670ae5640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f165006bf30 0x7f165006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.879+0000 7f16697fa640 1 -- 172.21.15.120:0/841034522 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f166c106f60 con 0x7f16640d4750 2024-09-10T10:27:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.879+0000 7f1670ae5640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f165006bf30 0x7f165006e3f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1660007850 tx=0x7f1660009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.882+0000 7f16697fa640 1 -- 172.21.15.120:0/841034522 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f16640da030 con 0x7f16640d4750 2024-09-10T10:27:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.883+0000 7fa9a3fff640 1 -- 172.21.15.120:0/2489505050 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa9a405a020 con 0x7fa9b00f66c0 2024-09-10T10:27:11.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.886+0000 7f1278ff9640 1 -- 172.21.15.120:0/3645098095 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f127c0a6070 con 0x7f1274047100 2024-09-10T10:27:11.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.887+0000 7f1278ff9640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f126006be30 0x7f126006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:11.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.896+0000 7f127bfff640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f126006be30 0x7f126006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:11.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.896+0000 7f127bfff640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f126006be30 0x7f126006e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1270007fd0 tx=0x7f1270009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:11.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.897+0000 7f1278ff9640 1 -- 172.21.15.120:0/3645098095 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f127c0dc330 con 0x7f1274047100 2024-09-10T10:27:11.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:11.898+0000 7f1278ff9640 1 -- 172.21.15.120:0/3645098095 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1274047ac0 con 0x7f1274047100 2024-09-10T10:27:12.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.097+0000 7f14bffff640 1 -- 172.21.15.120:0/946649861 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f14c00ca670 con 0x7f14c00b3020 2024-09-10T10:27:12.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.098+0000 7f14a7fff640 1 -- 172.21.15.120:0/946649861 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14b0079d70 con 0x7f14c00b3020 2024-09-10T10:27:12.099 INFO:teuthology.orchestra.run.smithi120.stdout:55834574858 2024-09-10T10:27:12.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.101+0000 7f14a5ffb640 1 -- 172.21.15.120:0/946649861 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14ac06bf30 msgr2=0x7f14ac06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.101+0000 7f14a5ffb640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14ac06bf30 0x7f14ac06e3f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009b30 tx=0x7f14b4009240 comp rx=0 tx=0).stop 2024-09-10T10:27:12.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.102+0000 7f14a5ffb640 1 -- 172.21.15.120:0/946649861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 msgr2=0x7f14c01510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.102+0000 7f14a5ffb640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c01510a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f14b002a800 tx=0x7f14b00046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.102+0000 7f14a5ffb640 1 -- 172.21.15.120:0/946649861 shutdown_connections 2024-09-10T10:27:12.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.102+0000 7f14a5ffb640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14ac06bf30 0x7f14ac06e3f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.102+0000 7f14a5ffb640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14c0154c90 0x7f14c00ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.102+0000 7f14a5ffb640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14c00b3020 0x7f14c01510a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.102+0000 7f14a5ffb640 1 --2- 172.21.15.120:0/946649861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c006d950 0x7f14c0152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.102+0000 7f14a5ffb640 1 -- 172.21.15.120:0/946649861 >> 172.21.15.120:0/946649861 conn(0x7f14c0067950 msgr2=0x7f14c0159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.102+0000 7f14a5ffb640 1 -- 172.21.15.120:0/946649861 shutdown_connections 2024-09-10T10:27:12.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.103+0000 7f14a5ffb640 1 -- 172.21.15.120:0/946649861 wait complete. 2024-09-10T10:27:12.120 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.1 2024-09-10T10:27:12.120 DEBUG:teuthology.parallel:result is None 2024-09-10T10:27:12.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.183+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa998005850 con 0x7fa9b00f66c0 2024-09-10T10:27:12.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.183+0000 7fa9a3fff640 1 -- 172.21.15.120:0/2489505050 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa9b0046900 con 0x7fa9b00f66c0 2024-09-10T10:27:12.185 INFO:teuthology.orchestra.run.smithi120.stdout:55834574858 2024-09-10T10:27:12.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.185+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa99c06be80 msgr2=0x7fa99c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.185+0000 7fa9bdbf8640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa99c06be80 0x7fa99c06e340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00bba0 tx=0x7fa9ac008040 comp rx=0 tx=0).stop 2024-09-10T10:27:12.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.185+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9b00f66c0 msgr2=0x7fa9b004e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.185+0000 7fa9bdbf8640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9b00f66c0 0x7fa9b004e020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4000c00 tx=0x7fa9a40046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.186+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 shutdown_connections 2024-09-10T10:27:12.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.186+0000 7fa9bdbf8640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa99c06be80 0x7fa99c06e340 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.186+0000 7fa9bdbf8640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9b00ff860 0x7fa9b004e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.186+0000 7fa9bdbf8640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9b00f66c0 0x7fa9b004e020 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.186+0000 7fa9bdbf8640 1 --2- 172.21.15.120:0/2489505050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9b004aa70 0x7fa9b00f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.186+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 >> 172.21.15.120:0/2489505050 conn(0x7fa9b004fc80 msgr2=0x7fa9b004c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.187+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 shutdown_connections 2024-09-10T10:27:12.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.187+0000 7fa9bdbf8640 1 -- 172.21.15.120:0/2489505050 wait complete. 2024-09-10T10:27:12.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.5 2024-09-10T10:27:12.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.198+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9a9c005850 con 0x7f9abc150a50 2024-09-10T10:27:12.199 DEBUG:teuthology.parallel:result is None 2024-09-10T10:27:12.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.199+0000 7f9aa3fff640 1 -- 172.21.15.120:0/408497353 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9ab407a620 con 0x7f9abc150a50 2024-09-10T10:27:12.200 INFO:teuthology.orchestra.run.smithi120.stdout:55834574859 2024-09-10T10:27:12.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.201+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aa806bf30 msgr2=0x7f9aa806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.201+0000 7f9ac0d57640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aa806bf30 0x7f9aa806e3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0096c0 tx=0x7f9aac006bd0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.201+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150a50 msgr2=0x7f9abc0c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.201+0000 7f9ac0d57640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150a50 0x7f9abc0c5a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400d3c0 tx=0x7f9ab4004a40 comp rx=0 tx=0).stop 2024-09-10T10:27:12.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.202+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 shutdown_connections 2024-09-10T10:27:12.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.202+0000 7f9ac0d57640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9aa806bf30 0x7f9aa806e3f0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0096c0 tx=0x7f9aac006bd0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.202+0000 7f9ac0d57640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9abc14cf10 0x7f9abc0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.202+0000 7f9ac0d57640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9abc150a50 0x7f9abc0c5a20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.202+0000 7f9ac0d57640 1 --2- 172.21.15.120:0/408497353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9abc0c6990 0x7f9abc0c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.202+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 >> 172.21.15.120:0/408497353 conn(0x7f9abc067950 msgr2=0x7f9abc148f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.202+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 shutdown_connections 2024-09-10T10:27:12.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.202+0000 7f9ac0d57640 1 -- 172.21.15.120:0/408497353 wait complete. 2024-09-10T10:27:12.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.4 2024-09-10T10:27:12.218 DEBUG:teuthology.parallel:result is None 2024-09-10T10:27:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.239+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f16640dc3f0 con 0x7f16640d4750 2024-09-10T10:27:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.240+0000 7f16697fa640 1 -- 172.21.15.120:0/841034522 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f166c09b060 con 0x7f16640d4750 2024-09-10T10:27:12.241 INFO:teuthology.orchestra.run.smithi120.stdout:55834574858 2024-09-10T10:27:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f165006bf30 msgr2=0x7f165006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f165006bf30 0x7f165006e3f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1660007850 tx=0x7f1660009040 comp rx=0 tx=0).stop 2024-09-10T10:27:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16640d4750 msgr2=0x7f16640d7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16640d4750 0x7f16640d7d30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f166c000b60 tx=0x7f166c069890 comp rx=0 tx=0).stop 2024-09-10T10:27:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 shutdown_connections 2024-09-10T10:27:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f165006bf30 0x7f165006e3f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16640d4750 0x7f16640d7d30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16640d8e60 0x7f16640068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 --2- 172.21.15.120:0/841034522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f165c004800 0x7f1664175ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 >> 172.21.15.120:0/841034522 conn(0x7f166404fc50 msgr2=0x7f16640d3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 shutdown_connections 2024-09-10T10:27:12.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.242+0000 7f1671ae7640 1 -- 172.21.15.120:0/841034522 wait complete. 2024-09-10T10:27:12.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.3 2024-09-10T10:27:12.255 DEBUG:teuthology.parallel:result is None 2024-09-10T10:27:12.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.267+0000 7f3ed68e4640 1 -- 172.21.15.120:0/2873775981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f3eb0005580 con 0x7f3ec8048130 2024-09-10T10:27:12.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.268+0000 7f3eccff9640 1 -- 172.21.15.120:0/2873775981 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ec4059980 con 0x7f3ec8048130 2024-09-10T10:27:12.269 INFO:teuthology.orchestra.run.smithi120.stdout:55834574859 2024-09-10T10:27:12.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3b380cb260 con 0x7f3b380c7770 2024-09-10T10:27:12.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 -- 172.21.15.120:0/2873775981 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3eb406c230 msgr2=0x7f3eb406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3eb406c230 0x7f3eb406e6f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ed002dd50 tx=0x7f3ed008f040 comp rx=0 tx=0).stop 2024-09-10T10:27:12.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 -- 172.21.15.120:0/2873775981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec8048130 msgr2=0x7f3ec80078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec8048130 0x7f3ec80078e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e5c0 tx=0x7f3ec400ea80 comp rx=0 tx=0).stop 2024-09-10T10:27:12.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 -- 172.21.15.120:0/2873775981 shutdown_connections 2024-09-10T10:27:12.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3eb406c230 0x7f3eb406e6f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ec8173b80 0x7f3ec8170100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec8048130 0x7f3ec80078e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 --2- 172.21.15.120:0/2873775981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ec00074f0 0x7f3ec80ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 -- 172.21.15.120:0/2873775981 >> 172.21.15.120:0/2873775981 conn(0x7f3ec804d830 msgr2=0x7f3ec80d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 -- 172.21.15.120:0/2873775981 shutdown_connections 2024-09-10T10:27:12.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3ed68e4640 1 -- 172.21.15.120:0/2873775981 wait complete. 2024-09-10T10:27:12.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.271+0000 7f3b2dffb640 1 -- 172.21.15.120:0/2579674557 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3b3407ee20 con 0x7f3b380c7770 2024-09-10T10:27:12.274 INFO:teuthology.orchestra.run.smithi120.stdout:55834574858 2024-09-10T10:27:12.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b1c06be30 msgr2=0x7f3b1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b1c06be30 0x7f3b1c06e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3b380ce830 tx=0x7f3b30003d90 comp rx=0 tx=0).stop 2024-09-10T10:27:12.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b380c7770 msgr2=0x7f3b38151080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b380c7770 0x7f3b38151080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b340098c0 tx=0x7f3b340046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 shutdown_connections 2024-09-10T10:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3b1c06be30 0x7f3b1c06e2f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b380d0950 0x7f3b3814c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b380d0570 0x7f3b3815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 --2- 172.21.15.120:0/2579674557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b380c7770 0x7f3b38151080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 >> 172.21.15.120:0/2579674557 conn(0x7f3b3806ca90 msgr2=0x7f3b3815ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 shutdown_connections 2024-09-10T10:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.275+0000 7f3b3f804640 1 -- 172.21.15.120:0/2579674557 wait complete. 2024-09-10T10:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.276+0000 7f998a5a6640 1 -- 172.21.15.120:0/2885573689 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f99840c7080 con 0x7f99840c75b0 2024-09-10T10:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.277+0000 7f9980ff9640 1 -- 172.21.15.120:0/2885573689 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f997405d330 con 0x7f99840c75b0 2024-09-10T10:27:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:55834574858 2024-09-10T10:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 -- 172.21.15.120:0/2885573689 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f997006be60 msgr2=0x7f997006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f997006be60 0x7f997006e320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f998406be30 tx=0x7f9978009040 comp rx=0 tx=0).stop 2024-09-10T10:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 -- 172.21.15.120:0/2885573689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99840c75b0 msgr2=0x7f99841fde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99840c75b0 0x7f99841fde10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f997400e8b0 tx=0x7f997400ed70 comp rx=0 tx=0).stop 2024-09-10T10:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 -- 172.21.15.120:0/2885573689 shutdown_connections 2024-09-10T10:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f997006be60 0x7f997006e320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99841591d0 0x7f998406c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99840c8a50 0x7f99841e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 --2- 172.21.15.120:0/2885573689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99840c75b0 0x7f99841fde10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 -- 172.21.15.120:0/2885573689 >> 172.21.15.120:0/2885573689 conn(0x7f99840ccbb0 msgr2=0x7f99840c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 -- 172.21.15.120:0/2885573689 shutdown_connections 2024-09-10T10:27:12.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.279+0000 7f998a5a6640 1 -- 172.21.15.120:0/2885573689 wait complete. 2024-09-10T10:27:12.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.287+0000 7f128152f640 1 -- 172.21.15.120:0/3645098095 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f127404a570 con 0x7f1274047100 2024-09-10T10:27:12.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.287+0000 7f1278ff9640 1 -- 172.21.15.120:0/3645098095 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1274047ac0 con 0x7f1274047100 2024-09-10T10:27:12.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.7 2024-09-10T10:27:12.289 INFO:teuthology.orchestra.run.smithi120.stdout:55834574859 2024-09-10T10:27:12.290 DEBUG:teuthology.parallel:result is None 2024-09-10T10:27:12.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.6 2024-09-10T10:27:12.291 DEBUG:teuthology.parallel:result is None 2024-09-10T10:27:12.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 -- 172.21.15.120:0/3645098095 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f126006be30 msgr2=0x7f126006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f126006be30 0x7f126006e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1270007fd0 tx=0x7f1270009040 comp rx=0 tx=0).stop 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 -- 172.21.15.120:0/3645098095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1274047100 msgr2=0x7f1274047580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1274047100 0x7f1274047580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f127c06a930 tx=0x7f127c069890 comp rx=0 tx=0).stop 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 -- 172.21.15.120:0/3645098095 shutdown_connections 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f126006be30 0x7f126006e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1274047100 0x7f1274047580 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740497b0 0x7f1274049bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 --2- 172.21.15.120:0/3645098095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f126c0059b0 0x7f1274048810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 -- 172.21.15.120:0/3645098095 >> 172.21.15.120:0/3645098095 conn(0x7f127404fc50 msgr2=0x7f127404ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 -- 172.21.15.120:0/3645098095 shutdown_connections 2024-09-10T10:27:12.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.289+0000 7f128152f640 1 -- 172.21.15.120:0/3645098095 wait complete. 2024-09-10T10:27:12.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.2 2024-09-10T10:27:12.294 DEBUG:teuthology.parallel:result is None 2024-09-10T10:27:12.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.0 2024-09-10T10:27:12.302 DEBUG:teuthology.parallel:result is None 2024-09-10T10:27:12.302 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:27:12.302 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:27:12.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.439+0000 7f160f14b640 1 Processor -- start 2024-09-10T10:27:12.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.439+0000 7f160f14b640 1 -- start start 2024-09-10T10:27:12.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.440+0000 7f160f14b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1608072170 0x7f160806cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.440+0000 7f160f14b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f160806d160 0x7f1608072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.440+0000 7f160f14b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16080c6cf0 con 0x7f1608154d40 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.440+0000 7f160f14b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1608072830 con 0x7f160806d160 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.440+0000 7f160f14b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f160815ab60 con 0x7f1608072170 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.440+0000 7f160d6e8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f160806d160 0x7f1608072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.440+0000 7f160d6e8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f160806d160 0x7f1608072cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34732/0 (socket says 172.21.15.120:34732) 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.440+0000 7f160d6e8640 1 -- 172.21.15.120:0/1267255620 learned_addr learned my addr 172.21.15.120:0/1267255620 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.440+0000 7f1607fff640 1 --2- 172.21.15.120:0/1267255620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1608072170 0x7f160806cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.441+0000 7f160d6e8640 1 -- 172.21.15.120:0/1267255620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1608072170 msgr2=0x7f160806cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.441+0000 7f160d6e8640 1 --2- 172.21.15.120:0/1267255620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1608072170 0x7f160806cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.441+0000 7f160d6e8640 1 -- 172.21.15.120:0/1267255620 >> v1:172.21.15.134:6789/0 conn(0x7f1608154d40 legacy=0x7f1608067950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.441+0000 7f160d6e8640 1 -- 172.21.15.120:0/1267255620 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160815ae00 con 0x7f160806d160 2024-09-10T10:27:12.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.441+0000 7f160d6e8640 1 --2- 172.21.15.120:0/1267255620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f160806d160 0x7f1608072cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1600009870 tx=0x7f160002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434b6fe38e110de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:12.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.441+0000 7f16077fe640 1 -- 172.21.15.120:0/1267255620 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600037070 con 0x7f160806d160 2024-09-10T10:27:12.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.441+0000 7f16077fe640 1 -- 172.21.15.120:0/1267255620 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1600032440 con 0x7f160806d160 2024-09-10T10:27:12.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.441+0000 7f16077fe640 1 -- 172.21.15.120:0/1267255620 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600032810 con 0x7f160806d160 2024-09-10T10:27:12.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.442+0000 7f160f14b640 1 -- 172.21.15.120:0/1267255620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f160806d160 msgr2=0x7f1608072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.442+0000 7f160f14b640 1 --2- 172.21.15.120:0/1267255620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f160806d160 0x7f1608072cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1600009870 tx=0x7f160002a270 comp rx=0 tx=0).stop 2024-09-10T10:27:12.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.442+0000 7f160f14b640 1 -- 172.21.15.120:0/1267255620 shutdown_connections 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.442+0000 7f160f14b640 1 --2- 172.21.15.120:0/1267255620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f160806d160 0x7f1608072cc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.442+0000 7f160f14b640 1 --2- 172.21.15.120:0/1267255620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1608072170 0x7f160806cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.442+0000 7f160f14b640 1 -- 172.21.15.120:0/1267255620 >> 172.21.15.120:0/1267255620 conn(0x7f160806d8e0 msgr2=0x7f160806dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.442+0000 7f160f14b640 1 -- 172.21.15.120:0/1267255620 shutdown_connections 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.442+0000 7f160f14b640 1 -- 172.21.15.120:0/1267255620 wait complete. 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.443+0000 7f160f14b640 1 Processor -- start 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.443+0000 7f160f14b640 1 -- start start 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160f14b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160806d160 0x7f160814cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160f14b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1608072170 0x7f16081f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160f14b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1608154d40 0x7f16081475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160f14b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f160815b6a0 con 0x7f1608154d40 2024-09-10T10:27:12.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160f14b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f160815b3c0 con 0x7f160806d160 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160f14b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f160815b980 con 0x7f1608072170 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160d6e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1608154d40 0x7f16081475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160d6e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1608154d40 0x7f16081475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34746/0 (socket says 172.21.15.120:34746) 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160d6e8640 1 -- 172.21.15.120:0/4084526978 learned_addr learned my addr 172.21.15.120:0/4084526978 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160cee7640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160806d160 0x7f160814cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f1607fff640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1608072170 0x7f16081f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160d6e8640 1 -- 172.21.15.120:0/4084526978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1608072170 msgr2=0x7f16081f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160d6e8640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1608072170 0x7f16081f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160d6e8640 1 -- 172.21.15.120:0/4084526978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160806d160 msgr2=0x7f160814cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160d6e8640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160806d160 0x7f160814cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.444+0000 7f160d6e8640 1 -- 172.21.15.120:0/4084526978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1608147b20 con 0x7f1608154d40 2024-09-10T10:27:12.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.445+0000 7f160d6e8640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1608154d40 0x7f16081475e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1600009870 tx=0x7f16000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:12.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.445+0000 7f16057fa640 1 -- 172.21.15.120:0/4084526978 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600037040 con 0x7f1608154d40 2024-09-10T10:27:12.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.445+0000 7f16057fa640 1 -- 172.21.15.120:0/4084526978 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1600042070 con 0x7f1608154d40 2024-09-10T10:27:12.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.445+0000 7f160f14b640 1 -- 172.21.15.120:0/4084526978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160814f4d0 con 0x7f1608154d40 2024-09-10T10:27:12.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.445+0000 7f16057fa640 1 -- 172.21.15.120:0/4084526978 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600030920 con 0x7f1608154d40 2024-09-10T10:27:12.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.445+0000 7f160f14b640 1 -- 172.21.15.120:0/4084526978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16081f3cc0 con 0x7f1608154d40 2024-09-10T10:27:12.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.446+0000 7f160f14b640 1 -- 172.21.15.120:0/4084526978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1608155120 con 0x7f1608154d40 2024-09-10T10:27:12.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.446+0000 7f16057fa640 1 -- 172.21.15.120:0/4084526978 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1600030ac0 con 0x7f1608154d40 2024-09-10T10:27:12.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.447+0000 7f16057fa640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f15f406be30 0x7f15f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.447+0000 7f16057fa640 1 -- 172.21.15.120:0/4084526978 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f16000ab9e0 con 0x7f1608154d40 2024-09-10T10:27:12.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.447+0000 7f160cee7640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f15f406be30 0x7f15f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.447+0000 7f160cee7640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f15f406be30 0x7f15f406e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f15f80016e0 tx=0x7f15f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:12.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.451+0000 7f16057fa640 1 -- 172.21.15.120:0/4084526978 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1600079a00 con 0x7f1608154d40 2024-09-10T10:27:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.619+0000 7f160f14b640 1 -- 172.21.15.120:0/4084526978 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f160814f280 con 0x7f15f406be30 2024-09-10T10:27:12.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.626+0000 7f16057fa640 1 -- 172.21.15.120:0/4084526978 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332761 (secure 0 0 0) 0x7f160814f280 con 0x7f15f406be30 2024-09-10T10:27:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:12.633 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:27:12.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 -- 172.21.15.120:0/4084526978 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f15f406be30 msgr2=0x7f15f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f15f406be30 0x7f15f406e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f15f80016e0 tx=0x7f15f8009040 comp rx=0 tx=0).stop 2024-09-10T10:27:12.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 -- 172.21.15.120:0/4084526978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1608154d40 msgr2=0x7f16081475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1608154d40 0x7f16081475e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1600009870 tx=0x7f16000046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 -- 172.21.15.120:0/4084526978 shutdown_connections 2024-09-10T10:27:12.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f15f406be30 0x7f15f406e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1608154d40 0x7f16081475e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1608072170 0x7f16081f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 --2- 172.21.15.120:0/4084526978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160806d160 0x7f160814cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 -- 172.21.15.120:0/4084526978 >> 172.21.15.120:0/4084526978 conn(0x7f160806d8e0 msgr2=0x7f160806bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 -- 172.21.15.120:0/4084526978 shutdown_connections 2024-09-10T10:27:12.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.630+0000 7f160f14b640 1 -- 172.21.15.120:0/4084526978 wait complete. 2024-09-10T10:27:12.642 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-10T10:27:12.096828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596416,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":246,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193,"ondisk_log_size":193,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218636,"kb_used_data":2392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756084,"statfs":{"total":773094113280,"available":772870230016,"internally_reserved":0,"allocated":2449408,"data_stored":2087888,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003015"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271924+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:26:48.271924+0000","last_peered":"2024-09-10T10:26:48.271924+0000","last_clean":"2024-09-10T10:26:48.271924+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:26:48.271924+0000","last_undegraded":"2024-09-10T10:26:48.271924+0000","last_fullsized":"2024-09-10T10:26:48.271924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827265+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:26:47.827265+0000","last_peered":"2024-09-10T10:26:47.827265+0000","last_clean":"2024-09-10T10:26:47.827265+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:26:47.827265+0000","last_undegraded":"2024-09-10T10:26:47.827265+0000","last_fullsized":"2024-09-10T10:26:47.827265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826403+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:26:47.826403+0000","last_peered":"2024-09-10T10:26:47.826403+0000","last_clean":"2024-09-10T10:26:47.826403+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:26:47.826403+0000","last_undegraded":"2024-09-10T10:26:47.826403+0000","last_fullsized":"2024-09-10T10:26:47.826403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827328+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:26:47.827328+0000","last_peered":"2024-09-10T10:26:47.827328+0000","last_clean":"2024-09-10T10:26:47.827328+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:26:47.827328+0000","last_undegraded":"2024-09-10T10:26:47.827328+0000","last_fullsized":"2024-09-10T10:26:47.827328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271728+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:26:48.271728+0000","last_peered":"2024-09-10T10:26:48.271728+0000","last_clean":"2024-09-10T10:26:48.271728+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:26:48.271728+0000","last_undegraded":"2024-09-10T10:26:48.271728+0000","last_fullsized":"2024-09-10T10:26:48.271728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829940+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:26:47.829940+0000","last_peered":"2024-09-10T10:26:47.829940+0000","last_clean":"2024-09-10T10:26:47.829940+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:26:47.829940+0000","last_undegraded":"2024-09-10T10:26:47.829940+0000","last_fullsized":"2024-09-10T10:26:47.829940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437245+0000","last_change":"2024-09-10T10:26:40.796337+0000","last_active":"2024-09-10T10:26:48.437245+0000","last_peered":"2024-09-10T10:26:48.437245+0000","last_clean":"2024-09-10T10:26:48.437245+0000","last_became_active":"2024-09-10T10:26:40.796029+0000","last_became_peered":"2024-09-10T10:26:40.796029+0000","last_unstale":"2024-09-10T10:26:48.437245+0000","last_undegraded":"2024-09-10T10:26:48.437245+0000","last_fullsized":"2024-09-10T10:26:48.437245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829990+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:26:47.829990+0000","last_peered":"2024-09-10T10:26:47.829990+0000","last_clean":"2024-09-10T10:26:47.829990+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:26:47.829990+0000","last_undegraded":"2024-09-10T10:26:47.829990+0000","last_fullsized":"2024-09-10T10:26:47.829990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831277+0000","last_change":"2024-09-10T10:26:45.814368+0000","last_active":"2024-09-10T10:26:47.831277+0000","last_peered":"2024-09-10T10:26:47.831277+0000","last_clean":"2024-09-10T10:26:47.831277+0000","last_became_active":"2024-09-10T10:26:45.813953+0000","last_became_peered":"2024-09-10T10:26:45.813953+0000","last_unstale":"2024-09-10T10:26:47.831277+0000","last_undegraded":"2024-09-10T10:26:47.831277+0000","last_fullsized":"2024-09-10T10:26:47.831277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436841+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:26:48.436841+0000","last_peered":"2024-09-10T10:26:48.436841+0000","last_clean":"2024-09-10T10:26:48.436841+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:26:48.436841+0000","last_undegraded":"2024-09-10T10:26:48.436841+0000","last_fullsized":"2024-09-10T10:26:48.436841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830425+0000","last_change":"2024-09-10T10:26:41.789145+0000","last_active":"2024-09-10T10:26:47.830425+0000","last_peered":"2024-09-10T10:26:47.830425+0000","last_clean":"2024-09-10T10:26:47.830425+0000","last_became_active":"2024-09-10T10:26:41.788874+0000","last_became_peered":"2024-09-10T10:26:41.788874+0000","last_unstale":"2024-09-10T10:26:47.830425+0000","last_undegraded":"2024-09-10T10:26:47.830425+0000","last_fullsized":"2024-09-10T10:26:47.830425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830738+0000","last_change":"2024-09-10T10:26:45.816587+0000","last_active":"2024-09-10T10:26:47.830738+0000","last_peered":"2024-09-10T10:26:47.830738+0000","last_clean":"2024-09-10T10:26:47.830738+0000","last_became_active":"2024-09-10T10:26:45.816138+0000","last_became_peered":"2024-09-10T10:26:45.816138+0000","last_unstale":"2024-09-10T10:26:47.830738+0000","last_undegraded":"2024-09-10T10:26:47.830738+0000","last_fullsized":"2024-09-10T10:26:47.830738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436252+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:26:48.436252+0000","last_peered":"2024-09-10T10:26:48.436252+0000","last_clean":"2024-09-10T10:26:48.436252+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:26:48.436252+0000","last_undegraded":"2024-09-10T10:26:48.436252+0000","last_fullsized":"2024-09-10T10:26:48.436252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.875798+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:27:08.875798+0000","last_peered":"2024-09-10T10:27:08.875798+0000","last_clean":"2024-09-10T10:27:08.875798+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:27:08.875798+0000","last_undegraded":"2024-09-10T10:27:08.875798+0000","last_fullsized":"2024-09-10T10:27:08.875798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831602+0000","last_change":"2024-09-10T10:26:45.816967+0000","last_active":"2024-09-10T10:26:47.831602+0000","last_peered":"2024-09-10T10:26:47.831602+0000","last_clean":"2024-09-10T10:26:47.831602+0000","last_became_active":"2024-09-10T10:26:45.816161+0000","last_became_peered":"2024-09-10T10:26:45.816161+0000","last_unstale":"2024-09-10T10:26:47.831602+0000","last_undegraded":"2024-09-10T10:26:47.831602+0000","last_fullsized":"2024-09-10T10:26:47.831602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826732+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:26:47.826732+0000","last_peered":"2024-09-10T10:26:47.826732+0000","last_clean":"2024-09-10T10:26:47.826732+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:26:47.826732+0000","last_undegraded":"2024-09-10T10:26:47.826732+0000","last_fullsized":"2024-09-10T10:26:47.826732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436716+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:26:48.436716+0000","last_peered":"2024-09-10T10:26:48.436716+0000","last_clean":"2024-09-10T10:26:48.436716+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:26:48.436716+0000","last_undegraded":"2024-09-10T10:26:48.436716+0000","last_fullsized":"2024-09-10T10:26:48.436716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830916+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:26:47.830916+0000","last_peered":"2024-09-10T10:26:47.830916+0000","last_clean":"2024-09-10T10:26:47.830916+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:26:47.830916+0000","last_undegraded":"2024-09-10T10:26:47.830916+0000","last_fullsized":"2024-09-10T10:26:47.830916+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826532+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:26:47.826532+0000","last_peered":"2024-09-10T10:26:47.826532+0000","last_clean":"2024-09-10T10:26:47.826532+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:26:47.826532+0000","last_undegraded":"2024-09-10T10:26:47.826532+0000","last_fullsized":"2024-09-10T10:26:47.826532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.883285+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:27:08.883285+0000","last_peered":"2024-09-10T10:27:08.883285+0000","last_clean":"2024-09-10T10:27:08.883285+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:27:08.883285+0000","last_undegraded":"2024-09-10T10:27:08.883285+0000","last_fullsized":"2024-09-10T10:27:08.883285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436960+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:26:48.436960+0000","last_peered":"2024-09-10T10:26:48.436960+0000","last_clean":"2024-09-10T10:26:48.436960+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:26:48.436960+0000","last_undegraded":"2024-09-10T10:26:48.436960+0000","last_fullsized":"2024-09-10T10:26:48.436960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830283+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:26:47.830283+0000","last_peered":"2024-09-10T10:26:47.830283+0000","last_clean":"2024-09-10T10:26:47.830283+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:26:47.830283+0000","last_undegraded":"2024-09-10T10:26:47.830283+0000","last_fullsized":"2024-09-10T10:26:47.830283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.876443+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:27:08.876443+0000","last_peered":"2024-09-10T10:27:08.876443+0000","last_clean":"2024-09-10T10:27:08.876443+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:27:08.876443+0000","last_undegraded":"2024-09-10T10:27:08.876443+0000","last_fullsized":"2024-09-10T10:27:08.876443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830167+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:26:47.830167+0000","last_peered":"2024-09-10T10:26:47.830167+0000","last_clean":"2024-09-10T10:26:47.830167+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:26:47.830167+0000","last_undegraded":"2024-09-10T10:26:47.830167+0000","last_fullsized":"2024-09-10T10:26:47.830167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271909+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:26:48.271909+0000","last_peered":"2024-09-10T10:26:48.271909+0000","last_clean":"2024-09-10T10:26:48.271909+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:26:48.271909+0000","last_undegraded":"2024-09-10T10:26:48.271909+0000","last_fullsized":"2024-09-10T10:26:48.271909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.876078+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:27:08.876078+0000","last_peered":"2024-09-10T10:27:08.876078+0000","last_clean":"2024-09-10T10:27:08.876078+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:27:08.876078+0000","last_undegraded":"2024-09-10T10:27:08.876078+0000","last_fullsized":"2024-09-10T10:27:08.876078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829937+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:26:47.829937+0000","last_peered":"2024-09-10T10:26:47.829937+0000","last_clean":"2024-09-10T10:26:47.829937+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:26:47.829937+0000","last_undegraded":"2024-09-10T10:26:47.829937+0000","last_fullsized":"2024-09-10T10:26:47.829937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827469+0000","last_change":"2024-09-10T10:26:41.795085+0000","last_active":"2024-09-10T10:26:47.827469+0000","last_peered":"2024-09-10T10:26:47.827469+0000","last_clean":"2024-09-10T10:26:47.827469+0000","last_became_active":"2024-09-10T10:26:41.794819+0000","last_became_peered":"2024-09-10T10:26:41.794819+0000","last_unstale":"2024-09-10T10:26:47.827469+0000","last_undegraded":"2024-09-10T10:26:47.827469+0000","last_fullsized":"2024-09-10T10:26:47.827469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436983+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:26:48.436983+0000","last_peered":"2024-09-10T10:26:48.436983+0000","last_clean":"2024-09-10T10:26:48.436983+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:26:48.436983+0000","last_undegraded":"2024-09-10T10:26:48.436983+0000","last_fullsized":"2024-09-10T10:26:48.436983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826343+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:26:47.826343+0000","last_peered":"2024-09-10T10:26:47.826343+0000","last_clean":"2024-09-10T10:26:47.826343+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:26:47.826343+0000","last_undegraded":"2024-09-10T10:26:47.826343+0000","last_fullsized":"2024-09-10T10:26:47.826343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436707+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:26:48.436707+0000","last_peered":"2024-09-10T10:26:48.436707+0000","last_clean":"2024-09-10T10:26:48.436707+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:26:48.436707+0000","last_undegraded":"2024-09-10T10:26:48.436707+0000","last_fullsized":"2024-09-10T10:26:48.436707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437443+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:26:48.437443+0000","last_peered":"2024-09-10T10:26:48.437443+0000","last_clean":"2024-09-10T10:26:48.437443+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:26:48.437443+0000","last_undegraded":"2024-09-10T10:26:48.437443+0000","last_fullsized":"2024-09-10T10:26:48.437443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271566+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:26:48.271566+0000","last_peered":"2024-09-10T10:26:48.271566+0000","last_clean":"2024-09-10T10:26:48.271566+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:26:48.271566+0000","last_undegraded":"2024-09-10T10:26:48.271566+0000","last_fullsized":"2024-09-10T10:26:48.271566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830633+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:26:47.830633+0000","last_peered":"2024-09-10T10:26:47.830633+0000","last_clean":"2024-09-10T10:26:47.830633+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:26:47.830633+0000","last_undegraded":"2024-09-10T10:26:47.830633+0000","last_fullsized":"2024-09-10T10:26:47.830633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436761+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:26:48.436761+0000","last_peered":"2024-09-10T10:26:48.436761+0000","last_clean":"2024-09-10T10:26:48.436761+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:26:48.436761+0000","last_undegraded":"2024-09-10T10:26:48.436761+0000","last_fullsized":"2024-09-10T10:26:48.436761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436477+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:26:48.436477+0000","last_peered":"2024-09-10T10:26:48.436477+0000","last_clean":"2024-09-10T10:26:48.436477+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:26:48.436477+0000","last_undegraded":"2024-09-10T10:26:48.436477+0000","last_fullsized":"2024-09-10T10:26:48.436477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831395+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:26:47.831395+0000","last_peered":"2024-09-10T10:26:47.831395+0000","last_clean":"2024-09-10T10:26:47.831395+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:26:47.831395+0000","last_undegraded":"2024-09-10T10:26:47.831395+0000","last_fullsized":"2024-09-10T10:26:47.831395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830140+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:26:47.830140+0000","last_peered":"2024-09-10T10:26:47.830140+0000","last_clean":"2024-09-10T10:26:47.830140+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:26:47.830140+0000","last_undegraded":"2024-09-10T10:26:47.830140+0000","last_fullsized":"2024-09-10T10:26:47.830140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436656+0000","last_change":"2024-09-10T10:26:45.814008+0000","last_active":"2024-09-10T10:26:48.436656+0000","last_peered":"2024-09-10T10:26:48.436656+0000","last_clean":"2024-09-10T10:26:48.436656+0000","last_became_active":"2024-09-10T10:26:45.813627+0000","last_became_peered":"2024-09-10T10:26:45.813627+0000","last_unstale":"2024-09-10T10:26:48.436656+0000","last_undegraded":"2024-09-10T10:26:48.436656+0000","last_fullsized":"2024-09-10T10:26:48.436656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436224+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:26:48.436224+0000","last_peered":"2024-09-10T10:26:48.436224+0000","last_clean":"2024-09-10T10:26:48.436224+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:26:48.436224+0000","last_undegraded":"2024-09-10T10:26:48.436224+0000","last_fullsized":"2024-09-10T10:26:48.436224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436077+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:26:48.436077+0000","last_peered":"2024-09-10T10:26:48.436077+0000","last_clean":"2024-09-10T10:26:48.436077+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:26:48.436077+0000","last_undegraded":"2024-09-10T10:26:48.436077+0000","last_fullsized":"2024-09-10T10:26:48.436077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436987+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:26:48.436987+0000","last_peered":"2024-09-10T10:26:48.436987+0000","last_clean":"2024-09-10T10:26:48.436987+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:26:48.436987+0000","last_undegraded":"2024-09-10T10:26:48.436987+0000","last_fullsized":"2024-09-10T10:26:48.436987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271316+0000","last_change":"2024-09-10T10:26:41.799224+0000","last_active":"2024-09-10T10:26:48.271316+0000","last_peered":"2024-09-10T10:26:48.271316+0000","last_clean":"2024-09-10T10:26:48.271316+0000","last_became_active":"2024-09-10T10:26:41.798948+0000","last_became_peered":"2024-09-10T10:26:41.798948+0000","last_unstale":"2024-09-10T10:26:48.271316+0000","last_undegraded":"2024-09-10T10:26:48.271316+0000","last_fullsized":"2024-09-10T10:26:48.271316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437014+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:26:48.437014+0000","last_peered":"2024-09-10T10:26:48.437014+0000","last_clean":"2024-09-10T10:26:48.437014+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:26:48.437014+0000","last_undegraded":"2024-09-10T10:26:48.437014+0000","last_fullsized":"2024-09-10T10:26:48.437014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271901+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:26:48.271901+0000","last_peered":"2024-09-10T10:26:48.271901+0000","last_clean":"2024-09-10T10:26:48.271901+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:26:48.271901+0000","last_undegraded":"2024-09-10T10:26:48.271901+0000","last_fullsized":"2024-09-10T10:26:48.271901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826634+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:26:47.826634+0000","last_peered":"2024-09-10T10:26:47.826634+0000","last_clean":"2024-09-10T10:26:47.826634+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:26:47.826634+0000","last_undegraded":"2024-09-10T10:26:47.826634+0000","last_fullsized":"2024-09-10T10:26:47.826634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.877722+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:27:08.877722+0000","last_peered":"2024-09-10T10:27:08.877722+0000","last_clean":"2024-09-10T10:27:08.877722+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:27:08.877722+0000","last_undegraded":"2024-09-10T10:27:08.877722+0000","last_fullsized":"2024-09-10T10:27:08.877722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437071+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:26:48.437071+0000","last_peered":"2024-09-10T10:26:48.437071+0000","last_clean":"2024-09-10T10:26:48.437071+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:26:48.437071+0000","last_undegraded":"2024-09-10T10:26:48.437071+0000","last_fullsized":"2024-09-10T10:26:48.437071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437061+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:26:48.437061+0000","last_peered":"2024-09-10T10:26:48.437061+0000","last_clean":"2024-09-10T10:26:48.437061+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:26:48.437061+0000","last_undegraded":"2024-09-10T10:26:48.437061+0000","last_fullsized":"2024-09-10T10:26:48.437061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437121+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:26:48.437121+0000","last_peered":"2024-09-10T10:26:48.437121+0000","last_clean":"2024-09-10T10:26:48.437121+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:26:48.437121+0000","last_undegraded":"2024-09-10T10:26:48.437121+0000","last_fullsized":"2024-09-10T10:26:48.437121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436307+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:26:48.436307+0000","last_peered":"2024-09-10T10:26:48.436307+0000","last_clean":"2024-09-10T10:26:48.436307+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:26:48.436307+0000","last_undegraded":"2024-09-10T10:26:48.436307+0000","last_fullsized":"2024-09-10T10:26:48.436307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436298+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:26:48.436298+0000","last_peered":"2024-09-10T10:26:48.436298+0000","last_clean":"2024-09-10T10:26:48.436298+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:26:48.436298+0000","last_undegraded":"2024-09-10T10:26:48.436298+0000","last_fullsized":"2024-09-10T10:26:48.436298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830079+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:26:47.830079+0000","last_peered":"2024-09-10T10:26:47.830079+0000","last_clean":"2024-09-10T10:26:47.830079+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:26:47.830079+0000","last_undegraded":"2024-09-10T10:26:47.830079+0000","last_fullsized":"2024-09-10T10:26:47.830079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436713+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:26:48.436713+0000","last_peered":"2024-09-10T10:26:48.436713+0000","last_clean":"2024-09-10T10:26:48.436713+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:26:48.436713+0000","last_undegraded":"2024-09-10T10:26:48.436713+0000","last_fullsized":"2024-09-10T10:26:48.436713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271787+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:26:48.271787+0000","last_peered":"2024-09-10T10:26:48.271787+0000","last_clean":"2024-09-10T10:26:48.271787+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:26:48.271787+0000","last_undegraded":"2024-09-10T10:26:48.271787+0000","last_fullsized":"2024-09-10T10:26:48.271787+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437650+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:26:48.437650+0000","last_peered":"2024-09-10T10:26:48.437650+0000","last_clean":"2024-09-10T10:26:48.437650+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:26:48.437650+0000","last_undegraded":"2024-09-10T10:26:48.437650+0000","last_fullsized":"2024-09-10T10:26:48.437650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436651+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:26:48.436651+0000","last_peered":"2024-09-10T10:26:48.436651+0000","last_clean":"2024-09-10T10:26:48.436651+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:26:48.436651+0000","last_undegraded":"2024-09-10T10:26:48.436651+0000","last_fullsized":"2024-09-10T10:26:48.436651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829425+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:26:47.829425+0000","last_peered":"2024-09-10T10:26:47.829425+0000","last_clean":"2024-09-10T10:26:47.829425+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:26:47.829425+0000","last_undegraded":"2024-09-10T10:26:47.829425+0000","last_fullsized":"2024-09-10T10:26:47.829425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437218+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:26:48.437218+0000","last_peered":"2024-09-10T10:26:48.437218+0000","last_clean":"2024-09-10T10:26:48.437218+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:26:48.437218+0000","last_undegraded":"2024-09-10T10:26:48.437218+0000","last_fullsized":"2024-09-10T10:26:48.437218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830430+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:26:47.830430+0000","last_peered":"2024-09-10T10:26:47.830430+0000","last_clean":"2024-09-10T10:26:47.830430+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:26:47.830430+0000","last_undegraded":"2024-09-10T10:26:47.830430+0000","last_fullsized":"2024-09-10T10:26:47.830430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829765+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:26:47.829765+0000","last_peered":"2024-09-10T10:26:47.829765+0000","last_clean":"2024-09-10T10:26:47.829765+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:26:47.829765+0000","last_undegraded":"2024-09-10T10:26:47.829765+0000","last_fullsized":"2024-09-10T10:26:47.829765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437037+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:26:48.437037+0000","last_peered":"2024-09-10T10:26:48.437037+0000","last_clean":"2024-09-10T10:26:48.437037+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:26:48.437037+0000","last_undegraded":"2024-09-10T10:26:48.437037+0000","last_fullsized":"2024-09-10T10:26:48.437037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'10","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.878414+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:27:08.878414+0000","last_peered":"2024-09-10T10:27:08.878414+0000","last_clean":"2024-09-10T10:27:08.878414+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:27:08.878414+0000","last_undegraded":"2024-09-10T10:27:08.878414+0000","last_fullsized":"2024-09-10T10:27:08.878414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271756+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:26:48.271756+0000","last_peered":"2024-09-10T10:26:48.271756+0000","last_clean":"2024-09-10T10:26:48.271756+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:26:48.271756+0000","last_undegraded":"2024-09-10T10:26:48.271756+0000","last_fullsized":"2024-09-10T10:26:48.271756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436218+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:26:48.436218+0000","last_peered":"2024-09-10T10:26:48.436218+0000","last_clean":"2024-09-10T10:26:48.436218+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:26:48.436218+0000","last_undegraded":"2024-09-10T10:26:48.436218+0000","last_fullsized":"2024-09-10T10:26:48.436218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436280+0000","last_change":"2024-09-10T10:26:40.794739+0000","last_active":"2024-09-10T10:26:48.436280+0000","last_peered":"2024-09-10T10:26:48.436280+0000","last_clean":"2024-09-10T10:26:48.436280+0000","last_became_active":"2024-09-10T10:26:40.794430+0000","last_became_peered":"2024-09-10T10:26:40.794430+0000","last_unstale":"2024-09-10T10:26:48.436280+0000","last_undegraded":"2024-09-10T10:26:48.436280+0000","last_fullsized":"2024-09-10T10:26:48.436280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827508+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:26:47.827508+0000","last_peered":"2024-09-10T10:26:47.827508+0000","last_clean":"2024-09-10T10:26:47.827508+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:26:47.827508+0000","last_undegraded":"2024-09-10T10:26:47.827508+0000","last_fullsized":"2024-09-10T10:26:47.827508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436803+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:26:48.436803+0000","last_peered":"2024-09-10T10:26:48.436803+0000","last_clean":"2024-09-10T10:26:48.436803+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:26:48.436803+0000","last_undegraded":"2024-09-10T10:26:48.436803+0000","last_fullsized":"2024-09-10T10:26:48.436803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.879564+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:27:08.879564+0000","last_peered":"2024-09-10T10:27:08.879564+0000","last_clean":"2024-09-10T10:27:08.879564+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:27:08.879564+0000","last_undegraded":"2024-09-10T10:27:08.879564+0000","last_fullsized":"2024-09-10T10:27:08.879564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827010+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:26:47.827010+0000","last_peered":"2024-09-10T10:26:47.827010+0000","last_clean":"2024-09-10T10:26:47.827010+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:26:47.827010+0000","last_undegraded":"2024-09-10T10:26:47.827010+0000","last_fullsized":"2024-09-10T10:26:47.827010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271524+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:26:48.271524+0000","last_peered":"2024-09-10T10:26:48.271524+0000","last_clean":"2024-09-10T10:26:48.271524+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:26:48.271524+0000","last_undegraded":"2024-09-10T10:26:48.271524+0000","last_fullsized":"2024-09-10T10:26:48.271524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.877837+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:27:08.877837+0000","last_peered":"2024-09-10T10:27:08.877837+0000","last_clean":"2024-09-10T10:27:08.877837+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:27:08.877837+0000","last_undegraded":"2024-09-10T10:27:08.877837+0000","last_fullsized":"2024-09-10T10:27:08.877837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830480+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:26:47.830480+0000","last_peered":"2024-09-10T10:26:47.830480+0000","last_clean":"2024-09-10T10:26:47.830480+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:26:47.830480+0000","last_undegraded":"2024-09-10T10:26:47.830480+0000","last_fullsized":"2024-09-10T10:26:47.830480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436991+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:26:48.436991+0000","last_peered":"2024-09-10T10:26:48.436991+0000","last_clean":"2024-09-10T10:26:48.436991+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:26:48.436991+0000","last_undegraded":"2024-09-10T10:26:48.436991+0000","last_fullsized":"2024-09-10T10:26:48.436991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'11","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.877358+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:27:08.877358+0000","last_peered":"2024-09-10T10:27:08.877358+0000","last_clean":"2024-09-10T10:27:08.877358+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:27:08.877358+0000","last_undegraded":"2024-09-10T10:27:08.877358+0000","last_fullsized":"2024-09-10T10:27:08.877358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436702+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:26:48.436702+0000","last_peered":"2024-09-10T10:26:48.436702+0000","last_clean":"2024-09-10T10:26:48.436702+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:26:48.436702+0000","last_undegraded":"2024-09-10T10:26:48.436702+0000","last_fullsized":"2024-09-10T10:26:48.436702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831260+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:26:47.831260+0000","last_peered":"2024-09-10T10:26:47.831260+0000","last_clean":"2024-09-10T10:26:47.831260+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:26:47.831260+0000","last_undegraded":"2024-09-10T10:26:47.831260+0000","last_fullsized":"2024-09-10T10:26:47.831260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436614+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:26:48.436614+0000","last_peered":"2024-09-10T10:26:48.436614+0000","last_clean":"2024-09-10T10:26:48.436614+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:26:48.436614+0000","last_undegraded":"2024-09-10T10:26:48.436614+0000","last_fullsized":"2024-09-10T10:26:48.436614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831349+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:26:47.831349+0000","last_peered":"2024-09-10T10:26:47.831349+0000","last_clean":"2024-09-10T10:26:47.831349+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:26:47.831349+0000","last_undegraded":"2024-09-10T10:26:47.831349+0000","last_fullsized":"2024-09-10T10:26:47.831349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271920+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:26:48.271920+0000","last_peered":"2024-09-10T10:26:48.271920+0000","last_clean":"2024-09-10T10:26:48.271920+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:26:48.271920+0000","last_undegraded":"2024-09-10T10:26:48.271920+0000","last_fullsized":"2024-09-10T10:26:48.271920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.883137+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:27:08.883137+0000","last_peered":"2024-09-10T10:27:08.883137+0000","last_clean":"2024-09-10T10:27:08.883137+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:27:08.883137+0000","last_undegraded":"2024-09-10T10:27:08.883137+0000","last_fullsized":"2024-09-10T10:27:08.883137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826511+0000","last_change":"2024-09-10T10:26:45.819680+0000","last_active":"2024-09-10T10:26:47.826511+0000","last_peered":"2024-09-10T10:26:47.826511+0000","last_clean":"2024-09-10T10:26:47.826511+0000","last_became_active":"2024-09-10T10:26:45.819319+0000","last_became_peered":"2024-09-10T10:26:45.819319+0000","last_unstale":"2024-09-10T10:26:47.826511+0000","last_undegraded":"2024-09-10T10:26:47.826511+0000","last_fullsized":"2024-09-10T10:26:47.826511+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:43.492188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830718+0000","last_change":"2024-09-10T10:26:41.792618+0000","last_active":"2024-09-10T10:26:47.830718+0000","last_peered":"2024-09-10T10:26:47.830718+0000","last_clean":"2024-09-10T10:26:47.830718+0000","last_became_active":"2024-09-10T10:26:41.792323+0000","last_became_peered":"2024-09-10T10:26:41.792323+0000","last_unstale":"2024-09-10T10:26:47.830718+0000","last_undegraded":"2024-09-10T10:26:47.830718+0000","last_fullsized":"2024-09-10T10:26:47.830718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.878001+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:27:08.878001+0000","last_peered":"2024-09-10T10:27:08.878001+0000","last_clean":"2024-09-10T10:27:08.878001+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:27:08.878001+0000","last_undegraded":"2024-09-10T10:27:08.878001+0000","last_fullsized":"2024-09-10T10:27:08.878001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826609+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:26:47.826609+0000","last_peered":"2024-09-10T10:26:47.826609+0000","last_clean":"2024-09-10T10:26:47.826609+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:26:47.826609+0000","last_undegraded":"2024-09-10T10:26:47.826609+0000","last_fullsized":"2024-09-10T10:26:47.826609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271720+0000","last_change":"2024-09-10T10:26:46.193656+0000","last_active":"2024-09-10T10:26:48.271720+0000","last_peered":"2024-09-10T10:26:48.271720+0000","last_clean":"2024-09-10T10:26:48.271720+0000","last_became_active":"2024-09-10T10:26:46.193109+0000","last_became_peered":"2024-09-10T10:26:46.193109+0000","last_unstale":"2024-09-10T10:26:48.271720+0000","last_undegraded":"2024-09-10T10:26:48.271720+0000","last_fullsized":"2024-09-10T10:26:48.271720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436035+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:26:48.436035+0000","last_peered":"2024-09-10T10:26:48.436035+0000","last_clean":"2024-09-10T10:26:48.436035+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:26:48.436035+0000","last_undegraded":"2024-09-10T10:26:48.436035+0000","last_fullsized":"2024-09-10T10:26:48.436035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829645+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:26:47.829645+0000","last_peered":"2024-09-10T10:26:47.829645+0000","last_clean":"2024-09-10T10:26:47.829645+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:26:47.829645+0000","last_undegraded":"2024-09-10T10:26:47.829645+0000","last_fullsized":"2024-09-10T10:26:47.829645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437648+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:26:48.437648+0000","last_peered":"2024-09-10T10:26:48.437648+0000","last_clean":"2024-09-10T10:26:48.437648+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:26:48.437648+0000","last_undegraded":"2024-09-10T10:26:48.437648+0000","last_fullsized":"2024-09-10T10:26:48.437648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826450+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:26:47.826450+0000","last_peered":"2024-09-10T10:26:47.826450+0000","last_clean":"2024-09-10T10:26:47.826450+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:26:47.826450+0000","last_undegraded":"2024-09-10T10:26:47.826450+0000","last_fullsized":"2024-09-10T10:26:47.826450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826672+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:26:47.826672+0000","last_peered":"2024-09-10T10:26:47.826672+0000","last_clean":"2024-09-10T10:26:47.826672+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:26:47.826672+0000","last_undegraded":"2024-09-10T10:26:47.826672+0000","last_fullsized":"2024-09-10T10:26:47.826672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436533+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:26:48.436533+0000","last_peered":"2024-09-10T10:26:48.436533+0000","last_clean":"2024-09-10T10:26:48.436533+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:26:48.436533+0000","last_undegraded":"2024-09-10T10:26:48.436533+0000","last_fullsized":"2024-09-10T10:26:48.436533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271816+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:26:48.271816+0000","last_peered":"2024-09-10T10:26:48.271816+0000","last_clean":"2024-09-10T10:26:48.271816+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:26:48.271816+0000","last_undegraded":"2024-09-10T10:26:48.271816+0000","last_fullsized":"2024-09-10T10:26:48.271816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436252+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:26:48.436252+0000","last_peered":"2024-09-10T10:26:48.436252+0000","last_clean":"2024-09-10T10:26:48.436252+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:26:48.436252+0000","last_undegraded":"2024-09-10T10:26:48.436252+0000","last_fullsized":"2024-09-10T10:26:48.436252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"26'5","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.884118+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:27:08.884118+0000","last_peered":"2024-09-10T10:27:08.884118+0000","last_clean":"2024-09-10T10:27:08.884118+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:27:08.884118+0000","last_undegraded":"2024-09-10T10:27:08.884118+0000","last_fullsized":"2024-09-10T10:27:08.884118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830947+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:26:47.830947+0000","last_peered":"2024-09-10T10:26:47.830947+0000","last_clean":"2024-09-10T10:26:47.830947+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:26:47.830947+0000","last_undegraded":"2024-09-10T10:26:47.830947+0000","last_fullsized":"2024-09-10T10:26:47.830947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827120+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:26:47.827120+0000","last_peered":"2024-09-10T10:26:47.827120+0000","last_clean":"2024-09-10T10:26:47.827120+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:26:47.827120+0000","last_undegraded":"2024-09-10T10:26:47.827120+0000","last_fullsized":"2024-09-10T10:26:47.827120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436082+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:26:48.436082+0000","last_peered":"2024-09-10T10:26:48.436082+0000","last_clean":"2024-09-10T10:26:48.436082+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:26:48.436082+0000","last_undegraded":"2024-09-10T10:26:48.436082+0000","last_fullsized":"2024-09-10T10:26:48.436082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.435938+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:26:48.435938+0000","last_peered":"2024-09-10T10:26:48.435938+0000","last_clean":"2024-09-10T10:26:48.435938+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:26:48.435938+0000","last_undegraded":"2024-09-10T10:26:48.435938+0000","last_fullsized":"2024-09-10T10:26:48.435938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830004+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:26:47.830004+0000","last_peered":"2024-09-10T10:26:47.830004+0000","last_clean":"2024-09-10T10:26:47.830004+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:26:47.830004+0000","last_undegraded":"2024-09-10T10:26:47.830004+0000","last_fullsized":"2024-09-10T10:26:47.830004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271738+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:26:48.271738+0000","last_peered":"2024-09-10T10:26:48.271738+0000","last_clean":"2024-09-10T10:26:48.271738+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:26:48.271738+0000","last_undegraded":"2024-09-10T10:26:48.271738+0000","last_fullsized":"2024-09-10T10:26:48.271738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827172+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:26:47.827172+0000","last_peered":"2024-09-10T10:26:47.827172+0000","last_clean":"2024-09-10T10:26:47.827172+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:26:47.827172+0000","last_undegraded":"2024-09-10T10:26:47.827172+0000","last_fullsized":"2024-09-10T10:26:47.827172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436417+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:26:48.436417+0000","last_peered":"2024-09-10T10:26:48.436417+0000","last_clean":"2024-09-10T10:26:48.436417+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:26:48.436417+0000","last_undegraded":"2024-09-10T10:26:48.436417+0000","last_fullsized":"2024-09-10T10:26:48.436417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437644+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:26:48.437644+0000","last_peered":"2024-09-10T10:26:48.437644+0000","last_clean":"2024-09-10T10:26:48.437644+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:26:48.437644+0000","last_undegraded":"2024-09-10T10:26:48.437644+0000","last_fullsized":"2024-09-10T10:26:48.437644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827413+0000","last_change":"2024-09-10T10:26:41.802903+0000","last_active":"2024-09-10T10:26:47.827413+0000","last_peered":"2024-09-10T10:26:47.827413+0000","last_clean":"2024-09-10T10:26:47.827413+0000","last_became_active":"2024-09-10T10:26:41.802586+0000","last_became_peered":"2024-09-10T10:26:41.802586+0000","last_unstale":"2024-09-10T10:26:47.827413+0000","last_undegraded":"2024-09-10T10:26:47.827413+0000","last_fullsized":"2024-09-10T10:26:47.827413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829500+0000","last_change":"2024-09-10T10:26:41.800060+0000","last_active":"2024-09-10T10:26:47.829500+0000","last_peered":"2024-09-10T10:26:47.829500+0000","last_clean":"2024-09-10T10:26:47.829500+0000","last_became_active":"2024-09-10T10:26:41.799729+0000","last_became_peered":"2024-09-10T10:26:41.799729+0000","last_unstale":"2024-09-10T10:26:47.829500+0000","last_undegraded":"2024-09-10T10:26:47.829500+0000","last_fullsized":"2024-09-10T10:26:47.829500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826575+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:26:47.826575+0000","last_peered":"2024-09-10T10:26:47.826575+0000","last_clean":"2024-09-10T10:26:47.826575+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:26:47.826575+0000","last_undegraded":"2024-09-10T10:26:47.826575+0000","last_fullsized":"2024-09-10T10:26:47.826575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829948+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:26:47.829948+0000","last_peered":"2024-09-10T10:26:47.829948+0000","last_clean":"2024-09-10T10:26:47.829948+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:26:47.829948+0000","last_undegraded":"2024-09-10T10:26:47.829948+0000","last_fullsized":"2024-09-10T10:26:47.829948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271616+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:26:48.271616+0000","last_peered":"2024-09-10T10:26:48.271616+0000","last_clean":"2024-09-10T10:26:48.271616+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:26:48.271616+0000","last_undegraded":"2024-09-10T10:26:48.271616+0000","last_fullsized":"2024-09-10T10:26:48.271616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436780+0000","last_change":"2024-09-10T10:26:41.787435+0000","last_active":"2024-09-10T10:26:48.436780+0000","last_peered":"2024-09-10T10:26:48.436780+0000","last_clean":"2024-09-10T10:26:48.436780+0000","last_became_active":"2024-09-10T10:26:41.787083+0000","last_became_peered":"2024-09-10T10:26:41.787083+0000","last_unstale":"2024-09-10T10:26:48.436780+0000","last_undegraded":"2024-09-10T10:26:48.436780+0000","last_fullsized":"2024-09-10T10:26:48.436780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437419+0000","last_change":"2024-09-10T10:26:41.792587+0000","last_active":"2024-09-10T10:26:48.437419+0000","last_peered":"2024-09-10T10:26:48.437419+0000","last_clean":"2024-09-10T10:26:48.437419+0000","last_became_active":"2024-09-10T10:26:41.792320+0000","last_became_peered":"2024-09-10T10:26:41.792320+0000","last_unstale":"2024-09-10T10:26:48.437419+0000","last_undegraded":"2024-09-10T10:26:48.437419+0000","last_fullsized":"2024-09-10T10:26:48.437419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436604+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:26:48.436604+0000","last_peered":"2024-09-10T10:26:48.436604+0000","last_clean":"2024-09-10T10:26:48.436604+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:26:48.436604+0000","last_undegraded":"2024-09-10T10:26:48.436604+0000","last_fullsized":"2024-09-10T10:26:48.436604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830936+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:26:47.830936+0000","last_peered":"2024-09-10T10:26:47.830936+0000","last_clean":"2024-09-10T10:26:47.830936+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:26:47.830936+0000","last_undegraded":"2024-09-10T10:26:47.830936+0000","last_fullsized":"2024-09-10T10:26:47.830936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829378+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:26:47.829378+0000","last_peered":"2024-09-10T10:26:47.829378+0000","last_clean":"2024-09-10T10:26:47.829378+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:26:47.829378+0000","last_undegraded":"2024-09-10T10:26:47.829378+0000","last_fullsized":"2024-09-10T10:26:47.829378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436127+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:26:48.436127+0000","last_peered":"2024-09-10T10:26:48.436127+0000","last_clean":"2024-09-10T10:26:48.436127+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:26:48.436127+0000","last_undegraded":"2024-09-10T10:26:48.436127+0000","last_fullsized":"2024-09-10T10:26:48.436127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271505+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:26:48.271505+0000","last_peered":"2024-09-10T10:26:48.271505+0000","last_clean":"2024-09-10T10:26:48.271505+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:26:48.271505+0000","last_undegraded":"2024-09-10T10:26:48.271505+0000","last_fullsized":"2024-09-10T10:26:48.271505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826694+0000","last_change":"2024-09-10T10:26:41.802117+0000","last_active":"2024-09-10T10:26:47.826694+0000","last_peered":"2024-09-10T10:26:47.826694+0000","last_clean":"2024-09-10T10:26:47.826694+0000","last_became_active":"2024-09-10T10:26:41.801798+0000","last_became_peered":"2024-09-10T10:26:41.801798+0000","last_unstale":"2024-09-10T10:26:47.826694+0000","last_undegraded":"2024-09-10T10:26:47.826694+0000","last_fullsized":"2024-09-10T10:26:47.826694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831168+0000","last_change":"2024-09-10T10:26:41.791156+0000","last_active":"2024-09-10T10:26:47.831168+0000","last_peered":"2024-09-10T10:26:47.831168+0000","last_clean":"2024-09-10T10:26:47.831168+0000","last_became_active":"2024-09-10T10:26:41.790893+0000","last_became_peered":"2024-09-10T10:26:41.790893+0000","last_unstale":"2024-09-10T10:26:47.831168+0000","last_undegraded":"2024-09-10T10:26:47.831168+0000","last_fullsized":"2024-09-10T10:26:47.831168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:46.923082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437194+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:26:48.437194+0000","last_peered":"2024-09-10T10:26:48.437194+0000","last_clean":"2024-09-10T10:26:48.437194+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:26:48.437194+0000","last_undegraded":"2024-09-10T10:26:48.437194+0000","last_fullsized":"2024-09-10T10:26:48.437194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437640+0000","last_change":"2024-09-10T10:26:41.795268+0000","last_active":"2024-09-10T10:26:48.437640+0000","last_peered":"2024-09-10T10:26:48.437640+0000","last_clean":"2024-09-10T10:26:48.437640+0000","last_became_active":"2024-09-10T10:26:41.794772+0000","last_became_peered":"2024-09-10T10:26:41.794772+0000","last_unstale":"2024-09-10T10:26:48.437640+0000","last_undegraded":"2024-09-10T10:26:48.437640+0000","last_fullsized":"2024-09-10T10:26:48.437640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437511+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:26:48.437511+0000","last_peered":"2024-09-10T10:26:48.437511+0000","last_clean":"2024-09-10T10:26:48.437511+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:26:48.437511+0000","last_undegraded":"2024-09-10T10:26:48.437511+0000","last_fullsized":"2024-09-10T10:26:48.437511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826362+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:26:47.826362+0000","last_peered":"2024-09-10T10:26:47.826362+0000","last_clean":"2024-09-10T10:26:47.826362+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:26:47.826362+0000","last_undegraded":"2024-09-10T10:26:47.826362+0000","last_fullsized":"2024-09-10T10:26:47.826362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436935+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:26:48.436935+0000","last_peered":"2024-09-10T10:26:48.436935+0000","last_clean":"2024-09-10T10:26:48.436935+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:26:48.436935+0000","last_undegraded":"2024-09-10T10:26:48.436935+0000","last_fullsized":"2024-09-10T10:26:48.436935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829829+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:26:47.829829+0000","last_peered":"2024-09-10T10:26:47.829829+0000","last_clean":"2024-09-10T10:26:47.829829+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:26:47.829829+0000","last_undegraded":"2024-09-10T10:26:47.829829+0000","last_fullsized":"2024-09-10T10:26:47.829829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436704+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:26:48.436704+0000","last_peered":"2024-09-10T10:26:48.436704+0000","last_clean":"2024-09-10T10:26:48.436704+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:26:48.436704+0000","last_undegraded":"2024-09-10T10:26:48.436704+0000","last_fullsized":"2024-09-10T10:26:48.436704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829829+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:26:47.829829+0000","last_peered":"2024-09-10T10:26:47.829829+0000","last_clean":"2024-09-10T10:26:47.829829+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:26:47.829829+0000","last_undegraded":"2024-09-10T10:26:47.829829+0000","last_fullsized":"2024-09-10T10:26:47.829829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271530+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:26:48.271530+0000","last_peered":"2024-09-10T10:26:48.271530+0000","last_clean":"2024-09-10T10:26:48.271530+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:26:48.271530+0000","last_undegraded":"2024-09-10T10:26:48.271530+0000","last_fullsized":"2024-09-10T10:26:48.271530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436873+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:26:48.436873+0000","last_peered":"2024-09-10T10:26:48.436873+0000","last_clean":"2024-09-10T10:26:48.436873+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:26:48.436873+0000","last_undegraded":"2024-09-10T10:26:48.436873+0000","last_fullsized":"2024-09-10T10:26:48.436873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437504+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:26:48.437504+0000","last_peered":"2024-09-10T10:26:48.437504+0000","last_clean":"2024-09-10T10:26:48.437504+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:26:48.437504+0000","last_undegraded":"2024-09-10T10:26:48.437504+0000","last_fullsized":"2024-09-10T10:26:48.437504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6048,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":703748,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":115908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":116530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":356352,"data_stored":703070,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:27:12.647 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:27:12.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.779+0000 7f42493cd640 1 Processor -- start 2024-09-10T10:27:12.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.779+0000 7f42493cd640 1 -- start start 2024-09-10T10:27:12.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f42493cd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4244152d60 0x7f42440c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f42493cd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f42440c8df0 0x7f4244159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f42493cd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4244159560 0x7f42440c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f42493cd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f424415cdb0 con 0x7f4244152d60 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f42493cd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f42440cf3c0 con 0x7f42440c8df0 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f42493cd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4244158b90 con 0x7f4244159560 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f4242ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4244152d60 0x7f42440c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f4242ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4244152d60 0x7f42440c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34750/0 (socket says 172.21.15.120:34750) 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f4242ffd640 1 -- 172.21.15.120:0/3922163384 learned_addr learned my addr 172.21.15.120:0/3922163384 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f42437fe640 1 --2- 172.21.15.120:0/3922163384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4244159560 0x7f42440c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.780+0000 7f42427fc640 1 --2- 172.21.15.120:0/3922163384 >> v2:172.21.15.134:3300/0 conn(0x7f42440c8df0 0x7f4244159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.781+0000 7f4242ffd640 1 -- 172.21.15.120:0/3922163384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4244159560 msgr2=0x7f42440c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.781+0000 7f4242ffd640 1 --2- 172.21.15.120:0/3922163384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4244159560 0x7f42440c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.781+0000 7f4242ffd640 1 -- 172.21.15.120:0/3922163384 >> v2:172.21.15.134:3300/0 conn(0x7f42440c8df0 msgr2=0x7f4244159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.781+0000 7f4242ffd640 1 --2- 172.21.15.120:0/3922163384 >> v2:172.21.15.134:3300/0 conn(0x7f42440c8df0 0x7f4244159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.781+0000 7f4242ffd640 1 -- 172.21.15.120:0/3922163384 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424415c230 con 0x7f4244152d60 2024-09-10T10:27:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.781+0000 7f42427fc640 1 --2- 172.21.15.120:0/3922163384 >> v2:172.21.15.134:3300/0 conn(0x7f42440c8df0 0x7f4244159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.781+0000 7f4242ffd640 1 --2- 172.21.15.120:0/3922163384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4244152d60 0x7f42440c3410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f42440c8ab0 tx=0x7f422c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ec607b6bb6d42e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.781+0000 7f4241ffb640 1 -- 172.21.15.120:0/3922163384 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c037070 con 0x7f4244152d60 2024-09-10T10:27:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.781+0000 7f4241ffb640 1 -- 172.21.15.120:0/3922163384 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f422c032440 con 0x7f4244152d60 2024-09-10T10:27:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.781+0000 7f4241ffb640 1 -- 172.21.15.120:0/3922163384 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c032810 con 0x7f4244152d60 2024-09-10T10:27:12.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.782+0000 7f42493cd640 1 -- 172.21.15.120:0/3922163384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4244152d60 msgr2=0x7f42440c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.782+0000 7f42493cd640 1 --2- 172.21.15.120:0/3922163384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4244152d60 0x7f42440c3410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f42440c8ab0 tx=0x7f422c02a270 comp rx=0 tx=0).stop 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.782+0000 7f42493cd640 1 -- 172.21.15.120:0/3922163384 shutdown_connections 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.782+0000 7f42493cd640 1 --2- 172.21.15.120:0/3922163384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4244159560 0x7f42440c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.782+0000 7f42493cd640 1 --2- 172.21.15.120:0/3922163384 >> v2:172.21.15.134:3300/0 conn(0x7f42440c8df0 0x7f4244159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.782+0000 7f42493cd640 1 --2- 172.21.15.120:0/3922163384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4244152d60 0x7f42440c3410 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.782+0000 7f42493cd640 1 -- 172.21.15.120:0/3922163384 >> 172.21.15.120:0/3922163384 conn(0x7f42440716e0 msgr2=0x7f4244071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.782+0000 7f42493cd640 1 -- 172.21.15.120:0/3922163384 shutdown_connections 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.782+0000 7f42493cd640 1 -- 172.21.15.120:0/3922163384 wait complete. 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.783+0000 7f42493cd640 1 Processor -- start 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.783+0000 7f42493cd640 1 -- start start 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f42493cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42440c8df0 0x7f42441f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f42493cd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4244152d60 0x7f424414aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f42493cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4244159560 0x7f424414afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f42493cd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f424415cad0 con 0x7f42440c8df0 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f42493cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f424415c7f0 con 0x7f4244159560 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f42493cd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f424415be60 con 0x7f4244152d60 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42440c8df0 0x7f42441f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42440c8df0 0x7f42441f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34764/0 (socket says 172.21.15.120:34764) 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f4242ffd640 1 -- 172.21.15.120:0/289485099 learned_addr learned my addr 172.21.15.120:0/289485099 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f42427fc640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4244152d60 0x7f424414aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.784+0000 7f42437fe640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4244159560 0x7f424414afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f4242ffd640 1 -- 172.21.15.120:0/289485099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4244152d60 msgr2=0x7f424414aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f4242ffd640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4244152d60 0x7f424414aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f4242ffd640 1 -- 172.21.15.120:0/289485099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4244159560 msgr2=0x7f424414afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f4242ffd640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4244159560 0x7f424414afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f4242ffd640 1 -- 172.21.15.120:0/289485099 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424414e3d0 con 0x7f42440c8df0 2024-09-10T10:27:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f4242ffd640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42440c8df0 0x7f42441f47d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f422c000c00 tx=0x7f422c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f423b7fe640 1 -- 172.21.15.120:0/289485099 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c041070 con 0x7f42440c8df0 2024-09-10T10:27:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f423b7fe640 1 -- 172.21.15.120:0/289485099 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f422c03c870 con 0x7f42440c8df0 2024-09-10T10:27:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f42493cd640 1 -- 172.21.15.120:0/289485099 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424414e6b0 con 0x7f42440c8df0 2024-09-10T10:27:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f423b7fe640 1 -- 172.21.15.120:0/289485099 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c037040 con 0x7f42440c8df0 2024-09-10T10:27:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.785+0000 7f42493cd640 1 -- 172.21.15.120:0/289485099 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42441f3cf0 con 0x7f42440c8df0 2024-09-10T10:27:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.786+0000 7f42493cd640 1 -- 172.21.15.120:0/289485099 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4244159020 con 0x7f42440c8df0 2024-09-10T10:27:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.787+0000 7f423b7fe640 1 -- 172.21.15.120:0/289485099 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f422c044070 con 0x7f42440c8df0 2024-09-10T10:27:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.787+0000 7f423b7fe640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f422406be30 0x7f422406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.787+0000 7f423b7fe640 1 -- 172.21.15.120:0/289485099 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f422c0abe80 con 0x7f42440c8df0 2024-09-10T10:27:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.787+0000 7f42427fc640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f422406be30 0x7f422406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.788+0000 7f42427fc640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f422406be30 0x7f422406e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f424414b790 tx=0x7f4234008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:12.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.791+0000 7f423b7fe640 1 -- 172.21.15.120:0/289485099 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f422c079f30 con 0x7f42440c8df0 2024-09-10T10:27:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.957+0000 7f42493cd640 1 -- 172.21.15.120:0/289485099 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f424406e8f0 con 0x7f422406be30 2024-09-10T10:27:12.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.965+0000 7f423b7fe640 1 -- 172.21.15.120:0/289485099 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332761 (secure 0 0 0) 0x7f424406e8f0 con 0x7f422406be30 2024-09-10T10:27:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:12.972 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:27:12.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 -- 172.21.15.120:0/289485099 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f422406be30 msgr2=0x7f422406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f422406be30 0x7f422406e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f424414b790 tx=0x7f4234008040 comp rx=0 tx=0).stop 2024-09-10T10:27:12.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 -- 172.21.15.120:0/289485099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42440c8df0 msgr2=0x7f42441f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:12.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42440c8df0 0x7f42441f47d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f422c000c00 tx=0x7f422c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 -- 172.21.15.120:0/289485099 shutdown_connections 2024-09-10T10:27:12.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f422406be30 0x7f422406e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4244159560 0x7f424414afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4244152d60 0x7f424414aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 --2- 172.21.15.120:0/289485099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42440c8df0 0x7f42441f47d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:12.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 -- 172.21.15.120:0/289485099 >> 172.21.15.120:0/289485099 conn(0x7f42440716e0 msgr2=0x7f42441580e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:12.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 -- 172.21.15.120:0/289485099 shutdown_connections 2024-09-10T10:27:12.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:12.969+0000 7f42493cd640 1 -- 172.21.15.120:0/289485099 wait complete. 2024-09-10T10:27:12.981 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-10T10:27:12.096828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596416,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":246,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193,"ondisk_log_size":193,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218636,"kb_used_data":2392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756084,"statfs":{"total":773094113280,"available":772870230016,"internally_reserved":0,"allocated":2449408,"data_stored":2087888,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003015"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271924+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:26:48.271924+0000","last_peered":"2024-09-10T10:26:48.271924+0000","last_clean":"2024-09-10T10:26:48.271924+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:26:48.271924+0000","last_undegraded":"2024-09-10T10:26:48.271924+0000","last_fullsized":"2024-09-10T10:26:48.271924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827265+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:26:47.827265+0000","last_peered":"2024-09-10T10:26:47.827265+0000","last_clean":"2024-09-10T10:26:47.827265+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:26:47.827265+0000","last_undegraded":"2024-09-10T10:26:47.827265+0000","last_fullsized":"2024-09-10T10:26:47.827265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826403+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:26:47.826403+0000","last_peered":"2024-09-10T10:26:47.826403+0000","last_clean":"2024-09-10T10:26:47.826403+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:26:47.826403+0000","last_undegraded":"2024-09-10T10:26:47.826403+0000","last_fullsized":"2024-09-10T10:26:47.826403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827328+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:26:47.827328+0000","last_peered":"2024-09-10T10:26:47.827328+0000","last_clean":"2024-09-10T10:26:47.827328+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:26:47.827328+0000","last_undegraded":"2024-09-10T10:26:47.827328+0000","last_fullsized":"2024-09-10T10:26:47.827328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271728+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:26:48.271728+0000","last_peered":"2024-09-10T10:26:48.271728+0000","last_clean":"2024-09-10T10:26:48.271728+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:26:48.271728+0000","last_undegraded":"2024-09-10T10:26:48.271728+0000","last_fullsized":"2024-09-10T10:26:48.271728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829940+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:26:47.829940+0000","last_peered":"2024-09-10T10:26:47.829940+0000","last_clean":"2024-09-10T10:26:47.829940+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:26:47.829940+0000","last_undegraded":"2024-09-10T10:26:47.829940+0000","last_fullsized":"2024-09-10T10:26:47.829940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437245+0000","last_change":"2024-09-10T10:26:40.796337+0000","last_active":"2024-09-10T10:26:48.437245+0000","last_peered":"2024-09-10T10:26:48.437245+0000","last_clean":"2024-09-10T10:26:48.437245+0000","last_became_active":"2024-09-10T10:26:40.796029+0000","last_became_peered":"2024-09-10T10:26:40.796029+0000","last_unstale":"2024-09-10T10:26:48.437245+0000","last_undegraded":"2024-09-10T10:26:48.437245+0000","last_fullsized":"2024-09-10T10:26:48.437245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829990+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:26:47.829990+0000","last_peered":"2024-09-10T10:26:47.829990+0000","last_clean":"2024-09-10T10:26:47.829990+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:26:47.829990+0000","last_undegraded":"2024-09-10T10:26:47.829990+0000","last_fullsized":"2024-09-10T10:26:47.829990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831277+0000","last_change":"2024-09-10T10:26:45.814368+0000","last_active":"2024-09-10T10:26:47.831277+0000","last_peered":"2024-09-10T10:26:47.831277+0000","last_clean":"2024-09-10T10:26:47.831277+0000","last_became_active":"2024-09-10T10:26:45.813953+0000","last_became_peered":"2024-09-10T10:26:45.813953+0000","last_unstale":"2024-09-10T10:26:47.831277+0000","last_undegraded":"2024-09-10T10:26:47.831277+0000","last_fullsized":"2024-09-10T10:26:47.831277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436841+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:26:48.436841+0000","last_peered":"2024-09-10T10:26:48.436841+0000","last_clean":"2024-09-10T10:26:48.436841+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:26:48.436841+0000","last_undegraded":"2024-09-10T10:26:48.436841+0000","last_fullsized":"2024-09-10T10:26:48.436841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830425+0000","last_change":"2024-09-10T10:26:41.789145+0000","last_active":"2024-09-10T10:26:47.830425+0000","last_peered":"2024-09-10T10:26:47.830425+0000","last_clean":"2024-09-10T10:26:47.830425+0000","last_became_active":"2024-09-10T10:26:41.788874+0000","last_became_peered":"2024-09-10T10:26:41.788874+0000","last_unstale":"2024-09-10T10:26:47.830425+0000","last_undegraded":"2024-09-10T10:26:47.830425+0000","last_fullsized":"2024-09-10T10:26:47.830425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830738+0000","last_change":"2024-09-10T10:26:45.816587+0000","last_active":"2024-09-10T10:26:47.830738+0000","last_peered":"2024-09-10T10:26:47.830738+0000","last_clean":"2024-09-10T10:26:47.830738+0000","last_became_active":"2024-09-10T10:26:45.816138+0000","last_became_peered":"2024-09-10T10:26:45.816138+0000","last_unstale":"2024-09-10T10:26:47.830738+0000","last_undegraded":"2024-09-10T10:26:47.830738+0000","last_fullsized":"2024-09-10T10:26:47.830738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436252+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:26:48.436252+0000","last_peered":"2024-09-10T10:26:48.436252+0000","last_clean":"2024-09-10T10:26:48.436252+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:26:48.436252+0000","last_undegraded":"2024-09-10T10:26:48.436252+0000","last_fullsized":"2024-09-10T10:26:48.436252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.875798+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:27:08.875798+0000","last_peered":"2024-09-10T10:27:08.875798+0000","last_clean":"2024-09-10T10:27:08.875798+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:27:08.875798+0000","last_undegraded":"2024-09-10T10:27:08.875798+0000","last_fullsized":"2024-09-10T10:27:08.875798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831602+0000","last_change":"2024-09-10T10:26:45.816967+0000","last_active":"2024-09-10T10:26:47.831602+0000","last_peered":"2024-09-10T10:26:47.831602+0000","last_clean":"2024-09-10T10:26:47.831602+0000","last_became_active":"2024-09-10T10:26:45.816161+0000","last_became_peered":"2024-09-10T10:26:45.816161+0000","last_unstale":"2024-09-10T10:26:47.831602+0000","last_undegraded":"2024-09-10T10:26:47.831602+0000","last_fullsized":"2024-09-10T10:26:47.831602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826732+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:26:47.826732+0000","last_peered":"2024-09-10T10:26:47.826732+0000","last_clean":"2024-09-10T10:26:47.826732+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:26:47.826732+0000","last_undegraded":"2024-09-10T10:26:47.826732+0000","last_fullsized":"2024-09-10T10:26:47.826732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436716+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:26:48.436716+0000","last_peered":"2024-09-10T10:26:48.436716+0000","last_clean":"2024-09-10T10:26:48.436716+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:26:48.436716+0000","last_undegraded":"2024-09-10T10:26:48.436716+0000","last_fullsized":"2024-09-10T10:26:48.436716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830916+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:26:47.830916+0000","last_peered":"2024-09-10T10:26:47.830916+0000","last_clean":"2024-09-10T10:26:47.830916+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:26:47.830916+0000","last_undegraded":"2024-09-10T10:26:47.830916+0000","last_fullsized":"2024-09-10T10:26:47.830916+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826532+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:26:47.826532+0000","last_peered":"2024-09-10T10:26:47.826532+0000","last_clean":"2024-09-10T10:26:47.826532+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:26:47.826532+0000","last_undegraded":"2024-09-10T10:26:47.826532+0000","last_fullsized":"2024-09-10T10:26:47.826532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.883285+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:27:08.883285+0000","last_peered":"2024-09-10T10:27:08.883285+0000","last_clean":"2024-09-10T10:27:08.883285+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:27:08.883285+0000","last_undegraded":"2024-09-10T10:27:08.883285+0000","last_fullsized":"2024-09-10T10:27:08.883285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436960+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:26:48.436960+0000","last_peered":"2024-09-10T10:26:48.436960+0000","last_clean":"2024-09-10T10:26:48.436960+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:26:48.436960+0000","last_undegraded":"2024-09-10T10:26:48.436960+0000","last_fullsized":"2024-09-10T10:26:48.436960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830283+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:26:47.830283+0000","last_peered":"2024-09-10T10:26:47.830283+0000","last_clean":"2024-09-10T10:26:47.830283+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:26:47.830283+0000","last_undegraded":"2024-09-10T10:26:47.830283+0000","last_fullsized":"2024-09-10T10:26:47.830283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.876443+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:27:08.876443+0000","last_peered":"2024-09-10T10:27:08.876443+0000","last_clean":"2024-09-10T10:27:08.876443+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:27:08.876443+0000","last_undegraded":"2024-09-10T10:27:08.876443+0000","last_fullsized":"2024-09-10T10:27:08.876443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830167+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:26:47.830167+0000","last_peered":"2024-09-10T10:26:47.830167+0000","last_clean":"2024-09-10T10:26:47.830167+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:26:47.830167+0000","last_undegraded":"2024-09-10T10:26:47.830167+0000","last_fullsized":"2024-09-10T10:26:47.830167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271909+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:26:48.271909+0000","last_peered":"2024-09-10T10:26:48.271909+0000","last_clean":"2024-09-10T10:26:48.271909+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:26:48.271909+0000","last_undegraded":"2024-09-10T10:26:48.271909+0000","last_fullsized":"2024-09-10T10:26:48.271909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.876078+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:27:08.876078+0000","last_peered":"2024-09-10T10:27:08.876078+0000","last_clean":"2024-09-10T10:27:08.876078+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:27:08.876078+0000","last_undegraded":"2024-09-10T10:27:08.876078+0000","last_fullsized":"2024-09-10T10:27:08.876078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829937+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:26:47.829937+0000","last_peered":"2024-09-10T10:26:47.829937+0000","last_clean":"2024-09-10T10:26:47.829937+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:26:47.829937+0000","last_undegraded":"2024-09-10T10:26:47.829937+0000","last_fullsized":"2024-09-10T10:26:47.829937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827469+0000","last_change":"2024-09-10T10:26:41.795085+0000","last_active":"2024-09-10T10:26:47.827469+0000","last_peered":"2024-09-10T10:26:47.827469+0000","last_clean":"2024-09-10T10:26:47.827469+0000","last_became_active":"2024-09-10T10:26:41.794819+0000","last_became_peered":"2024-09-10T10:26:41.794819+0000","last_unstale":"2024-09-10T10:26:47.827469+0000","last_undegraded":"2024-09-10T10:26:47.827469+0000","last_fullsized":"2024-09-10T10:26:47.827469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436983+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:26:48.436983+0000","last_peered":"2024-09-10T10:26:48.436983+0000","last_clean":"2024-09-10T10:26:48.436983+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:26:48.436983+0000","last_undegraded":"2024-09-10T10:26:48.436983+0000","last_fullsized":"2024-09-10T10:26:48.436983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826343+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:26:47.826343+0000","last_peered":"2024-09-10T10:26:47.826343+0000","last_clean":"2024-09-10T10:26:47.826343+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:26:47.826343+0000","last_undegraded":"2024-09-10T10:26:47.826343+0000","last_fullsized":"2024-09-10T10:26:47.826343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436707+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:26:48.436707+0000","last_peered":"2024-09-10T10:26:48.436707+0000","last_clean":"2024-09-10T10:26:48.436707+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:26:48.436707+0000","last_undegraded":"2024-09-10T10:26:48.436707+0000","last_fullsized":"2024-09-10T10:26:48.436707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437443+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:26:48.437443+0000","last_peered":"2024-09-10T10:26:48.437443+0000","last_clean":"2024-09-10T10:26:48.437443+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:26:48.437443+0000","last_undegraded":"2024-09-10T10:26:48.437443+0000","last_fullsized":"2024-09-10T10:26:48.437443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271566+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:26:48.271566+0000","last_peered":"2024-09-10T10:26:48.271566+0000","last_clean":"2024-09-10T10:26:48.271566+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:26:48.271566+0000","last_undegraded":"2024-09-10T10:26:48.271566+0000","last_fullsized":"2024-09-10T10:26:48.271566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830633+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:26:47.830633+0000","last_peered":"2024-09-10T10:26:47.830633+0000","last_clean":"2024-09-10T10:26:47.830633+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:26:47.830633+0000","last_undegraded":"2024-09-10T10:26:47.830633+0000","last_fullsized":"2024-09-10T10:26:47.830633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436761+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:26:48.436761+0000","last_peered":"2024-09-10T10:26:48.436761+0000","last_clean":"2024-09-10T10:26:48.436761+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:26:48.436761+0000","last_undegraded":"2024-09-10T10:26:48.436761+0000","last_fullsized":"2024-09-10T10:26:48.436761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436477+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:26:48.436477+0000","last_peered":"2024-09-10T10:26:48.436477+0000","last_clean":"2024-09-10T10:26:48.436477+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:26:48.436477+0000","last_undegraded":"2024-09-10T10:26:48.436477+0000","last_fullsized":"2024-09-10T10:26:48.436477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831395+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:26:47.831395+0000","last_peered":"2024-09-10T10:26:47.831395+0000","last_clean":"2024-09-10T10:26:47.831395+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:26:47.831395+0000","last_undegraded":"2024-09-10T10:26:47.831395+0000","last_fullsized":"2024-09-10T10:26:47.831395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830140+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:26:47.830140+0000","last_peered":"2024-09-10T10:26:47.830140+0000","last_clean":"2024-09-10T10:26:47.830140+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:26:47.830140+0000","last_undegraded":"2024-09-10T10:26:47.830140+0000","last_fullsized":"2024-09-10T10:26:47.830140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436656+0000","last_change":"2024-09-10T10:26:45.814008+0000","last_active":"2024-09-10T10:26:48.436656+0000","last_peered":"2024-09-10T10:26:48.436656+0000","last_clean":"2024-09-10T10:26:48.436656+0000","last_became_active":"2024-09-10T10:26:45.813627+0000","last_became_peered":"2024-09-10T10:26:45.813627+0000","last_unstale":"2024-09-10T10:26:48.436656+0000","last_undegraded":"2024-09-10T10:26:48.436656+0000","last_fullsized":"2024-09-10T10:26:48.436656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436224+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:26:48.436224+0000","last_peered":"2024-09-10T10:26:48.436224+0000","last_clean":"2024-09-10T10:26:48.436224+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:26:48.436224+0000","last_undegraded":"2024-09-10T10:26:48.436224+0000","last_fullsized":"2024-09-10T10:26:48.436224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436077+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:26:48.436077+0000","last_peered":"2024-09-10T10:26:48.436077+0000","last_clean":"2024-09-10T10:26:48.436077+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:26:48.436077+0000","last_undegraded":"2024-09-10T10:26:48.436077+0000","last_fullsized":"2024-09-10T10:26:48.436077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436987+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:26:48.436987+0000","last_peered":"2024-09-10T10:26:48.436987+0000","last_clean":"2024-09-10T10:26:48.436987+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:26:48.436987+0000","last_undegraded":"2024-09-10T10:26:48.436987+0000","last_fullsized":"2024-09-10T10:26:48.436987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271316+0000","last_change":"2024-09-10T10:26:41.799224+0000","last_active":"2024-09-10T10:26:48.271316+0000","last_peered":"2024-09-10T10:26:48.271316+0000","last_clean":"2024-09-10T10:26:48.271316+0000","last_became_active":"2024-09-10T10:26:41.798948+0000","last_became_peered":"2024-09-10T10:26:41.798948+0000","last_unstale":"2024-09-10T10:26:48.271316+0000","last_undegraded":"2024-09-10T10:26:48.271316+0000","last_fullsized":"2024-09-10T10:26:48.271316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437014+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:26:48.437014+0000","last_peered":"2024-09-10T10:26:48.437014+0000","last_clean":"2024-09-10T10:26:48.437014+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:26:48.437014+0000","last_undegraded":"2024-09-10T10:26:48.437014+0000","last_fullsized":"2024-09-10T10:26:48.437014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271901+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:26:48.271901+0000","last_peered":"2024-09-10T10:26:48.271901+0000","last_clean":"2024-09-10T10:26:48.271901+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:26:48.271901+0000","last_undegraded":"2024-09-10T10:26:48.271901+0000","last_fullsized":"2024-09-10T10:26:48.271901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826634+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:26:47.826634+0000","last_peered":"2024-09-10T10:26:47.826634+0000","last_clean":"2024-09-10T10:26:47.826634+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:26:47.826634+0000","last_undegraded":"2024-09-10T10:26:47.826634+0000","last_fullsized":"2024-09-10T10:26:47.826634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.877722+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:27:08.877722+0000","last_peered":"2024-09-10T10:27:08.877722+0000","last_clean":"2024-09-10T10:27:08.877722+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:27:08.877722+0000","last_undegraded":"2024-09-10T10:27:08.877722+0000","last_fullsized":"2024-09-10T10:27:08.877722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437071+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:26:48.437071+0000","last_peered":"2024-09-10T10:26:48.437071+0000","last_clean":"2024-09-10T10:26:48.437071+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:26:48.437071+0000","last_undegraded":"2024-09-10T10:26:48.437071+0000","last_fullsized":"2024-09-10T10:26:48.437071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437061+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:26:48.437061+0000","last_peered":"2024-09-10T10:26:48.437061+0000","last_clean":"2024-09-10T10:26:48.437061+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:26:48.437061+0000","last_undegraded":"2024-09-10T10:26:48.437061+0000","last_fullsized":"2024-09-10T10:26:48.437061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437121+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:26:48.437121+0000","last_peered":"2024-09-10T10:26:48.437121+0000","last_clean":"2024-09-10T10:26:48.437121+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:26:48.437121+0000","last_undegraded":"2024-09-10T10:26:48.437121+0000","last_fullsized":"2024-09-10T10:26:48.437121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436307+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:26:48.436307+0000","last_peered":"2024-09-10T10:26:48.436307+0000","last_clean":"2024-09-10T10:26:48.436307+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:26:48.436307+0000","last_undegraded":"2024-09-10T10:26:48.436307+0000","last_fullsized":"2024-09-10T10:26:48.436307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436298+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:26:48.436298+0000","last_peered":"2024-09-10T10:26:48.436298+0000","last_clean":"2024-09-10T10:26:48.436298+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:26:48.436298+0000","last_undegraded":"2024-09-10T10:26:48.436298+0000","last_fullsized":"2024-09-10T10:26:48.436298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830079+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:26:47.830079+0000","last_peered":"2024-09-10T10:26:47.830079+0000","last_clean":"2024-09-10T10:26:47.830079+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:26:47.830079+0000","last_undegraded":"2024-09-10T10:26:47.830079+0000","last_fullsized":"2024-09-10T10:26:47.830079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436713+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:26:48.436713+0000","last_peered":"2024-09-10T10:26:48.436713+0000","last_clean":"2024-09-10T10:26:48.436713+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:26:48.436713+0000","last_undegraded":"2024-09-10T10:26:48.436713+0000","last_fullsized":"2024-09-10T10:26:48.436713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271787+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:26:48.271787+0000","last_peered":"2024-09-10T10:26:48.271787+0000","last_clean":"2024-09-10T10:26:48.271787+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:26:48.271787+0000","last_undegraded":"2024-09-10T10:26:48.271787+0000","last_fullsized":"2024-09-10T10:26:48.271787+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437650+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:26:48.437650+0000","last_peered":"2024-09-10T10:26:48.437650+0000","last_clean":"2024-09-10T10:26:48.437650+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:26:48.437650+0000","last_undegraded":"2024-09-10T10:26:48.437650+0000","last_fullsized":"2024-09-10T10:26:48.437650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436651+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:26:48.436651+0000","last_peered":"2024-09-10T10:26:48.436651+0000","last_clean":"2024-09-10T10:26:48.436651+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:26:48.436651+0000","last_undegraded":"2024-09-10T10:26:48.436651+0000","last_fullsized":"2024-09-10T10:26:48.436651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829425+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:26:47.829425+0000","last_peered":"2024-09-10T10:26:47.829425+0000","last_clean":"2024-09-10T10:26:47.829425+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:26:47.829425+0000","last_undegraded":"2024-09-10T10:26:47.829425+0000","last_fullsized":"2024-09-10T10:26:47.829425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437218+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:26:48.437218+0000","last_peered":"2024-09-10T10:26:48.437218+0000","last_clean":"2024-09-10T10:26:48.437218+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:26:48.437218+0000","last_undegraded":"2024-09-10T10:26:48.437218+0000","last_fullsized":"2024-09-10T10:26:48.437218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830430+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:26:47.830430+0000","last_peered":"2024-09-10T10:26:47.830430+0000","last_clean":"2024-09-10T10:26:47.830430+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:26:47.830430+0000","last_undegraded":"2024-09-10T10:26:47.830430+0000","last_fullsized":"2024-09-10T10:26:47.830430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829765+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:26:47.829765+0000","last_peered":"2024-09-10T10:26:47.829765+0000","last_clean":"2024-09-10T10:26:47.829765+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:26:47.829765+0000","last_undegraded":"2024-09-10T10:26:47.829765+0000","last_fullsized":"2024-09-10T10:26:47.829765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437037+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:26:48.437037+0000","last_peered":"2024-09-10T10:26:48.437037+0000","last_clean":"2024-09-10T10:26:48.437037+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:26:48.437037+0000","last_undegraded":"2024-09-10T10:26:48.437037+0000","last_fullsized":"2024-09-10T10:26:48.437037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'10","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.878414+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:27:08.878414+0000","last_peered":"2024-09-10T10:27:08.878414+0000","last_clean":"2024-09-10T10:27:08.878414+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:27:08.878414+0000","last_undegraded":"2024-09-10T10:27:08.878414+0000","last_fullsized":"2024-09-10T10:27:08.878414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271756+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:26:48.271756+0000","last_peered":"2024-09-10T10:26:48.271756+0000","last_clean":"2024-09-10T10:26:48.271756+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:26:48.271756+0000","last_undegraded":"2024-09-10T10:26:48.271756+0000","last_fullsized":"2024-09-10T10:26:48.271756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436218+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:26:48.436218+0000","last_peered":"2024-09-10T10:26:48.436218+0000","last_clean":"2024-09-10T10:26:48.436218+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:26:48.436218+0000","last_undegraded":"2024-09-10T10:26:48.436218+0000","last_fullsized":"2024-09-10T10:26:48.436218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436280+0000","last_change":"2024-09-10T10:26:40.794739+0000","last_active":"2024-09-10T10:26:48.436280+0000","last_peered":"2024-09-10T10:26:48.436280+0000","last_clean":"2024-09-10T10:26:48.436280+0000","last_became_active":"2024-09-10T10:26:40.794430+0000","last_became_peered":"2024-09-10T10:26:40.794430+0000","last_unstale":"2024-09-10T10:26:48.436280+0000","last_undegraded":"2024-09-10T10:26:48.436280+0000","last_fullsized":"2024-09-10T10:26:48.436280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827508+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:26:47.827508+0000","last_peered":"2024-09-10T10:26:47.827508+0000","last_clean":"2024-09-10T10:26:47.827508+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:26:47.827508+0000","last_undegraded":"2024-09-10T10:26:47.827508+0000","last_fullsized":"2024-09-10T10:26:47.827508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436803+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:26:48.436803+0000","last_peered":"2024-09-10T10:26:48.436803+0000","last_clean":"2024-09-10T10:26:48.436803+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:26:48.436803+0000","last_undegraded":"2024-09-10T10:26:48.436803+0000","last_fullsized":"2024-09-10T10:26:48.436803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.879564+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:27:08.879564+0000","last_peered":"2024-09-10T10:27:08.879564+0000","last_clean":"2024-09-10T10:27:08.879564+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:27:08.879564+0000","last_undegraded":"2024-09-10T10:27:08.879564+0000","last_fullsized":"2024-09-10T10:27:08.879564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827010+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:26:47.827010+0000","last_peered":"2024-09-10T10:26:47.827010+0000","last_clean":"2024-09-10T10:26:47.827010+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:26:47.827010+0000","last_undegraded":"2024-09-10T10:26:47.827010+0000","last_fullsized":"2024-09-10T10:26:47.827010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271524+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:26:48.271524+0000","last_peered":"2024-09-10T10:26:48.271524+0000","last_clean":"2024-09-10T10:26:48.271524+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:26:48.271524+0000","last_undegraded":"2024-09-10T10:26:48.271524+0000","last_fullsized":"2024-09-10T10:26:48.271524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.877837+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:27:08.877837+0000","last_peered":"2024-09-10T10:27:08.877837+0000","last_clean":"2024-09-10T10:27:08.877837+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:27:08.877837+0000","last_undegraded":"2024-09-10T10:27:08.877837+0000","last_fullsized":"2024-09-10T10:27:08.877837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830480+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:26:47.830480+0000","last_peered":"2024-09-10T10:26:47.830480+0000","last_clean":"2024-09-10T10:26:47.830480+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:26:47.830480+0000","last_undegraded":"2024-09-10T10:26:47.830480+0000","last_fullsized":"2024-09-10T10:26:47.830480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436991+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:26:48.436991+0000","last_peered":"2024-09-10T10:26:48.436991+0000","last_clean":"2024-09-10T10:26:48.436991+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:26:48.436991+0000","last_undegraded":"2024-09-10T10:26:48.436991+0000","last_fullsized":"2024-09-10T10:26:48.436991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'11","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.877358+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:27:08.877358+0000","last_peered":"2024-09-10T10:27:08.877358+0000","last_clean":"2024-09-10T10:27:08.877358+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:27:08.877358+0000","last_undegraded":"2024-09-10T10:27:08.877358+0000","last_fullsized":"2024-09-10T10:27:08.877358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436702+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:26:48.436702+0000","last_peered":"2024-09-10T10:26:48.436702+0000","last_clean":"2024-09-10T10:26:48.436702+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:26:48.436702+0000","last_undegraded":"2024-09-10T10:26:48.436702+0000","last_fullsized":"2024-09-10T10:26:48.436702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831260+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:26:47.831260+0000","last_peered":"2024-09-10T10:26:47.831260+0000","last_clean":"2024-09-10T10:26:47.831260+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:26:47.831260+0000","last_undegraded":"2024-09-10T10:26:47.831260+0000","last_fullsized":"2024-09-10T10:26:47.831260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436614+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:26:48.436614+0000","last_peered":"2024-09-10T10:26:48.436614+0000","last_clean":"2024-09-10T10:26:48.436614+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:26:48.436614+0000","last_undegraded":"2024-09-10T10:26:48.436614+0000","last_fullsized":"2024-09-10T10:26:48.436614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831349+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:26:47.831349+0000","last_peered":"2024-09-10T10:26:47.831349+0000","last_clean":"2024-09-10T10:26:47.831349+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:26:47.831349+0000","last_undegraded":"2024-09-10T10:26:47.831349+0000","last_fullsized":"2024-09-10T10:26:47.831349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271920+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:26:48.271920+0000","last_peered":"2024-09-10T10:26:48.271920+0000","last_clean":"2024-09-10T10:26:48.271920+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:26:48.271920+0000","last_undegraded":"2024-09-10T10:26:48.271920+0000","last_fullsized":"2024-09-10T10:26:48.271920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.883137+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:27:08.883137+0000","last_peered":"2024-09-10T10:27:08.883137+0000","last_clean":"2024-09-10T10:27:08.883137+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:27:08.883137+0000","last_undegraded":"2024-09-10T10:27:08.883137+0000","last_fullsized":"2024-09-10T10:27:08.883137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826511+0000","last_change":"2024-09-10T10:26:45.819680+0000","last_active":"2024-09-10T10:26:47.826511+0000","last_peered":"2024-09-10T10:26:47.826511+0000","last_clean":"2024-09-10T10:26:47.826511+0000","last_became_active":"2024-09-10T10:26:45.819319+0000","last_became_peered":"2024-09-10T10:26:45.819319+0000","last_unstale":"2024-09-10T10:26:47.826511+0000","last_undegraded":"2024-09-10T10:26:47.826511+0000","last_fullsized":"2024-09-10T10:26:47.826511+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:43.492188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830718+0000","last_change":"2024-09-10T10:26:41.792618+0000","last_active":"2024-09-10T10:26:47.830718+0000","last_peered":"2024-09-10T10:26:47.830718+0000","last_clean":"2024-09-10T10:26:47.830718+0000","last_became_active":"2024-09-10T10:26:41.792323+0000","last_became_peered":"2024-09-10T10:26:41.792323+0000","last_unstale":"2024-09-10T10:26:47.830718+0000","last_undegraded":"2024-09-10T10:26:47.830718+0000","last_fullsized":"2024-09-10T10:26:47.830718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.878001+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:27:08.878001+0000","last_peered":"2024-09-10T10:27:08.878001+0000","last_clean":"2024-09-10T10:27:08.878001+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:27:08.878001+0000","last_undegraded":"2024-09-10T10:27:08.878001+0000","last_fullsized":"2024-09-10T10:27:08.878001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826609+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:26:47.826609+0000","last_peered":"2024-09-10T10:26:47.826609+0000","last_clean":"2024-09-10T10:26:47.826609+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:26:47.826609+0000","last_undegraded":"2024-09-10T10:26:47.826609+0000","last_fullsized":"2024-09-10T10:26:47.826609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271720+0000","last_change":"2024-09-10T10:26:46.193656+0000","last_active":"2024-09-10T10:26:48.271720+0000","last_peered":"2024-09-10T10:26:48.271720+0000","last_clean":"2024-09-10T10:26:48.271720+0000","last_became_active":"2024-09-10T10:26:46.193109+0000","last_became_peered":"2024-09-10T10:26:46.193109+0000","last_unstale":"2024-09-10T10:26:48.271720+0000","last_undegraded":"2024-09-10T10:26:48.271720+0000","last_fullsized":"2024-09-10T10:26:48.271720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436035+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:26:48.436035+0000","last_peered":"2024-09-10T10:26:48.436035+0000","last_clean":"2024-09-10T10:26:48.436035+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:26:48.436035+0000","last_undegraded":"2024-09-10T10:26:48.436035+0000","last_fullsized":"2024-09-10T10:26:48.436035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829645+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:26:47.829645+0000","last_peered":"2024-09-10T10:26:47.829645+0000","last_clean":"2024-09-10T10:26:47.829645+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:26:47.829645+0000","last_undegraded":"2024-09-10T10:26:47.829645+0000","last_fullsized":"2024-09-10T10:26:47.829645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437648+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:26:48.437648+0000","last_peered":"2024-09-10T10:26:48.437648+0000","last_clean":"2024-09-10T10:26:48.437648+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:26:48.437648+0000","last_undegraded":"2024-09-10T10:26:48.437648+0000","last_fullsized":"2024-09-10T10:26:48.437648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826450+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:26:47.826450+0000","last_peered":"2024-09-10T10:26:47.826450+0000","last_clean":"2024-09-10T10:26:47.826450+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:26:47.826450+0000","last_undegraded":"2024-09-10T10:26:47.826450+0000","last_fullsized":"2024-09-10T10:26:47.826450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826672+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:26:47.826672+0000","last_peered":"2024-09-10T10:26:47.826672+0000","last_clean":"2024-09-10T10:26:47.826672+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:26:47.826672+0000","last_undegraded":"2024-09-10T10:26:47.826672+0000","last_fullsized":"2024-09-10T10:26:47.826672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436533+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:26:48.436533+0000","last_peered":"2024-09-10T10:26:48.436533+0000","last_clean":"2024-09-10T10:26:48.436533+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:26:48.436533+0000","last_undegraded":"2024-09-10T10:26:48.436533+0000","last_fullsized":"2024-09-10T10:26:48.436533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271816+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:26:48.271816+0000","last_peered":"2024-09-10T10:26:48.271816+0000","last_clean":"2024-09-10T10:26:48.271816+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:26:48.271816+0000","last_undegraded":"2024-09-10T10:26:48.271816+0000","last_fullsized":"2024-09-10T10:26:48.271816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436252+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:26:48.436252+0000","last_peered":"2024-09-10T10:26:48.436252+0000","last_clean":"2024-09-10T10:26:48.436252+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:26:48.436252+0000","last_undegraded":"2024-09-10T10:26:48.436252+0000","last_fullsized":"2024-09-10T10:26:48.436252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"26'5","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:27:08.884118+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:27:08.884118+0000","last_peered":"2024-09-10T10:27:08.884118+0000","last_clean":"2024-09-10T10:27:08.884118+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:27:08.884118+0000","last_undegraded":"2024-09-10T10:27:08.884118+0000","last_fullsized":"2024-09-10T10:27:08.884118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830947+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:26:47.830947+0000","last_peered":"2024-09-10T10:26:47.830947+0000","last_clean":"2024-09-10T10:26:47.830947+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:26:47.830947+0000","last_undegraded":"2024-09-10T10:26:47.830947+0000","last_fullsized":"2024-09-10T10:26:47.830947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827120+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:26:47.827120+0000","last_peered":"2024-09-10T10:26:47.827120+0000","last_clean":"2024-09-10T10:26:47.827120+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:26:47.827120+0000","last_undegraded":"2024-09-10T10:26:47.827120+0000","last_fullsized":"2024-09-10T10:26:47.827120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436082+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:26:48.436082+0000","last_peered":"2024-09-10T10:26:48.436082+0000","last_clean":"2024-09-10T10:26:48.436082+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:26:48.436082+0000","last_undegraded":"2024-09-10T10:26:48.436082+0000","last_fullsized":"2024-09-10T10:26:48.436082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.435938+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:26:48.435938+0000","last_peered":"2024-09-10T10:26:48.435938+0000","last_clean":"2024-09-10T10:26:48.435938+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:26:48.435938+0000","last_undegraded":"2024-09-10T10:26:48.435938+0000","last_fullsized":"2024-09-10T10:26:48.435938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830004+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:26:47.830004+0000","last_peered":"2024-09-10T10:26:47.830004+0000","last_clean":"2024-09-10T10:26:47.830004+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:26:47.830004+0000","last_undegraded":"2024-09-10T10:26:47.830004+0000","last_fullsized":"2024-09-10T10:26:47.830004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271738+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:26:48.271738+0000","last_peered":"2024-09-10T10:26:48.271738+0000","last_clean":"2024-09-10T10:26:48.271738+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:26:48.271738+0000","last_undegraded":"2024-09-10T10:26:48.271738+0000","last_fullsized":"2024-09-10T10:26:48.271738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827172+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:26:47.827172+0000","last_peered":"2024-09-10T10:26:47.827172+0000","last_clean":"2024-09-10T10:26:47.827172+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:26:47.827172+0000","last_undegraded":"2024-09-10T10:26:47.827172+0000","last_fullsized":"2024-09-10T10:26:47.827172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436417+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:26:48.436417+0000","last_peered":"2024-09-10T10:26:48.436417+0000","last_clean":"2024-09-10T10:26:48.436417+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:26:48.436417+0000","last_undegraded":"2024-09-10T10:26:48.436417+0000","last_fullsized":"2024-09-10T10:26:48.436417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437644+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:26:48.437644+0000","last_peered":"2024-09-10T10:26:48.437644+0000","last_clean":"2024-09-10T10:26:48.437644+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:26:48.437644+0000","last_undegraded":"2024-09-10T10:26:48.437644+0000","last_fullsized":"2024-09-10T10:26:48.437644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.827413+0000","last_change":"2024-09-10T10:26:41.802903+0000","last_active":"2024-09-10T10:26:47.827413+0000","last_peered":"2024-09-10T10:26:47.827413+0000","last_clean":"2024-09-10T10:26:47.827413+0000","last_became_active":"2024-09-10T10:26:41.802586+0000","last_became_peered":"2024-09-10T10:26:41.802586+0000","last_unstale":"2024-09-10T10:26:47.827413+0000","last_undegraded":"2024-09-10T10:26:47.827413+0000","last_fullsized":"2024-09-10T10:26:47.827413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829500+0000","last_change":"2024-09-10T10:26:41.800060+0000","last_active":"2024-09-10T10:26:47.829500+0000","last_peered":"2024-09-10T10:26:47.829500+0000","last_clean":"2024-09-10T10:26:47.829500+0000","last_became_active":"2024-09-10T10:26:41.799729+0000","last_became_peered":"2024-09-10T10:26:41.799729+0000","last_unstale":"2024-09-10T10:26:47.829500+0000","last_undegraded":"2024-09-10T10:26:47.829500+0000","last_fullsized":"2024-09-10T10:26:47.829500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826575+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:26:47.826575+0000","last_peered":"2024-09-10T10:26:47.826575+0000","last_clean":"2024-09-10T10:26:47.826575+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:26:47.826575+0000","last_undegraded":"2024-09-10T10:26:47.826575+0000","last_fullsized":"2024-09-10T10:26:47.826575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829948+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:26:47.829948+0000","last_peered":"2024-09-10T10:26:47.829948+0000","last_clean":"2024-09-10T10:26:47.829948+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:26:47.829948+0000","last_undegraded":"2024-09-10T10:26:47.829948+0000","last_fullsized":"2024-09-10T10:26:47.829948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271616+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:26:48.271616+0000","last_peered":"2024-09-10T10:26:48.271616+0000","last_clean":"2024-09-10T10:26:48.271616+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:26:48.271616+0000","last_undegraded":"2024-09-10T10:26:48.271616+0000","last_fullsized":"2024-09-10T10:26:48.271616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436780+0000","last_change":"2024-09-10T10:26:41.787435+0000","last_active":"2024-09-10T10:26:48.436780+0000","last_peered":"2024-09-10T10:26:48.436780+0000","last_clean":"2024-09-10T10:26:48.436780+0000","last_became_active":"2024-09-10T10:26:41.787083+0000","last_became_peered":"2024-09-10T10:26:41.787083+0000","last_unstale":"2024-09-10T10:26:48.436780+0000","last_undegraded":"2024-09-10T10:26:48.436780+0000","last_fullsized":"2024-09-10T10:26:48.436780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437419+0000","last_change":"2024-09-10T10:26:41.792587+0000","last_active":"2024-09-10T10:26:48.437419+0000","last_peered":"2024-09-10T10:26:48.437419+0000","last_clean":"2024-09-10T10:26:48.437419+0000","last_became_active":"2024-09-10T10:26:41.792320+0000","last_became_peered":"2024-09-10T10:26:41.792320+0000","last_unstale":"2024-09-10T10:26:48.437419+0000","last_undegraded":"2024-09-10T10:26:48.437419+0000","last_fullsized":"2024-09-10T10:26:48.437419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436604+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:26:48.436604+0000","last_peered":"2024-09-10T10:26:48.436604+0000","last_clean":"2024-09-10T10:26:48.436604+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:26:48.436604+0000","last_undegraded":"2024-09-10T10:26:48.436604+0000","last_fullsized":"2024-09-10T10:26:48.436604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.830936+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:26:47.830936+0000","last_peered":"2024-09-10T10:26:47.830936+0000","last_clean":"2024-09-10T10:26:47.830936+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:26:47.830936+0000","last_undegraded":"2024-09-10T10:26:47.830936+0000","last_fullsized":"2024-09-10T10:26:47.830936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829378+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:26:47.829378+0000","last_peered":"2024-09-10T10:26:47.829378+0000","last_clean":"2024-09-10T10:26:47.829378+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:26:47.829378+0000","last_undegraded":"2024-09-10T10:26:47.829378+0000","last_fullsized":"2024-09-10T10:26:47.829378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436127+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:26:48.436127+0000","last_peered":"2024-09-10T10:26:48.436127+0000","last_clean":"2024-09-10T10:26:48.436127+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:26:48.436127+0000","last_undegraded":"2024-09-10T10:26:48.436127+0000","last_fullsized":"2024-09-10T10:26:48.436127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271505+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:26:48.271505+0000","last_peered":"2024-09-10T10:26:48.271505+0000","last_clean":"2024-09-10T10:26:48.271505+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:26:48.271505+0000","last_undegraded":"2024-09-10T10:26:48.271505+0000","last_fullsized":"2024-09-10T10:26:48.271505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826694+0000","last_change":"2024-09-10T10:26:41.802117+0000","last_active":"2024-09-10T10:26:47.826694+0000","last_peered":"2024-09-10T10:26:47.826694+0000","last_clean":"2024-09-10T10:26:47.826694+0000","last_became_active":"2024-09-10T10:26:41.801798+0000","last_became_peered":"2024-09-10T10:26:41.801798+0000","last_unstale":"2024-09-10T10:26:47.826694+0000","last_undegraded":"2024-09-10T10:26:47.826694+0000","last_fullsized":"2024-09-10T10:26:47.826694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.831168+0000","last_change":"2024-09-10T10:26:41.791156+0000","last_active":"2024-09-10T10:26:47.831168+0000","last_peered":"2024-09-10T10:26:47.831168+0000","last_clean":"2024-09-10T10:26:47.831168+0000","last_became_active":"2024-09-10T10:26:41.790893+0000","last_became_peered":"2024-09-10T10:26:41.790893+0000","last_unstale":"2024-09-10T10:26:47.831168+0000","last_undegraded":"2024-09-10T10:26:47.831168+0000","last_fullsized":"2024-09-10T10:26:47.831168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:46.923082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437194+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:26:48.437194+0000","last_peered":"2024-09-10T10:26:48.437194+0000","last_clean":"2024-09-10T10:26:48.437194+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:26:48.437194+0000","last_undegraded":"2024-09-10T10:26:48.437194+0000","last_fullsized":"2024-09-10T10:26:48.437194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437640+0000","last_change":"2024-09-10T10:26:41.795268+0000","last_active":"2024-09-10T10:26:48.437640+0000","last_peered":"2024-09-10T10:26:48.437640+0000","last_clean":"2024-09-10T10:26:48.437640+0000","last_became_active":"2024-09-10T10:26:41.794772+0000","last_became_peered":"2024-09-10T10:26:41.794772+0000","last_unstale":"2024-09-10T10:26:48.437640+0000","last_undegraded":"2024-09-10T10:26:48.437640+0000","last_fullsized":"2024-09-10T10:26:48.437640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437511+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:26:48.437511+0000","last_peered":"2024-09-10T10:26:48.437511+0000","last_clean":"2024-09-10T10:26:48.437511+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:26:48.437511+0000","last_undegraded":"2024-09-10T10:26:48.437511+0000","last_fullsized":"2024-09-10T10:26:48.437511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.826362+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:26:47.826362+0000","last_peered":"2024-09-10T10:26:47.826362+0000","last_clean":"2024-09-10T10:26:47.826362+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:26:47.826362+0000","last_undegraded":"2024-09-10T10:26:47.826362+0000","last_fullsized":"2024-09-10T10:26:47.826362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436935+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:26:48.436935+0000","last_peered":"2024-09-10T10:26:48.436935+0000","last_clean":"2024-09-10T10:26:48.436935+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:26:48.436935+0000","last_undegraded":"2024-09-10T10:26:48.436935+0000","last_fullsized":"2024-09-10T10:26:48.436935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829829+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:26:47.829829+0000","last_peered":"2024-09-10T10:26:47.829829+0000","last_clean":"2024-09-10T10:26:47.829829+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:26:47.829829+0000","last_undegraded":"2024-09-10T10:26:47.829829+0000","last_fullsized":"2024-09-10T10:26:47.829829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436704+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:26:48.436704+0000","last_peered":"2024-09-10T10:26:48.436704+0000","last_clean":"2024-09-10T10:26:48.436704+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:26:48.436704+0000","last_undegraded":"2024-09-10T10:26:48.436704+0000","last_fullsized":"2024-09-10T10:26:48.436704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:47.829829+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:26:47.829829+0000","last_peered":"2024-09-10T10:26:47.829829+0000","last_clean":"2024-09-10T10:26:47.829829+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:26:47.829829+0000","last_undegraded":"2024-09-10T10:26:47.829829+0000","last_fullsized":"2024-09-10T10:26:47.829829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.271530+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:26:48.271530+0000","last_peered":"2024-09-10T10:26:48.271530+0000","last_clean":"2024-09-10T10:26:48.271530+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:26:48.271530+0000","last_undegraded":"2024-09-10T10:26:48.271530+0000","last_fullsized":"2024-09-10T10:26:48.271530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.436873+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:26:48.436873+0000","last_peered":"2024-09-10T10:26:48.436873+0000","last_clean":"2024-09-10T10:26:48.436873+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:26:48.436873+0000","last_undegraded":"2024-09-10T10:26:48.436873+0000","last_fullsized":"2024-09-10T10:26:48.436873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T10:26:48.437504+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:26:48.437504+0000","last_peered":"2024-09-10T10:26:48.437504+0000","last_clean":"2024-09-10T10:26:48.437504+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:26:48.437504+0000","last_undegraded":"2024-09-10T10:26:48.437504+0000","last_fullsized":"2024-09-10T10:26:48.437504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6048,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":12096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":703748,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":115908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":116530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":356352,"data_stored":703070,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:27:12.986 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:27:12.986 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T10:27:12.986 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T10:27:12.986 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T10:27:13.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.117+0000 7f95e91c6640 1 Processor -- start 2024-09-10T10:27:13.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.117+0000 7f95e91c6640 1 -- start start 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95e91c6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f95e40caaf0 0x7f95e40c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95e91c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 0x7f95e4071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95e91c6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e40c99a0 con 0x7f95e4154d80 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95e91c6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f95e40c9780 con 0x7f95e40caaf0 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95e91c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f95e415eff0 con 0x7f95e40715d0 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95e359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 0x7f95e4071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95dbfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f95e40caaf0 0x7f95e40c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95e2d9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f95e4154d80 0x7f95e415ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:35904/0 (socket says 172.21.15.120:35904) 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95e2d9d640 1 -- 172.21.15.120:0/2302622597 learned_addr learned my addr 172.21.15.120:0/2302622597 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95e359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 0x7f95e4071a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:43268/0 (socket says 172.21.15.120:43268) 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95dbfff640 1 -- 172.21.15.120:0/2302622597 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 msgr2=0x7f95e4071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:13.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95dbfff640 1 --2- 172.21.15.120:0/2302622597 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 0x7f95e4071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95dbfff640 1 -- 172.21.15.120:0/2302622597 >> v1:172.21.15.134:6789/0 conn(0x7f95e4154d80 legacy=0x7f95e415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:13.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95dbfff640 1 -- 172.21.15.120:0/2302622597 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e415e230 con 0x7f95e40caaf0 2024-09-10T10:27:13.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.118+0000 7f95e259c640 1 -- 172.21.15.120:0/2302622597 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3719263568 0 0) 0x7f95e40c99a0 con 0x7f95e4154d80 2024-09-10T10:27:13.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.119+0000 7f95e359e640 1 --2- 172.21.15.120:0/2302622597 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 0x7f95e4071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:13.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.119+0000 7f95dbfff640 1 --2- 172.21.15.120:0/2302622597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e40caaf0 0x7f95e40c9240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0097a0 tx=0x7f95cc02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7ff20129d5db98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:13.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.119+0000 7f95e259c640 1 -- 172.21.15.120:0/2302622597 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc037070 con 0x7f95e40caaf0 2024-09-10T10:27:13.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.119+0000 7f95e259c640 1 -- 172.21.15.120:0/2302622597 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc032440 con 0x7f95e40caaf0 2024-09-10T10:27:13.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.119+0000 7f95e259c640 1 -- 172.21.15.120:0/2302622597 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc032810 con 0x7f95e40caaf0 2024-09-10T10:27:13.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.120+0000 7f95e91c6640 1 -- 172.21.15.120:0/2302622597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e40caaf0 msgr2=0x7f95e40c9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:13.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.120+0000 7f95e91c6640 1 --2- 172.21.15.120:0/2302622597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e40caaf0 0x7f95e40c9240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0097a0 tx=0x7f95cc02a210 comp rx=0 tx=0).stop 2024-09-10T10:27:13.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.120+0000 7f95e91c6640 1 -- 172.21.15.120:0/2302622597 shutdown_connections 2024-09-10T10:27:13.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.120+0000 7f95e91c6640 1 --2- 172.21.15.120:0/2302622597 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 0x7f95e4071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.120+0000 7f95e91c6640 1 --2- 172.21.15.120:0/2302622597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e40caaf0 0x7f95e40c9240 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.120+0000 7f95e91c6640 1 -- 172.21.15.120:0/2302622597 >> 172.21.15.120:0/2302622597 conn(0x7f95e4070810 msgr2=0x7f95e4070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:13.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.120+0000 7f95e91c6640 1 -- 172.21.15.120:0/2302622597 shutdown_connections 2024-09-10T10:27:13.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.120+0000 7f95e91c6640 1 -- 172.21.15.120:0/2302622597 wait complete. 2024-09-10T10:27:13.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.121+0000 7f95e91c6640 1 Processor -- start 2024-09-10T10:27:13.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.121+0000 7f95e91c6640 1 -- start start 2024-09-10T10:27:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95e91c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 0x7f95e40cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95e91c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e40caaf0 0x7f95e41f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95e91c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4154d80 0x7f95e406d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95e91c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f95e415e7f0 con 0x7f95e40caaf0 2024-09-10T10:27:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95e91c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95e415eb60 con 0x7f95e4154d80 2024-09-10T10:27:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95e91c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f95e4155160 con 0x7f95e40715d0 2024-09-10T10:27:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e40caaf0 0x7f95e41f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e40caaf0 0x7f95e41f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34768/0 (socket says 172.21.15.120:34768) 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95dbfff640 1 -- 172.21.15.120:0/3457279515 learned_addr learned my addr 172.21.15.120:0/3457279515 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95e2d9d640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 0x7f95e40cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95e359e640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4154d80 0x7f95e406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95dbfff640 1 -- 172.21.15.120:0/3457279515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 msgr2=0x7f95e40cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95dbfff640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 0x7f95e40cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95dbfff640 1 -- 172.21.15.120:0/3457279515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4154d80 msgr2=0x7f95e406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95dbfff640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4154d80 0x7f95e406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.122+0000 7f95dbfff640 1 -- 172.21.15.120:0/3457279515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e40c5800 con 0x7f95e40caaf0 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.123+0000 7f95dbfff640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e40caaf0 0x7f95e41f3aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0097a0 tx=0x7f95cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:13.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.123+0000 7f95db7fe640 1 -- 172.21.15.120:0/3457279515 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc037050 con 0x7f95e40caaf0 2024-09-10T10:27:13.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.123+0000 7f95db7fe640 1 -- 172.21.15.120:0/3457279515 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc038040 con 0x7f95e40caaf0 2024-09-10T10:27:13.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.123+0000 7f95db7fe640 1 -- 172.21.15.120:0/3457279515 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc004490 con 0x7f95e40caaf0 2024-09-10T10:27:13.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.123+0000 7f95e91c6640 1 -- 172.21.15.120:0/3457279515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e40cc410 con 0x7f95e40caaf0 2024-09-10T10:27:13.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.123+0000 7f95e91c6640 1 -- 172.21.15.120:0/3457279515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e40cd480 con 0x7f95e40caaf0 2024-09-10T10:27:13.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.124+0000 7f95e91c6640 1 -- 172.21.15.120:0/3457279515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e40cbcb0 con 0x7f95e40caaf0 2024-09-10T10:27:13.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.124+0000 7f95db7fe640 1 -- 172.21.15.120:0/3457279515 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95cc003610 con 0x7f95e40caaf0 2024-09-10T10:27:13.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.125+0000 7f95db7fe640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95c406be30 0x7f95c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:13.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.125+0000 7f95e2d9d640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95c406be30 0x7f95c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:13.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.125+0000 7f95db7fe640 1 -- 172.21.15.120:0/3457279515 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f95cc0ab9e0 con 0x7f95e40caaf0 2024-09-10T10:27:13.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.125+0000 7f95e2d9d640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95c406be30 0x7f95c406e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f95d4007fd0 tx=0x7f95d4007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:13.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.129+0000 7f95db7fe640 1 -- 172.21.15.120:0/3457279515 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f95cc07a0e0 con 0x7f95e40caaf0 2024-09-10T10:27:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.328+0000 7f95e91c6640 1 -- 172.21.15.120:0/3457279515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f95e415d260 con 0x7f95e40caaf0 2024-09-10T10:27:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.328+0000 7f95db7fe640 1 -- 172.21.15.120:0/3457279515 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f95cc0799e0 con 0x7f95e40caaf0 2024-09-10T10:27:13.330 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:13.330 INFO:teuthology.orchestra.run.smithi120.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-09-10T10:27:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.330+0000 7f95e91c6640 1 -- 172.21.15.120:0/3457279515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95c406be30 msgr2=0x7f95c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.330+0000 7f95e91c6640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95c406be30 0x7f95c406e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f95d4007fd0 tx=0x7f95d4007de0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.330+0000 7f95e91c6640 1 -- 172.21.15.120:0/3457279515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e40caaf0 msgr2=0x7f95e41f3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.330+0000 7f95e91c6640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e40caaf0 0x7f95e41f3aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0097a0 tx=0x7f95cc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.330+0000 7f95e91c6640 1 -- 172.21.15.120:0/3457279515 shutdown_connections 2024-09-10T10:27:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.330+0000 7f95e91c6640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f95c406be30 0x7f95c406e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.330+0000 7f95e91c6640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4154d80 0x7f95e406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.330+0000 7f95e91c6640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e40caaf0 0x7f95e41f3aa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.330+0000 7f95e91c6640 1 --2- 172.21.15.120:0/3457279515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95e40715d0 0x7f95e40cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:13.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.330+0000 7f95e91c6640 1 -- 172.21.15.120:0/3457279515 >> 172.21.15.120:0/3457279515 conn(0x7f95e4070810 msgr2=0x7f95e40b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:13.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.331+0000 7f95e91c6640 1 -- 172.21.15.120:0/3457279515 shutdown_connections 2024-09-10T10:27:13.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:13.331+0000 7f95e91c6640 1 -- 172.21.15.120:0/3457279515 wait complete. 2024-09-10T10:27:16.343 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T10:27:16.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.451+0000 7f4a2c065640 1 Processor -- start 2024-09-10T10:27:16.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.451+0000 7f4a2c065640 1 -- start start 2024-09-10T10:27:16.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.452+0000 7f4a2c065640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a24154920 0x7f4a240c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:16.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.452+0000 7f4a2c065640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4a240cc540 0x7f4a240c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:16.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.452+0000 7f4a2c065640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a240c9200 con 0x7f4a240cc940 2024-09-10T10:27:16.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.452+0000 7f4a2c065640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4a240c8e20 con 0x7f4a240cc540 2024-09-10T10:27:16.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.452+0000 7f4a2c065640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4a240c9e10 con 0x7f4a24154920 2024-09-10T10:27:16.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.452+0000 7f4a29600640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4a240cc540 0x7f4a240c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:16.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.452+0000 7f4a29600640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4a240cc540 0x7f4a240c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34770/0 (socket says 172.21.15.120:34770) 2024-09-10T10:27:16.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.452+0000 7f4a29600640 1 -- 172.21.15.120:0/4234344351 learned_addr learned my addr 172.21.15.120:0/4234344351 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:16.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.453+0000 7f4a29600640 1 -- 172.21.15.120:0/4234344351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a24154920 msgr2=0x7f4a240c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:16.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.453+0000 7f4a29e01640 1 --2- 172.21.15.120:0/4234344351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a24154920 0x7f4a240c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:16.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.453+0000 7f4a29600640 1 --2- 172.21.15.120:0/4234344351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a24154920 0x7f4a240c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:16.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.453+0000 7f4a29600640 1 -- 172.21.15.120:0/4234344351 >> v1:172.21.15.134:6789/0 conn(0x7f4a240cc940 legacy=0x7f4a2415e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:16.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.453+0000 7f4a29600640 1 -- 172.21.15.120:0/4234344351 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a240c94e0 con 0x7f4a240cc540 2024-09-10T10:27:16.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.454+0000 7f4a29600640 1 --2- 172.21.15.120:0/4234344351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a240cc540 0x7f4a240c6ff0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009870 tx=0x7f4a1402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abddf9f8451f1bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:16.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.454+0000 7f4a28dff640 1 -- 172.21.15.120:0/4234344351 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14037070 con 0x7f4a240cc540 2024-09-10T10:27:16.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.454+0000 7f4a28dff640 1 -- 172.21.15.120:0/4234344351 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a14032440 con 0x7f4a240cc540 2024-09-10T10:27:16.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.454+0000 7f4a28dff640 1 -- 172.21.15.120:0/4234344351 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14032810 con 0x7f4a240cc540 2024-09-10T10:27:16.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.455+0000 7f4a2c065640 1 -- 172.21.15.120:0/4234344351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a240cc540 msgr2=0x7f4a240c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.455+0000 7f4a2c065640 1 --2- 172.21.15.120:0/4234344351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a240cc540 0x7f4a240c6ff0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009870 tx=0x7f4a1402a2a0 comp rx=0 tx=0).stop 2024-09-10T10:27:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.455+0000 7f4a2c065640 1 -- 172.21.15.120:0/4234344351 shutdown_connections 2024-09-10T10:27:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.455+0000 7f4a2c065640 1 --2- 172.21.15.120:0/4234344351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a240cc540 0x7f4a240c6ff0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.455+0000 7f4a2c065640 1 --2- 172.21.15.120:0/4234344351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a24154920 0x7f4a240c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.455+0000 7f4a2c065640 1 -- 172.21.15.120:0/4234344351 >> 172.21.15.120:0/4234344351 conn(0x7f4a2406f9e0 msgr2=0x7f4a2406fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.455+0000 7f4a2c065640 1 -- 172.21.15.120:0/4234344351 shutdown_connections 2024-09-10T10:27:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.455+0000 7f4a2c065640 1 -- 172.21.15.120:0/4234344351 wait complete. 2024-09-10T10:27:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.456+0000 7f4a2c065640 1 Processor -- start 2024-09-10T10:27:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.456+0000 7f4a2c065640 1 -- start start 2024-09-10T10:27:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2c065640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a240cc540 0x7f4a2415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2c065640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a240cc940 0x7f4a2415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2c065640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a24154920 0x7f4a2414ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2c065640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4a24154d00 con 0x7f4a24154920 2024-09-10T10:27:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2c065640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a240c9aa0 con 0x7f4a240cc540 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2c065640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4a24154e80 con 0x7f4a240cc940 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2a602640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a24154920 0x7f4a2414ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2a602640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a24154920 0x7f4a2414ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34786/0 (socket says 172.21.15.120:34786) 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2a602640 1 -- 172.21.15.120:0/2765946138 learned_addr learned my addr 172.21.15.120:0/2765946138 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a29600640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a240cc940 0x7f4a2415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a29e01640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a240cc540 0x7f4a2415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2a602640 1 -- 172.21.15.120:0/2765946138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a240cc940 msgr2=0x7f4a2415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2a602640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a240cc940 0x7f4a2415d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2a602640 1 -- 172.21.15.120:0/2765946138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a240cc540 msgr2=0x7f4a2415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2a602640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a240cc540 0x7f4a2415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2a602640 1 -- 172.21.15.120:0/2765946138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a241f0f70 con 0x7f4a24154920 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2a602640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a24154920 0x7f4a2414ddb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00c7b0 tx=0x7f4a1c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a1a7fc640 1 -- 172.21.15.120:0/2765946138 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00ce60 con 0x7f4a24154920 2024-09-10T10:27:16.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a1a7fc640 1 -- 172.21.15.120:0/2765946138 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a1c004b10 con 0x7f4a24154920 2024-09-10T10:27:16.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2c065640 1 -- 172.21.15.120:0/2765946138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a241f1200 con 0x7f4a24154920 2024-09-10T10:27:16.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a1a7fc640 1 -- 172.21.15.120:0/2765946138 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00a850 con 0x7f4a24154920 2024-09-10T10:27:16.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.457+0000 7f4a2c065640 1 -- 172.21.15.120:0/2765946138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a2415c9a0 con 0x7f4a24154920 2024-09-10T10:27:16.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.459+0000 7f4a2c065640 1 -- 172.21.15.120:0/2765946138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a240c88e0 con 0x7f4a24154920 2024-09-10T10:27:16.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.459+0000 7f4a1a7fc640 1 -- 172.21.15.120:0/2765946138 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a1c00a9f0 con 0x7f4a24154920 2024-09-10T10:27:16.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.459+0000 7f4a1a7fc640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4a0806be60 0x7f4a0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:16.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.459+0000 7f4a1a7fc640 1 -- 172.21.15.120:0/2765946138 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4a1c08b320 con 0x7f4a24154920 2024-09-10T10:27:16.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.460+0000 7f4a29e01640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4a0806be60 0x7f4a0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:16.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.460+0000 7f4a29e01640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4a0806be60 0x7f4a0806e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000a970 tx=0x7f4a20008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.464+0000 7f4a1a7fc640 1 -- 172.21.15.120:0/2765946138 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4a1c020070 con 0x7f4a24154920 2024-09-10T10:27:16.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.663+0000 7f4a2c065640 1 -- 172.21.15.120:0/2765946138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4a240c7aa0 con 0x7f4a24154920 2024-09-10T10:27:16.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.663+0000 7f4a1a7fc640 1 -- 172.21.15.120:0/2765946138 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f4a1c059a20 con 0x7f4a24154920 2024-09-10T10:27:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:16.665 INFO:teuthology.orchestra.run.smithi120.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-09-10T10:27:16.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 -- 172.21.15.120:0/2765946138 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4a0806be60 msgr2=0x7f4a0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:16.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4a0806be60 0x7f4a0806e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000a970 tx=0x7f4a20008040 comp rx=0 tx=0).stop 2024-09-10T10:27:16.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 -- 172.21.15.120:0/2765946138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a24154920 msgr2=0x7f4a2414ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:16.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a24154920 0x7f4a2414ddb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00c7b0 tx=0x7f4a1c00cc70 comp rx=0 tx=0).stop 2024-09-10T10:27:16.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 -- 172.21.15.120:0/2765946138 shutdown_connections 2024-09-10T10:27:16.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4a0806be60 0x7f4a0806e320 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:16.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a24154920 0x7f4a2414ddb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:16.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a240cc940 0x7f4a2415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:16.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 --2- 172.21.15.120:0/2765946138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a240cc540 0x7f4a2415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:16.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 -- 172.21.15.120:0/2765946138 >> 172.21.15.120:0/2765946138 conn(0x7f4a2406f9e0 msgr2=0x7f4a2406fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:16.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 -- 172.21.15.120:0/2765946138 shutdown_connections 2024-09-10T10:27:16.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:16.665+0000 7f4a2c065640 1 -- 172.21.15.120:0/2765946138 wait complete. 2024-09-10T10:27:16.765 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-09-10T10:27:16.764+0000 7f890ecec640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-10T10:27:19.678 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T10:27:19.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.787+0000 7ffb35d68640 1 Processor -- start 2024-09-10T10:27:19.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.787+0000 7ffb35d68640 1 -- start start 2024-09-10T10:27:19.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb35d68640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ffb30071120 0x7ffb30157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:19.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb35d68640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb300cd3e0 con 0x7ffb30070d40 2024-09-10T10:27:19.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb35d68640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb300cd6c0 con 0x7ffb300c8f80 2024-09-10T10:27:19.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb35d68640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ffb300c8af0 con 0x7ffb30071120 2024-09-10T10:27:19.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb2f7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ffb30070d40 0x7ffb3015ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:43782/0 (socket says 172.21.15.120:43782) 2024-09-10T10:27:19.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb2f7fe640 1 -- 172.21.15.120:0/443648937 learned_addr learned my addr 172.21.15.120:0/443648937 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:19.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb2ffff640 1 --2- 172.21.15.120:0/443648937 >> v2:172.21.15.120:3300/0 conn(0x7ffb30071120 0x7ffb30157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:19.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb2ffff640 1 -- 172.21.15.120:0/443648937 >> v1:172.21.15.134:6789/0 conn(0x7ffb300c8f80 legacy=0x7ffb301577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:19.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb2ffff640 1 -- 172.21.15.120:0/443648937 >> v1:172.21.15.120:6789/0 conn(0x7ffb30070d40 legacy=0x7ffb3015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:19.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb2effd640 1 -- 172.21.15.120:0/443648937 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4077362377 0 0) 0x7ffb300cd3e0 con 0x7ffb30070d40 2024-09-10T10:27:19.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.788+0000 7ffb2ffff640 1 -- 172.21.15.120:0/443648937 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb300cd9a0 con 0x7ffb30071120 2024-09-10T10:27:19.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.789+0000 7ffb2ffff640 1 --2- 172.21.15.120:0/443648937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb30071120 0x7ffb30157ee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009d90 tx=0x7ffb2802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ec93d3f40b9e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:19.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.789+0000 7ffb2effd640 1 -- 172.21.15.120:0/443648937 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28037070 con 0x7ffb30071120 2024-09-10T10:27:19.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.789+0000 7ffb2effd640 1 -- 172.21.15.120:0/443648937 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb28007740 con 0x7ffb30071120 2024-09-10T10:27:19.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.789+0000 7ffb2effd640 1 -- 172.21.15.120:0/443648937 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28007b10 con 0x7ffb30071120 2024-09-10T10:27:19.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.790+0000 7ffb35d68640 1 -- 172.21.15.120:0/443648937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb30071120 msgr2=0x7ffb30157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:19.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.790+0000 7ffb35d68640 1 --2- 172.21.15.120:0/443648937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb30071120 0x7ffb30157ee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009d90 tx=0x7ffb2802a780 comp rx=0 tx=0).stop 2024-09-10T10:27:19.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.790+0000 7ffb35d68640 1 -- 172.21.15.120:0/443648937 shutdown_connections 2024-09-10T10:27:19.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.790+0000 7ffb35d68640 1 --2- 172.21.15.120:0/443648937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb30071120 0x7ffb30157ee0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:19.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.790+0000 7ffb35d68640 1 -- 172.21.15.120:0/443648937 >> 172.21.15.120:0/443648937 conn(0x7ffb30072a60 msgr2=0x7ffb30072e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:19.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.790+0000 7ffb35d68640 1 -- 172.21.15.120:0/443648937 shutdown_connections 2024-09-10T10:27:19.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.790+0000 7ffb35d68640 1 -- 172.21.15.120:0/443648937 wait complete. 2024-09-10T10:27:19.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.791+0000 7ffb35d68640 1 Processor -- start 2024-09-10T10:27:19.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.791+0000 7ffb35d68640 1 -- start start 2024-09-10T10:27:19.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb35d68640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb30070d40 0x7ffb300c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:19.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb35d68640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb30071120 0x7ffb301e79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:19.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb35d68640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb300c8f80 0x7ffb300c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:19.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb35d68640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffb30158420 con 0x7ffb300c8f80 2024-09-10T10:27:19.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb35d68640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffb300c8780 con 0x7ffb30071120 2024-09-10T10:27:19.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb35d68640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffb301585a0 con 0x7ffb30070d40 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb300c8f80 0x7ffb300c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb300c8f80 0x7ffb300c32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56102/0 (socket says 172.21.15.120:56102) 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2ffff640 1 -- 172.21.15.120:0/489118775 learned_addr learned my addr 172.21.15.120:0/489118775 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2f7fe640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb30070d40 0x7ffb300c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb27fff640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb30071120 0x7ffb301e79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2ffff640 1 -- 172.21.15.120:0/489118775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb30070d40 msgr2=0x7ffb300c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2ffff640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb30070d40 0x7ffb300c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2ffff640 1 -- 172.21.15.120:0/489118775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb30071120 msgr2=0x7ffb301e79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2ffff640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb30071120 0x7ffb301e79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2ffff640 1 -- 172.21.15.120:0/489118775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb301f44a0 con 0x7ffb300c8f80 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2ffff640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb300c8f80 0x7ffb300c32c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009d90 tx=0x7ffb28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2cff9640 1 -- 172.21.15.120:0/489118775 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28042020 con 0x7ffb300c8f80 2024-09-10T10:27:19.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2cff9640 1 -- 172.21.15.120:0/489118775 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb28032990 con 0x7ffb300c8f80 2024-09-10T10:27:19.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb2cff9640 1 -- 172.21.15.120:0/489118775 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28037040 con 0x7ffb300c8f80 2024-09-10T10:27:19.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb35d68640 1 -- 172.21.15.120:0/489118775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb300bf260 con 0x7ffb300c8f80 2024-09-10T10:27:19.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.792+0000 7ffb35d68640 1 -- 172.21.15.120:0/489118775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb300bfdc0 con 0x7ffb300c8f80 2024-09-10T10:27:19.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.793+0000 7ffb35d68640 1 -- 172.21.15.120:0/489118775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb300c7c40 con 0x7ffb300c8f80 2024-09-10T10:27:19.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.794+0000 7ffb2cff9640 1 -- 172.21.15.120:0/489118775 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb28045020 con 0x7ffb300c8f80 2024-09-10T10:27:19.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.794+0000 7ffb2cff9640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffb1006be30 0x7ffb1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:19.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.794+0000 7ffb2cff9640 1 -- 172.21.15.120:0/489118775 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ffb280ac1b0 con 0x7ffb300c8f80 2024-09-10T10:27:19.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.794+0000 7ffb2f7fe640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffb1006be30 0x7ffb1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:19.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.795+0000 7ffb2f7fe640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffb1006be30 0x7ffb1006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffb20000f60 tx=0x7ffb20007b90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:19.799+0000 7ffb2cff9640 1 -- 172.21.15.120:0/489118775 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffb2804c070 con 0x7ffb300c8f80 2024-09-10T10:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.004+0000 7ffb35d68640 1 -- 172.21.15.120:0/489118775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ffb30158080 con 0x7ffb300c8f80 2024-09-10T10:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.004+0000 7ffb2cff9640 1 -- 172.21.15.120:0/489118775 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7ffb2807a850 con 0x7ffb300c8f80 2024-09-10T10:27:20.006 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:20.006 INFO:teuthology.orchestra.run.smithi120.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-09-10T10:27:20.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 -- 172.21.15.120:0/489118775 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffb1006be30 msgr2=0x7ffb1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffb1006be30 0x7ffb1006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffb20000f60 tx=0x7ffb20007b90 comp rx=0 tx=0).stop 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 -- 172.21.15.120:0/489118775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb300c8f80 msgr2=0x7ffb300c32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb300c8f80 0x7ffb300c32c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009d90 tx=0x7ffb28004a40 comp rx=0 tx=0).stop 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 -- 172.21.15.120:0/489118775 shutdown_connections 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffb1006be30 0x7ffb1006e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb300c8f80 0x7ffb300c32c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb30071120 0x7ffb301e79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 --2- 172.21.15.120:0/489118775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb30070d40 0x7ffb300c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 -- 172.21.15.120:0/489118775 >> 172.21.15.120:0/489118775 conn(0x7ffb30072a60 msgr2=0x7ffb3006fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 -- 172.21.15.120:0/489118775 shutdown_connections 2024-09-10T10:27:20.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:20.006+0000 7ffb35d68640 1 -- 172.21.15.120:0/489118775 wait complete. 2024-09-10T10:27:21.765 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-09-10T10:27:21.764+0000 7f890ecec640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-10T10:27:23.019 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T10:27:23.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.128+0000 7f9b87b98640 1 Processor -- start 2024-09-10T10:27:23.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.128+0000 7f9b87b98640 1 -- start start 2024-09-10T10:27:23.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.129+0000 7f9b87b98640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9b80154d80 0x7f9b800caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.129+0000 7f9b87b98640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b800c99a0 con 0x7f9b800c9240 2024-09-10T10:27:23.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.129+0000 7f9b87b98640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8015e230 con 0x7f9b800715d0 2024-09-10T10:27:23.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.129+0000 7f9b87b98640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8015eff0 con 0x7f9b80154d80 2024-09-10T10:27:23.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.129+0000 7f9b85133640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f9b800c9240 0x7f9b8015ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:43790/0 (socket says 172.21.15.120:43790) 2024-09-10T10:27:23.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.129+0000 7f9b85133640 1 -- 172.21.15.120:0/1462597777 learned_addr learned my addr 172.21.15.120:0/1462597777 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:23.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b85934640 1 --2- 172.21.15.120:0/1462597777 >> v2:172.21.15.134:3300/0 conn(0x7f9b80154d80 0x7f9b800caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3270446551 0 0) 0x7f9b800c99a0 con 0x7f9b800c9240 2024-09-10T10:27:23.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b6c003520 con 0x7f9b800c9240 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 585351049 0 0) 0x7f9b6c003520 con 0x7f9b800c9240 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 >> v2:172.21.15.134:3300/0 conn(0x7f9b80154d80 msgr2=0x7f9b800caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 --2- 172.21.15.120:0/1462597777 >> v2:172.21.15.134:3300/0 conn(0x7f9b80154d80 0x7f9b800caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 >> v1:172.21.15.134:6789/0 conn(0x7f9b800715d0 legacy=0x7f9b801653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8015e510 con 0x7f9b800c9240 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f9b70001670 con 0x7f9b800c9240 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 >> v1:172.21.15.120:6789/0 conn(0x7f9b800c9240 legacy=0x7f9b8015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b6c0039b0 0x7f9b6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b6c004800 0x7f9b6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b6c0051c0 0x7f9b6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c00c040 con 0x7f9b6c0051c0 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c00c1f0 con 0x7f9b6c004800 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6c00c3f0 con 0x7f9b6c0039b0 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b86135640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b6c0051c0 0x7f9b6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b85934640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b6c0039b0 0x7f9b6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b85133640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b6c004800 0x7f9b6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b86135640 1 -- 172.21.15.120:0/1462597777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b6c0039b0 msgr2=0x7f9b6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b86135640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b6c0039b0 0x7f9b6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b86135640 1 -- 172.21.15.120:0/1462597777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b6c004800 msgr2=0x7f9b6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b86135640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b6c004800 0x7f9b6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.130+0000 7f9b86135640 1 -- 172.21.15.120:0/1462597777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6c003520 con 0x7f9b6c0051c0 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.131+0000 7f9b86135640 1 -- 172.21.15.120:0/1462597777 reap_dead start 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.131+0000 7f9b86135640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b6c0051c0 0x7f9b6c00b9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001e50 tx=0x7f9b6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa44e6a899f6fab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.131+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78001a70 con 0x7f9b6c0051c0 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.131+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b78003970 con 0x7f9b6c0051c0 2024-09-10T10:27:23.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.131+0000 7f9b84932640 1 -- 172.21.15.120:0/1462597777 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78003d60 con 0x7f9b6c0051c0 2024-09-10T10:27:23.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.132+0000 7f9b87b98640 1 -- 172.21.15.120:0/1462597777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b6c0051c0 msgr2=0x7f9b6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.132+0000 7f9b87b98640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b6c0051c0 0x7f9b6c00b9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001e50 tx=0x7f9b6c005830 comp rx=0 tx=0).stop 2024-09-10T10:27:23.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.132+0000 7f9b87b98640 1 -- 172.21.15.120:0/1462597777 shutdown_connections 2024-09-10T10:27:23.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.132+0000 7f9b87b98640 1 --2- 172.21.15.120:0/1462597777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b6c0051c0 0x7f9b6c00b9c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.132+0000 7f9b87b98640 1 -- 172.21.15.120:0/1462597777 >> 172.21.15.120:0/1462597777 conn(0x7f9b80070810 msgr2=0x7f9b80070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:23.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.132+0000 7f9b87b98640 1 -- 172.21.15.120:0/1462597777 shutdown_connections 2024-09-10T10:27:23.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.132+0000 7f9b87b98640 1 -- 172.21.15.120:0/1462597777 wait complete. 2024-09-10T10:27:23.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.133+0000 7f9b87b98640 1 Processor -- start 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.133+0000 7f9b87b98640 1 -- start start 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b87b98640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b6c0051c0 0x7f9b801f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b87b98640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b8015b7d0 0x7f9b8015aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b87b98640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b801578d0 0x7f9b80159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b87b98640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9b800c9670 con 0x7f9b8015b7d0 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b87b98640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8015eb60 con 0x7f9b6c0051c0 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b87b98640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9b80155160 con 0x7f9b801578d0 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85133640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b8015b7d0 0x7f9b8015aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85133640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b8015b7d0 0x7f9b8015aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56106/0 (socket says 172.21.15.120:56106) 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85133640 1 -- 172.21.15.120:0/3814401345 learned_addr learned my addr 172.21.15.120:0/3814401345 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b86135640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b801578d0 0x7f9b80159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85934640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b6c0051c0 0x7f9b801f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85133640 1 -- 172.21.15.120:0/3814401345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b801578d0 msgr2=0x7f9b80159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85133640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b801578d0 0x7f9b80159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85133640 1 -- 172.21.15.120:0/3814401345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b6c0051c0 msgr2=0x7f9b801f07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85133640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b6c0051c0 0x7f9b801f07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85133640 1 -- 172.21.15.120:0/3814401345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b800cc330 con 0x7f9b8015b7d0 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b86135640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b801578d0 0x7f9b80159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85934640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b6c0051c0 0x7f9b801f07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.134+0000 7f9b85133640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b8015b7d0 0x7f9b8015aef0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b70007370 tx=0x7f9b70007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.135+0000 7f9b767fc640 1 -- 172.21.15.120:0/3814401345 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7001a070 con 0x7f9b8015b7d0 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.135+0000 7f9b767fc640 1 -- 172.21.15.120:0/3814401345 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7000f9c0 con 0x7f9b8015b7d0 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.135+0000 7f9b87b98640 1 -- 172.21.15.120:0/3814401345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b800cc610 con 0x7f9b8015b7d0 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.135+0000 7f9b767fc640 1 -- 172.21.15.120:0/3814401345 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7000fdb0 con 0x7f9b8015b7d0 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.135+0000 7f9b87b98640 1 -- 172.21.15.120:0/3814401345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b800ccbd0 con 0x7f9b8015b7d0 2024-09-10T10:27:23.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.136+0000 7f9b87b98640 1 -- 172.21.15.120:0/3814401345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8006ec00 con 0x7f9b8015b7d0 2024-09-10T10:27:23.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.136+0000 7f9b767fc640 1 -- 172.21.15.120:0/3814401345 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b7001d070 con 0x7f9b8015b7d0 2024-09-10T10:27:23.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.137+0000 7f9b767fc640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9b6406be30 0x7f9b6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.137+0000 7f9b767fc640 1 -- 172.21.15.120:0/3814401345 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9b7008ac90 con 0x7f9b8015b7d0 2024-09-10T10:27:23.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.137+0000 7f9b85934640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9b6406be30 0x7f9b6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.137+0000 7f9b85934640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9b6406be30 0x7f9b6406e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009b40 tx=0x7f9b7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:23.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.141+0000 7f9b767fc640 1 -- 172.21.15.120:0/3814401345 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9b8006ec00 con 0x7f9b8015b7d0 2024-09-10T10:27:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.342+0000 7f9b87b98640 1 -- 172.21.15.120:0/3814401345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9b800af390 con 0x7f9b8015b7d0 2024-09-10T10:27:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.343+0000 7f9b767fc640 1 -- 172.21.15.120:0/3814401345 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9b70059240 con 0x7f9b8015b7d0 2024-09-10T10:27:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:23.344 INFO:teuthology.orchestra.run.smithi120.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T10:27:23.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 -- 172.21.15.120:0/3814401345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9b6406be30 msgr2=0x7f9b6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9b6406be30 0x7f9b6406e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009b40 tx=0x7f9b7c008040 comp rx=0 tx=0).stop 2024-09-10T10:27:23.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 -- 172.21.15.120:0/3814401345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b8015b7d0 msgr2=0x7f9b8015aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b8015b7d0 0x7f9b8015aef0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b70007370 tx=0x7f9b70007830 comp rx=0 tx=0).stop 2024-09-10T10:27:23.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 -- 172.21.15.120:0/3814401345 shutdown_connections 2024-09-10T10:27:23.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9b6406be30 0x7f9b6406e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b801578d0 0x7f9b80159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b8015b7d0 0x7f9b8015aef0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 --2- 172.21.15.120:0/3814401345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b6c0051c0 0x7f9b801f07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 -- 172.21.15.120:0/3814401345 >> 172.21.15.120:0/3814401345 conn(0x7f9b80070810 msgr2=0x7f9b8015cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:23.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 -- 172.21.15.120:0/3814401345 shutdown_connections 2024-09-10T10:27:23.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.345+0000 7f9b87b98640 1 -- 172.21.15.120:0/3814401345 wait complete. 2024-09-10T10:27:23.357 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T10:27:23.357 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:27:23.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.454+0000 7fcae75fd640 1 Processor -- start 2024-09-10T10:27:23.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.454+0000 7fcae75fd640 1 -- start start 2024-09-10T10:27:23.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae75fd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcae0151a90 0x7fcae00c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae75fd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c6410 0x7fcae0158040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae75fd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcae00c5280 0x7fcae00c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae75fd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fcae0151e70 con 0x7fcae0151a90 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae75fd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcae00c8130 con 0x7fcae00c5280 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae75fd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcae00c8410 con 0x7fcae00c6410 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae5399640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcae0151a90 0x7fcae00c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae5399640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcae0151a90 0x7fcae00c36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56108/0 (socket says 172.21.15.120:56108) 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae5399640 1 -- 172.21.15.120:0/4214223423 learned_addr learned my addr 172.21.15.120:0/4214223423 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae5b9a640 1 --2- 172.21.15.120:0/4214223423 >> v2:172.21.15.134:3300/0 conn(0x7fcae00c5280 0x7fcae00c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae4b98640 1 --2- 172.21.15.120:0/4214223423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c6410 0x7fcae0158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae5399640 1 -- 172.21.15.120:0/4214223423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c6410 msgr2=0x7fcae0158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae5399640 1 --2- 172.21.15.120:0/4214223423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c6410 0x7fcae0158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae5399640 1 -- 172.21.15.120:0/4214223423 >> v2:172.21.15.134:3300/0 conn(0x7fcae00c5280 msgr2=0x7fcae00c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae5399640 1 --2- 172.21.15.120:0/4214223423 >> v2:172.21.15.134:3300/0 conn(0x7fcae00c5280 0x7fcae00c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae5399640 1 -- 172.21.15.120:0/4214223423 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae00c86f0 con 0x7fcae0151a90 2024-09-10T10:27:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae5b9a640 1 --2- 172.21.15.120:0/4214223423 >> v2:172.21.15.134:3300/0 conn(0x7fcae00c5280 0x7fcae00c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.455+0000 7fcae4b98640 1 --2- 172.21.15.120:0/4214223423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c6410 0x7fcae0158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:23.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.456+0000 7fcae5399640 1 --2- 172.21.15.120:0/4214223423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 0x7fcae00c36b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcae015bd40 tx=0x7fcadc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66c8d79c7c91b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:23.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.456+0000 7fcad7fff640 1 -- 172.21.15.120:0/4214223423 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc037070 con 0x7fcae0151a90 2024-09-10T10:27:23.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.456+0000 7fcad7fff640 1 -- 172.21.15.120:0/4214223423 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcadc032440 con 0x7fcae0151a90 2024-09-10T10:27:23.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.456+0000 7fcad7fff640 1 -- 172.21.15.120:0/4214223423 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc032810 con 0x7fcae0151a90 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.457+0000 7fcae75fd640 1 -- 172.21.15.120:0/4214223423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 msgr2=0x7fcae00c36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.457+0000 7fcae75fd640 1 --2- 172.21.15.120:0/4214223423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 0x7fcae00c36b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcae015bd40 tx=0x7fcadc02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.457+0000 7fcae75fd640 1 -- 172.21.15.120:0/4214223423 shutdown_connections 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.457+0000 7fcae75fd640 1 --2- 172.21.15.120:0/4214223423 >> v2:172.21.15.134:3300/0 conn(0x7fcae00c5280 0x7fcae00c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.457+0000 7fcae75fd640 1 --2- 172.21.15.120:0/4214223423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c6410 0x7fcae0158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.457+0000 7fcae75fd640 1 --2- 172.21.15.120:0/4214223423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 0x7fcae00c36b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.457+0000 7fcae75fd640 1 -- 172.21.15.120:0/4214223423 >> 172.21.15.120:0/4214223423 conn(0x7fcae006f3c0 msgr2=0x7fcae006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.457+0000 7fcae75fd640 1 -- 172.21.15.120:0/4214223423 shutdown_connections 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.457+0000 7fcae75fd640 1 -- 172.21.15.120:0/4214223423 wait complete. 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.458+0000 7fcae75fd640 1 Processor -- start 2024-09-10T10:27:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.458+0000 7fcae75fd640 1 -- start start 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae75fd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5280 0x7fcae01563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae75fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00c6410 0x7fcae01ede60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae75fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 0x7fcae00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae75fd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcae0157b20 con 0x7fcae0151a90 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae75fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcae0157840 con 0x7fcae00c6410 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae75fd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcae00c3bf0 con 0x7fcae00c5280 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae5b9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 0x7fcae00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae5b9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 0x7fcae00cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56122/0 (socket says 172.21.15.120:56122) 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae5b9a640 1 -- 172.21.15.120:0/2776542218 learned_addr learned my addr 172.21.15.120:0/2776542218 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae5399640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5280 0x7fcae01563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae4b98640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00c6410 0x7fcae01ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae5b9a640 1 -- 172.21.15.120:0/2776542218 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5280 msgr2=0x7fcae01563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae5b9a640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5280 0x7fcae01563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae5b9a640 1 -- 172.21.15.120:0/2776542218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00c6410 msgr2=0x7fcae01ede60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae5b9a640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00c6410 0x7fcae01ede60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae5b9a640 1 -- 172.21.15.120:0/2776542218 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae01e4850 con 0x7fcae0151a90 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae5b9a640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 0x7fcae00cc430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcad8007e40 tx=0x7fcad800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcad5ffb640 1 -- 172.21.15.120:0/2776542218 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad801a070 con 0x7fcae0151a90 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcad5ffb640 1 -- 172.21.15.120:0/2776542218 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad8003b90 con 0x7fcae0151a90 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcad5ffb640 1 -- 172.21.15.120:0/2776542218 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad800de80 con 0x7fcae0151a90 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.459+0000 7fcae75fd640 1 -- 172.21.15.120:0/2776542218 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae01f1f80 con 0x7fcae0151a90 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.460+0000 7fcae75fd640 1 -- 172.21.15.120:0/2776542218 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae01f2260 con 0x7fcae0151a90 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.461+0000 7fcae75fd640 1 -- 172.21.15.120:0/2776542218 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae00c71f0 con 0x7fcae0151a90 2024-09-10T10:27:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.461+0000 7fcad5ffb640 1 -- 172.21.15.120:0/2776542218 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcad801d070 con 0x7fcae0151a90 2024-09-10T10:27:23.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.461+0000 7fcad5ffb640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcac406be60 0x7fcac406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.461+0000 7fcad5ffb640 1 -- 172.21.15.120:0/2776542218 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcad808b440 con 0x7fcae0151a90 2024-09-10T10:27:23.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.461+0000 7fcae5399640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcac406be60 0x7fcac406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.462+0000 7fcae5399640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcac406be60 0x7fcac406e320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcadc034000 tx=0x7fcadc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:23.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.465+0000 7fcad5ffb640 1 -- 172.21.15.120:0/2776542218 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcad8010070 con 0x7fcae0151a90 2024-09-10T10:27:23.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.660+0000 7fcae75fd640 1 -- 172.21.15.120:0/2776542218 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcae0152140 con 0x7fcae0151a90 2024-09-10T10:27:23.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.661+0000 7fcad5ffb640 1 -- 172.21.15.120:0/2776542218 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6958 (secure 0 0 0) 0x7fcad80593a0 con 0x7fcae0151a90 2024-09-10T10:27:23.662 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:23.663 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":17,"btime":"2024-09-10T10:27:20:935106+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:26:41.113510+0000","modified":"2024-09-10T10:27:20.935087+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4303,"mds_2":4288,"mds_3":4284,"mds_4":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":4,"incarnation":17,"state":"up:creating","state_seq":1,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":13,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306,4303,4288,4284]},"id":1}]} 2024-09-10T10:27:23.663 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 17 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 -- 172.21.15.120:0/2776542218 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcac406be60 msgr2=0x7fcac406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcac406be60 0x7fcac406e320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcadc034000 tx=0x7fcadc02e000 comp rx=0 tx=0).stop 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 -- 172.21.15.120:0/2776542218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 msgr2=0x7fcae00cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 0x7fcae00cc430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcad8007e40 tx=0x7fcad800d4a0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 -- 172.21.15.120:0/2776542218 shutdown_connections 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcac406be60 0x7fcac406e320 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcae0151a90 0x7fcae00cc430 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00c6410 0x7fcae01ede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 --2- 172.21.15.120:0/2776542218 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5280 0x7fcae01563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 -- 172.21.15.120:0/2776542218 >> 172.21.15.120:0/2776542218 conn(0x7fcae006f3c0 msgr2=0x7fcae01e0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 -- 172.21.15.120:0/2776542218 shutdown_connections 2024-09-10T10:27:23.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.663+0000 7fcae75fd640 1 -- 172.21.15.120:0/2776542218 wait complete. 2024-09-10T10:27:23.676 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:27:23.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.773+0000 7f9dc180a640 1 Processor -- start 2024-09-10T10:27:23.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.773+0000 7f9dc180a640 1 -- start start 2024-09-10T10:27:23.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.774+0000 7f9dc180a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9dbc0ca3d0 0x7f9dbc15e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.774+0000 7f9dc180a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0ca7b0 0x7f9dbc0c6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.774+0000 7f9dc180a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dbc15ead0 con 0x7f9dbc0c8a70 2024-09-10T10:27:23.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.774+0000 7f9dc180a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9dbc0c9240 con 0x7f9dbc0ca3d0 2024-09-10T10:27:23.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.774+0000 7f9dc180a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9dbc15edb0 con 0x7f9dbc0ca7b0 2024-09-10T10:27:23.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.774+0000 7f9dbb7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f9dbc0c8a70 0x7f9dbc06d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:43806/0 (socket says 172.21.15.120:43806) 2024-09-10T10:27:23.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.774+0000 7f9dbb7fe640 1 -- 172.21.15.120:0/1532554745 learned_addr learned my addr 172.21.15.120:0/1532554745 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:23.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4029404246 0 0) 0x7f9dbc15ead0 con 0x7f9dbc0c8a70 2024-09-10T10:27:23.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9da4003520 con 0x7f9dbc0c8a70 2024-09-10T10:27:23.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dba7fc640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0ca7b0 0x7f9dbc0c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dbaffd640 1 --2- 172.21.15.120:0/1532554745 >> v2:172.21.15.134:3300/0 conn(0x7f9dbc0ca3d0 0x7f9dbc15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4191646166 0 0) 0x7f9da4003520 con 0x7f9dbc0c8a70 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0ca7b0 msgr2=0x7f9dbc0c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0ca7b0 0x7f9dbc0c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 >> v2:172.21.15.134:3300/0 conn(0x7f9dbc0ca3d0 msgr2=0x7f9dbc15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 --2- 172.21.15.120:0/1532554745 >> v2:172.21.15.134:3300/0 conn(0x7f9dbc0ca3d0 0x7f9dbc15e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc15f090 con 0x7f9dbc0c8a70 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f9db4001670 con 0x7f9dbc0c8a70 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dbaffd640 1 --2- 172.21.15.120:0/1532554745 >> v2:172.21.15.134:3300/0 conn(0x7f9dbc0ca3d0 0x7f9dbc15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 >> v1:172.21.15.120:6789/0 conn(0x7f9dbc0c8a70 legacy=0x7f9dbc06d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da40039b0 0x7f9da4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0ca7b0 0x7f9da4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 0x7f9da40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc0c9240 con 0x7f9da4004d40 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc15edb0 con 0x7f9da40039b0 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9da400baa0 con 0x7f9dbc0ca7b0 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dbb7fe640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 0x7f9da40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dbaffd640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da40039b0 0x7f9da4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dba7fc640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0ca7b0 0x7f9da4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dbb7fe640 1 -- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0ca7b0 msgr2=0x7f9da4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dbb7fe640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0ca7b0 0x7f9da4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dbb7fe640 1 -- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da40039b0 msgr2=0x7f9da4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dbb7fe640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da40039b0 0x7f9da4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.775+0000 7f9dbb7fe640 1 -- 172.21.15.120:0/1532554745 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da4003520 con 0x7f9da4004d40 2024-09-10T10:27:23.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.776+0000 7f9dbb7fe640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 0x7f9da40051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c240 tx=0x7f9da4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b2ded54dbc0220 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.776+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4007d30 con 0x7f9da4004d40 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.776+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db40058d0 con 0x7f9da4004d40 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.776+0000 7f9db9ffb640 1 -- 172.21.15.120:0/1532554745 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4005d00 con 0x7f9da4004d40 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.777+0000 7f9dc180a640 1 -- 172.21.15.120:0/1532554745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 msgr2=0x7f9da40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.777+0000 7f9dc180a640 1 --2- 172.21.15.120:0/1532554745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 0x7f9da40051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c240 tx=0x7f9da4005830 comp rx=0 tx=0).stop 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.777+0000 7f9dbb7fe640 1 -- 172.21.15.120:0/1532554745 reap_dead start 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.777+0000 7f9dc180a640 1 -- 172.21.15.120:0/1532554745 shutdown_connections 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.777+0000 7f9dc180a640 1 -- 172.21.15.120:0/1532554745 >> 172.21.15.120:0/1532554745 conn(0x7f9dbc0cac70 msgr2=0x7f9dbc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.777+0000 7f9dc180a640 1 -- 172.21.15.120:0/1532554745 shutdown_connections 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.777+0000 7f9dc180a640 1 -- 172.21.15.120:0/1532554745 wait complete. 2024-09-10T10:27:23.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.778+0000 7f9dc180a640 1 Processor -- start 2024-09-10T10:27:23.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.778+0000 7f9dc180a640 1 -- start start 2024-09-10T10:27:23.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dc180a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 0x7f9dbc0c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dc180a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc0c20e0 0x7f9dbc1f1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dc180a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc1f4590 0x7f9dbc1f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dc180a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc0cba40 con 0x7f9da4004d40 2024-09-10T10:27:23.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dc180a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc0c6420 con 0x7f9dbc0c20e0 2024-09-10T10:27:23.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dc180a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9dbc0cbbc0 con 0x7f9dbc1f4590 2024-09-10T10:27:23.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 0x7f9dbc0c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 0x7f9dbc0c44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56136/0 (socket says 172.21.15.120:56136) 2024-09-10T10:27:23.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbaffd640 1 -- 172.21.15.120:0/164935525 learned_addr learned my addr 172.21.15.120:0/164935525 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbb7fe640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc1f4590 0x7f9dbc1f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dba7fc640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc0c20e0 0x7f9dbc1f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbaffd640 1 -- 172.21.15.120:0/164935525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc1f4590 msgr2=0x7f9dbc1f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbaffd640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc1f4590 0x7f9dbc1f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbaffd640 1 -- 172.21.15.120:0/164935525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc0c20e0 msgr2=0x7f9dbc1f1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbaffd640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc0c20e0 0x7f9dbc1f1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbaffd640 1 -- 172.21.15.120:0/164935525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc1f18d0 con 0x7f9da4004d40 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbb7fe640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc1f4590 0x7f9dbc1f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.779+0000 7f9dbaffd640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 0x7f9dbc0c44e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9dac007f70 tx=0x7f9dac0105f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.780+0000 7f9da3fff640 1 -- 172.21.15.120:0/164935525 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac003990 con 0x7f9da4004d40 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.780+0000 7f9da3fff640 1 -- 172.21.15.120:0/164935525 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dac003b30 con 0x7f9da4004d40 2024-09-10T10:27:23.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.780+0000 7f9da3fff640 1 -- 172.21.15.120:0/164935525 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac003d20 con 0x7f9da4004d40 2024-09-10T10:27:23.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.780+0000 7f9dc180a640 1 -- 172.21.15.120:0/164935525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dbc0c3470 con 0x7f9da4004d40 2024-09-10T10:27:23.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.780+0000 7f9dc180a640 1 -- 172.21.15.120:0/164935525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dbc0c50f0 con 0x7f9da4004d40 2024-09-10T10:27:23.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.781+0000 7f9dc180a640 1 -- 172.21.15.120:0/164935525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dbc0cb7d0 con 0x7f9da4004d40 2024-09-10T10:27:23.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.781+0000 7f9da3fff640 1 -- 172.21.15.120:0/164935525 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9dac01e020 con 0x7f9da4004d40 2024-09-10T10:27:23.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.781+0000 7f9da3fff640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da806be60 0x7f9da806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:23.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.782+0000 7f9da3fff640 1 -- 172.21.15.120:0/164935525 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9dac08ba80 con 0x7f9da4004d40 2024-09-10T10:27:23.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.782+0000 7f9dba7fc640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da806be60 0x7f9da806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:23.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.782+0000 7f9dba7fc640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da806be60 0x7f9da806e320 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc1590d0 tx=0x7f9db0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:23.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.786+0000 7f9da3fff640 1 -- 172.21.15.120:0/164935525 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9dbc0cb7d0 con 0x7f9da4004d40 2024-09-10T10:27:23.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.993+0000 7f9dc180a640 1 -- 172.21.15.120:0/164935525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dbc06e5a0 con 0x7f9da4004d40 2024-09-10T10:27:23.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.993+0000 7f9da3fff640 1 -- 172.21.15.120:0/164935525 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6958 (secure 0 0 0) 0x7f9dbc06e5a0 con 0x7f9da4004d40 2024-09-10T10:27:23.995 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:23.995 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":17,"btime":"2024-09-10T10:27:20:935106+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:26:41.113510+0000","modified":"2024-09-10T10:27:20.935087+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4303,"mds_2":4288,"mds_3":4284,"mds_4":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":4,"incarnation":17,"state":"up:creating","state_seq":1,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":13,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306,4303,4288,4284]},"id":1}]} 2024-09-10T10:27:23.995 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 17 2024-09-10T10:27:23.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.995+0000 7f9dc180a640 1 -- 172.21.15.120:0/164935525 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da806be60 msgr2=0x7f9da806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.995+0000 7f9dc180a640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da806be60 0x7f9da806e320 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc1590d0 tx=0x7f9db0008040 comp rx=0 tx=0).stop 2024-09-10T10:27:23.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.995+0000 7f9dc180a640 1 -- 172.21.15.120:0/164935525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 msgr2=0x7f9dbc0c44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:23.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.995+0000 7f9dc180a640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 0x7f9dbc0c44e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9dac007f70 tx=0x7f9dac0105f0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.995+0000 7f9dc180a640 1 -- 172.21.15.120:0/164935525 shutdown_connections 2024-09-10T10:27:23.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.995+0000 7f9dc180a640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da806be60 0x7f9da806e320 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.995+0000 7f9dc180a640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc1f4590 0x7f9dbc1f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.995+0000 7f9dc180a640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc0c20e0 0x7f9dbc1f1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.995+0000 7f9dc180a640 1 --2- 172.21.15.120:0/164935525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da4004d40 0x7f9dbc0c44e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:23.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.995+0000 7f9dc180a640 1 -- 172.21.15.120:0/164935525 >> 172.21.15.120:0/164935525 conn(0x7f9dbc0cac70 msgr2=0x7f9dbc158100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:23.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.996+0000 7f9dc180a640 1 -- 172.21.15.120:0/164935525 shutdown_connections 2024-09-10T10:27:23.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:23.996+0000 7f9dc180a640 1 -- 172.21.15.120:0/164935525 wait complete. 2024-09-10T10:27:24.008 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:27:24.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.105+0000 7f267f59e640 1 Processor -- start 2024-09-10T10:27:24.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.105+0000 7f267f59e640 1 -- start start 2024-09-10T10:27:24.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267f59e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f26800c6b70 0x7f268015b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267f59e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f26800cc040 0x7f26800cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267f59e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f268006d140 con 0x7f26800c75a0 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267f59e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f268015b880 con 0x7f26800cc040 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267f59e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f26800cab20 con 0x7f26800c6b70 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267ed9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f26800cc040 0x7f26800cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267ed9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f26800cc040 0x7f26800cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56152/0 (socket says 172.21.15.120:56152) 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267ed9d640 1 -- 172.21.15.120:0/1865407966 learned_addr learned my addr 172.21.15.120:0/1865407966 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267dd9b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f26800c75a0 0x7f26800cb570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:43810/0 (socket says 172.21.15.120:43810) 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267e59c640 1 --2- 172.21.15.120:0/1865407966 >> v2:172.21.15.134:3300/0 conn(0x7f26800c6b70 0x7f268015b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267ed9d640 1 -- 172.21.15.120:0/1865407966 >> v2:172.21.15.134:3300/0 conn(0x7f26800c6b70 msgr2=0x7f268015b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267ed9d640 1 --2- 172.21.15.120:0/1865407966 >> v2:172.21.15.134:3300/0 conn(0x7f26800c6b70 0x7f268015b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267ed9d640 1 -- 172.21.15.120:0/1865407966 >> v1:172.21.15.120:6789/0 conn(0x7f26800c75a0 legacy=0x7f26800cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267d59a640 1 -- 172.21.15.120:0/1865407966 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540821041 0 0) 0x7f268006d140 con 0x7f26800c75a0 2024-09-10T10:27:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.106+0000 7f267ed9d640 1 -- 172.21.15.120:0/1865407966 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268006d420 con 0x7f26800cc040 2024-09-10T10:27:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.107+0000 7f267ed9d640 1 --2- 172.21.15.120:0/1865407966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800cc040 0x7f26800cc4c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2680075ec0 tx=0x7f267802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7989e7b6b60223 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.107+0000 7f267d59a640 1 -- 172.21.15.120:0/1865407966 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678036070 con 0x7f26800cc040 2024-09-10T10:27:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.107+0000 7f267d59a640 1 -- 172.21.15.120:0/1865407966 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2678031600 con 0x7f26800cc040 2024-09-10T10:27:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.107+0000 7f267d59a640 1 -- 172.21.15.120:0/1865407966 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780319d0 con 0x7f26800cc040 2024-09-10T10:27:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.107+0000 7f267f59e640 1 -- 172.21.15.120:0/1865407966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800cc040 msgr2=0x7f26800cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.107+0000 7f267f59e640 1 --2- 172.21.15.120:0/1865407966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800cc040 0x7f26800cc4c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2680075ec0 tx=0x7f267802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.108+0000 7f267f59e640 1 -- 172.21.15.120:0/1865407966 shutdown_connections 2024-09-10T10:27:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.108+0000 7f267f59e640 1 --2- 172.21.15.120:0/1865407966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800cc040 0x7f26800cc4c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.108+0000 7f267f59e640 1 --2- 172.21.15.120:0/1865407966 >> v2:172.21.15.134:3300/0 conn(0x7f26800c6b70 0x7f268015b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.108+0000 7f267f59e640 1 -- 172.21.15.120:0/1865407966 >> 172.21.15.120:0/1865407966 conn(0x7f26800ccc20 msgr2=0x7f26800cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:24.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.108+0000 7f267f59e640 1 -- 172.21.15.120:0/1865407966 shutdown_connections 2024-09-10T10:27:24.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.108+0000 7f267f59e640 1 -- 172.21.15.120:0/1865407966 wait complete. 2024-09-10T10:27:24.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.109+0000 7f267f59e640 1 Processor -- start 2024-09-10T10:27:24.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.109+0000 7f267f59e640 1 -- start start 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267f59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800c6b70 0x7f268015cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26800c75a0 0x7f2680155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267f59e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26800cc040 0x7f2680155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267f59e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f26800cb050 con 0x7f26800c6b70 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267f59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26800cad70 con 0x7f26800c75a0 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267f59e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f26800cbc80 con 0x7f26800cc040 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800c6b70 0x7f268015cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800c6b70 0x7f268015cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56166/0 (socket says 172.21.15.120:56166) 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267e59c640 1 -- 172.21.15.120:0/1805028209 learned_addr learned my addr 172.21.15.120:0/1805028209 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267dd9b640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26800c75a0 0x7f2680155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267ed9d640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26800cc040 0x7f2680155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267e59c640 1 -- 172.21.15.120:0/1805028209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26800cc040 msgr2=0x7f2680155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267e59c640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26800cc040 0x7f2680155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267e59c640 1 -- 172.21.15.120:0/1805028209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26800c75a0 msgr2=0x7f2680155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267e59c640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26800c75a0 0x7f2680155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267e59c640 1 -- 172.21.15.120:0/1805028209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268015d1c0 con 0x7f26800c6b70 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267dd9b640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26800c75a0 0x7f2680155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267ed9d640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26800cc040 0x7f2680155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267e59c640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800c6b70 0x7f268015cc80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f267400e8a0 tx=0x7f267400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f266effd640 1 -- 172.21.15.120:0/1805028209 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674017070 con 0x7f26800c6b70 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f266effd640 1 -- 172.21.15.120:0/1805028209 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f267400c940 con 0x7f26800c6b70 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267f59e640 1 -- 172.21.15.120:0/1805028209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26801f1560 con 0x7f26800c6b70 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f266effd640 1 -- 172.21.15.120:0/1805028209 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267400cd30 con 0x7f26800c6b70 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.110+0000 7f267f59e640 1 -- 172.21.15.120:0/1805028209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268015bd00 con 0x7f26800c6b70 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.111+0000 7f267f59e640 1 -- 172.21.15.120:0/1805028209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268008de70 con 0x7f26800c6b70 2024-09-10T10:27:24.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.112+0000 7f266effd640 1 -- 172.21.15.120:0/1805028209 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f267401a070 con 0x7f26800c6b70 2024-09-10T10:27:24.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.112+0000 7f266effd640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f266006be60 0x7f266006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.112+0000 7f267dd9b640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f266006be60 0x7f266006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.112+0000 7f266effd640 1 -- 172.21.15.120:0/1805028209 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f267408aff0 con 0x7f26800c6b70 2024-09-10T10:27:24.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.113+0000 7f267dd9b640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f266006be60 0x7f266006e320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2680158430 tx=0x7f2670007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:24.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.117+0000 7f266effd640 1 -- 172.21.15.120:0/1805028209 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f268008de70 con 0x7f26800c6b70 2024-09-10T10:27:24.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.282+0000 7f267f59e640 1 -- 172.21.15.120:0/1805028209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f268006da90 con 0x7f26800c6b70 2024-09-10T10:27:24.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.283+0000 7f266effd640 1 -- 172.21.15.120:0/1805028209 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+19012 (secure 0 0 0) 0x7f2674059630 con 0x7f26800c6b70 2024-09-10T10:27:24.284 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:24.285 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:27:23.940655+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 -- 172.21.15.120:0/1805028209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f266006be60 msgr2=0x7f266006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f266006be60 0x7f266006e320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2680158430 tx=0x7f2670007ca0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 -- 172.21.15.120:0/1805028209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800c6b70 msgr2=0x7f268015cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800c6b70 0x7f268015cc80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f267400e8a0 tx=0x7f267400ed60 comp rx=0 tx=0).stop 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 -- 172.21.15.120:0/1805028209 shutdown_connections 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f266006be60 0x7f266006e320 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26800cc040 0x7f2680155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26800c75a0 0x7f2680155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 --2- 172.21.15.120:0/1805028209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26800c6b70 0x7f268015cc80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 -- 172.21.15.120:0/1805028209 >> 172.21.15.120:0/1805028209 conn(0x7f26800ccc20 msgr2=0x7f26800cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 -- 172.21.15.120:0/1805028209 shutdown_connections 2024-09-10T10:27:24.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.285+0000 7f267f59e640 1 -- 172.21.15.120:0/1805028209 wait complete. 2024-09-10T10:27:24.298 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:27:24.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.395+0000 7f2fe7df8640 1 Processor -- start 2024-09-10T10:27:24.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.395+0000 7f2fe7df8640 1 -- start start 2024-09-10T10:27:24.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.396+0000 7f2fe7df8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2fe015aab0 0x7f2fe00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.396+0000 7f2fe7df8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe015b1d0 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.396+0000 7f2fe7df8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe015b4b0 con 0x7f2fe00cb2f0 2024-09-10T10:27:24.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.396+0000 7f2fe7df8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe00cbd00 con 0x7f2fe015aab0 2024-09-10T10:27:24.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.396+0000 7f2fe6395640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2fe015aab0 0x7f2fe00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.396+0000 7f2fe5b94640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f2fe00cc0c0 0x7f2fe015e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:43812/0 (socket says 172.21.15.120:43812) 2024-09-10T10:27:24.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.396+0000 7f2fe6395640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2fe015aab0 0x7f2fe00cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56174/0 (socket says 172.21.15.120:56174) 2024-09-10T10:27:24.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.396+0000 7f2fe6395640 1 -- 172.21.15.120:0/2457200684 learned_addr learned my addr 172.21.15.120:0/2457200684 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:24.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.397+0000 7f2fe6395640 1 -- 172.21.15.120:0/2457200684 >> v1:172.21.15.134:6789/0 conn(0x7f2fe00cb2f0 legacy=0x7f2fe0163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.397+0000 7f2fe6395640 1 -- 172.21.15.120:0/2457200684 >> v1:172.21.15.120:6789/0 conn(0x7f2fe00cc0c0 legacy=0x7f2fe015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.397+0000 7f2fe6395640 1 -- 172.21.15.120:0/2457200684 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe015b790 con 0x7f2fe015aab0 2024-09-10T10:27:24.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.397+0000 7f2fe4b92640 1 -- 172.21.15.120:0/2457200684 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1472222160 0 0) 0x7f2fe015b1d0 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.397+0000 7f2fe6395640 1 --2- 172.21.15.120:0/2457200684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe015aab0 0x7f2fe00cc4a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009870 tx=0x7f2fd802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8607337ced702e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:24.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.398+0000 7f2fe4b92640 1 -- 172.21.15.120:0/2457200684 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8037070 con 0x7f2fe015aab0 2024-09-10T10:27:24.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.398+0000 7f2fe4b92640 1 -- 172.21.15.120:0/2457200684 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd802ae50 con 0x7f2fe015aab0 2024-09-10T10:27:24.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.398+0000 7f2fe4b92640 1 -- 172.21.15.120:0/2457200684 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8032860 con 0x7f2fe015aab0 2024-09-10T10:27:24.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.398+0000 7f2fe7df8640 1 -- 172.21.15.120:0/2457200684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe015aab0 msgr2=0x7f2fe00cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.398+0000 7f2fe7df8640 1 --2- 172.21.15.120:0/2457200684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe015aab0 0x7f2fe00cc4a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009870 tx=0x7f2fd802a300 comp rx=0 tx=0).stop 2024-09-10T10:27:24.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.398+0000 7f2fe7df8640 1 -- 172.21.15.120:0/2457200684 shutdown_connections 2024-09-10T10:27:24.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.398+0000 7f2fe7df8640 1 --2- 172.21.15.120:0/2457200684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe015aab0 0x7f2fe00cc4a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.398+0000 7f2fe7df8640 1 -- 172.21.15.120:0/2457200684 >> 172.21.15.120:0/2457200684 conn(0x7f2fe006d560 msgr2=0x7f2fe006d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:24.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.398+0000 7f2fe7df8640 1 -- 172.21.15.120:0/2457200684 shutdown_connections 2024-09-10T10:27:24.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.398+0000 7f2fe7df8640 1 -- 172.21.15.120:0/2457200684 wait complete. 2024-09-10T10:27:24.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.399+0000 7f2fe7df8640 1 Processor -- start 2024-09-10T10:27:24.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.399+0000 7f2fe7df8640 1 -- start start 2024-09-10T10:27:24.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe7df8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fe00cb2f0 0x7f2fe015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe7df8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe00cc0c0 0x7f2fe014b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe7df8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fe015aab0 0x7f2fe01f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe7df8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe015e540 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe7df8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe015ba70 con 0x7f2fe015aab0 2024-09-10T10:27:24.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe7df8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe015aec0 con 0x7f2fe00cb2f0 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe5393640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe00cc0c0 0x7f2fe014b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe5393640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe00cc0c0 0x7f2fe014b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56178/0 (socket says 172.21.15.120:56178) 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe5393640 1 -- 172.21.15.120:0/3119438919 learned_addr learned my addr 172.21.15.120:0/3119438919 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe5b94640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fe00cb2f0 0x7f2fe015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe6395640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fe015aab0 0x7f2fe01f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.400+0000 7f2fe5393640 1 -- 172.21.15.120:0/3119438919 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fe00cb2f0 msgr2=0x7f2fe015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fe5393640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fe00cb2f0 0x7f2fe015da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fe5393640 1 -- 172.21.15.120:0/3119438919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fe015aab0 msgr2=0x7f2fe01f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fe5393640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fe015aab0 0x7f2fe01f3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fe5393640 1 -- 172.21.15.120:0/3119438919 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe014d9f0 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fe6395640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fe015aab0 0x7f2fe01f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fe5393640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe00cc0c0 0x7f2fe014b270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c430 tx=0x7f2fd400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:24.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fde7fc640 1 -- 172.21.15.120:0/3119438919 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4017070 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fde7fc640 1 -- 172.21.15.120:0/3119438919 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd40052d0 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fe7df8640 1 -- 172.21.15.120:0/3119438919 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe01f05d0 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fde7fc640 1 -- 172.21.15.120:0/3119438919 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4003830 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.401+0000 7f2fe7df8640 1 -- 172.21.15.120:0/3119438919 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe01f0b90 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.402+0000 7f2fe7df8640 1 -- 172.21.15.120:0/3119438919 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe015b1d0 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.402+0000 7f2fde7fc640 1 -- 172.21.15.120:0/3119438919 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fd401a070 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.403+0000 7f2fde7fc640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2fcc06be60 0x7f2fcc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:24.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.403+0000 7f2fde7fc640 1 -- 172.21.15.120:0/3119438919 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f2fd408a600 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.403+0000 7f2fe5b94640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2fcc06be60 0x7f2fcc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:24.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.403+0000 7f2fe5b94640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2fcc06be60 0x7f2fcc06e320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00012e0 tx=0x7f2fd0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:24.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.407+0000 7f2fde7fc640 1 -- 172.21.15.120:0/3119438919 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2fe015b1d0 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.604+0000 7f2fe7df8640 1 -- 172.21.15.120:0/3119438919 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fe00c5560 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.604+0000 7f2fde7fc640 1 -- 172.21.15.120:0/3119438919 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6958 (secure 0 0 0) 0x7f2fd4058b20 con 0x7f2fe00cc0c0 2024-09-10T10:27:24.605 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:24.606 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":17,"btime":"2024-09-10T10:27:20:935106+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:26:41.113510+0000","modified":"2024-09-10T10:27:20.935087+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4303,"mds_2":4288,"mds_3":4284,"mds_4":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":4,"incarnation":17,"state":"up:creating","state_seq":1,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":13,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306,4303,4288,4284]},"id":1}]} 2024-09-10T10:27:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 17 2024-09-10T10:27:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 -- 172.21.15.120:0/3119438919 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2fcc06be60 msgr2=0x7f2fcc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2fcc06be60 0x7f2fcc06e320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00012e0 tx=0x7f2fd0009040 comp rx=0 tx=0).stop 2024-09-10T10:27:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 -- 172.21.15.120:0/3119438919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe00cc0c0 msgr2=0x7f2fe014b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe00cc0c0 0x7f2fe014b270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c430 tx=0x7f2fd400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 -- 172.21.15.120:0/3119438919 shutdown_connections 2024-09-10T10:27:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2fcc06be60 0x7f2fcc06e320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fe015aab0 0x7f2fe01f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fe00cc0c0 0x7f2fe014b270 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 --2- 172.21.15.120:0/3119438919 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fe00cb2f0 0x7f2fe015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 -- 172.21.15.120:0/3119438919 >> 172.21.15.120:0/3119438919 conn(0x7f2fe006d560 msgr2=0x7f2fe00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 -- 172.21.15.120:0/3119438919 shutdown_connections 2024-09-10T10:27:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:24.606+0000 7f2fe7df8640 1 -- 172.21.15.120:0/3119438919 wait complete. 2024-09-10T10:27:25.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:26:41.113510+0000', 'modified': '2024-09-10T10:27:20.935087+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4306, 'mds_1': 4303, 'mds_2': 4288, 'mds_3': 4284, 'mds_4': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'c', 'rank': 4, 'incarnation': 17, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.120:6836/2787145148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6835', 'nonce': 2787145148}, {'type': 'v1', 'addr': '172.21.15.120:6836', 'nonce': 2787145148}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4284': {'gid': 4284, 'name': 'e', 'rank': 3, 'incarnation': 15, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.120:6839/1710396335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1710396335}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1710396335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4288': {'gid': 4288, 'name': 'a', 'rank': 2, 'incarnation': 13, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.120:6837/3208935627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 3208935627}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 3208935627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4303': {'gid': 4303, 'name': 'd', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.134:6834/231686968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 231686968}, {'type': 'v1', 'addr': '172.21.15.134:6834', 'nonce': 231686968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4306': {'gid': 4306, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6835/1434442356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6833', 'nonce': 1434442356}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1434442356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4306, 'qdb_cluster': [4306, 4303, 4288, 4284]} 2024-09-10T10:27:25.222 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4281:up:creating 2024-09-10T10:27:26.222 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:27:26.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.331+0000 7f0a67842640 1 Processor -- start 2024-09-10T10:27:26.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.331+0000 7f0a67842640 1 -- start start 2024-09-10T10:27:26.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.332+0000 7f0a67842640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a600c6980 0x7f0a600cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:26.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.332+0000 7f0a67842640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6015b1a0 con 0x7f0a600cb410 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.332+0000 7f0a67842640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6015b420 con 0x7f0a600c7770 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.332+0000 7f0a67842640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6015b9e0 con 0x7f0a600c6980 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.332+0000 7f0a65ddf640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f0a600cb410 0x7f0a600cbfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:43816/0 (socket says 172.21.15.120:43816) 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.332+0000 7f0a65ddf640 1 -- 172.21.15.120:0/1245148775 learned_addr learned my addr 172.21.15.120:0/1245148775 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a655de640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a600c6980 0x7f0a600cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3273914926 0 0) 0x7f0a58001490 con 0x7f0a600cb410 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a4c003520 con 0x7f0a600cb410 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873907689 0 0) 0x7f0a6015b420 con 0x7f0a600c7770 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a58001490 con 0x7f0a600c7770 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 740777556 0 0) 0x7f0a4c003520 con 0x7f0a600cb410 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a600c6980 msgr2=0x7f0a600cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:26.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a600c6980 0x7f0a600cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 >> v1:172.21.15.134:6789/0 conn(0x7f0a600c7770 legacy=0x7f0a6015aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6015b700 con 0x7f0a600cb410 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f0a58001bc0 con 0x7f0a600cb410 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 >> v1:172.21.15.120:6789/0 conn(0x7f0a600cb410 legacy=0x7f0a600cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a4c003b70 0x7f0a4c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a4c004800 0x7f0a4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a600c6980 0x7f0a4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6015b9e0 con 0x7f0a4c003b70 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c014280 con 0x7f0a4c004800 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4c014450 con 0x7f0a600c6980 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a655de640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a4c003b70 0x7f0a4c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a64ddd640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a4c004800 0x7f0a4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a65ddf640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a600c6980 0x7f0a4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:26.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a655de640 1 -- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a600c6980 msgr2=0x7f0a4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:26.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a655de640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a600c6980 0x7f0a4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:26.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a655de640 1 -- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a4c004800 msgr2=0x7f0a4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:26.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a655de640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a4c004800 0x7f0a4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:26.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.333+0000 7f0a655de640 1 -- 172.21.15.120:0/1245148775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c003520 con 0x7f0a4c003b70 2024-09-10T10:27:26.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.334+0000 7f0a655de640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a4c003b70 0x7f0a4c003f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009ac0 tx=0x7f0a4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d579fc8b232c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:26.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.334+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c036070 con 0x7f0a4c003b70 2024-09-10T10:27:26.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.334+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5c031400 con 0x7f0a4c003b70 2024-09-10T10:27:26.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.334+0000 7f0a57fff640 1 -- 172.21.15.120:0/1245148775 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c009690 con 0x7f0a4c003b70 2024-09-10T10:27:26.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.335+0000 7f0a67842640 1 -- 172.21.15.120:0/1245148775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a4c003b70 msgr2=0x7f0a4c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.335+0000 7f0a67842640 1 --2- 172.21.15.120:0/1245148775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a4c003b70 0x7f0a4c003f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009ac0 tx=0x7f0a4c005830 comp rx=0 tx=0).stop 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.335+0000 7f0a65ddf640 1 -- 172.21.15.120:0/1245148775 reap_dead start 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.335+0000 7f0a67842640 1 -- 172.21.15.120:0/1245148775 shutdown_connections 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.335+0000 7f0a67842640 1 -- 172.21.15.120:0/1245148775 >> 172.21.15.120:0/1245148775 conn(0x7f0a6006cb70 msgr2=0x7f0a6006e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.335+0000 7f0a67842640 1 -- 172.21.15.120:0/1245148775 shutdown_connections 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.335+0000 7f0a67842640 1 -- 172.21.15.120:0/1245148775 wait complete. 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.336+0000 7f0a67842640 1 Processor -- start 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.336+0000 7f0a67842640 1 -- start start 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a67842640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4c003b70 0x7f0a6006f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a67842640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a601ec390 0x7f0a6006fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a67842640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a600c4220 0x7f0a60153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a67842640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0a600cd210 con 0x7f0a601ec390 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a67842640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a600ccf30 con 0x7f0a600c4220 2024-09-10T10:27:26.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a67842640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0a600d0fe0 con 0x7f0a4c003b70 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a64ddd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a601ec390 0x7f0a6006fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a64ddd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a601ec390 0x7f0a6006fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56200/0 (socket says 172.21.15.120:56200) 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a64ddd640 1 -- 172.21.15.120:0/1152338285 learned_addr learned my addr 172.21.15.120:0/1152338285 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a655de640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4c003b70 0x7f0a6006f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a65ddf640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a600c4220 0x7f0a60153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a64ddd640 1 -- 172.21.15.120:0/1152338285 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4c003b70 msgr2=0x7f0a6006f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a64ddd640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4c003b70 0x7f0a6006f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a64ddd640 1 -- 172.21.15.120:0/1152338285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a600c4220 msgr2=0x7f0a60153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a64ddd640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a600c4220 0x7f0a60153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a64ddd640 1 -- 172.21.15.120:0/1152338285 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a601ec880 con 0x7f0a601ec390 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.337+0000 7f0a64ddd640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a601ec390 0x7f0a6006fe10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000f7a0 tx=0x7f0a5000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.338+0000 7f0a55ffb640 1 -- 172.21.15.120:0/1152338285 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5000fe50 con 0x7f0a601ec390 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.338+0000 7f0a55ffb640 1 -- 172.21.15.120:0/1152338285 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a50018070 con 0x7f0a601ec390 2024-09-10T10:27:26.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.338+0000 7f0a55ffb640 1 -- 172.21.15.120:0/1152338285 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5000d6d0 con 0x7f0a601ec390 2024-09-10T10:27:26.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.338+0000 7f0a67842640 1 -- 172.21.15.120:0/1152338285 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a600c2ca0 con 0x7f0a601ec390 2024-09-10T10:27:26.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.338+0000 7f0a67842640 1 -- 172.21.15.120:0/1152338285 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6006f0d0 con 0x7f0a601ec390 2024-09-10T10:27:26.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.339+0000 7f0a67842640 1 -- 172.21.15.120:0/1152338285 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a600cd730 con 0x7f0a601ec390 2024-09-10T10:27:26.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.339+0000 7f0a55ffb640 1 -- 172.21.15.120:0/1152338285 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a5000d870 con 0x7f0a601ec390 2024-09-10T10:27:26.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.340+0000 7f0a55ffb640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0a4406be60 0x7f0a4406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:26.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.340+0000 7f0a55ffb640 1 -- 172.21.15.120:0/1152338285 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f0a5008b000 con 0x7f0a601ec390 2024-09-10T10:27:26.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.340+0000 7f0a655de640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0a4406be60 0x7f0a4406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:26.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.340+0000 7f0a655de640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0a4406be60 0x7f0a4406e320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c033000 tx=0x7f0a5c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:26.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.344+0000 7f0a55ffb640 1 -- 172.21.15.120:0/1152338285 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0a50058ed0 con 0x7f0a601ec390 2024-09-10T10:27:26.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.542+0000 7f0a67842640 1 -- 172.21.15.120:0/1152338285 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a600cbdc0 con 0x7f0a601ec390 2024-09-10T10:27:26.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.543+0000 7f0a55ffb640 1 -- 172.21.15.120:0/1152338285 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6958 (secure 0 0 0) 0x7f0a5005d1d0 con 0x7f0a601ec390 2024-09-10T10:27:26.544 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:26.544 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":17,"btime":"2024-09-10T10:27:20:935106+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:26:41.113510+0000","modified":"2024-09-10T10:27:20.935087+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4303,"mds_2":4288,"mds_3":4284,"mds_4":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":4,"incarnation":17,"state":"up:creating","state_seq":1,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":13,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306,4303,4288,4284]},"id":1}]} 2024-09-10T10:27:26.545 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 17 2024-09-10T10:27:26.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 -- 172.21.15.120:0/1152338285 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0a4406be60 msgr2=0x7f0a4406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:26.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0a4406be60 0x7f0a4406e320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c033000 tx=0x7f0a5c008030 comp rx=0 tx=0).stop 2024-09-10T10:27:26.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 -- 172.21.15.120:0/1152338285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a601ec390 msgr2=0x7f0a6006fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:26.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a601ec390 0x7f0a6006fe10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000f7a0 tx=0x7f0a5000fc60 comp rx=0 tx=0).stop 2024-09-10T10:27:26.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 -- 172.21.15.120:0/1152338285 shutdown_connections 2024-09-10T10:27:26.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0a4406be60 0x7f0a4406e320 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:26.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a600c4220 0x7f0a60153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:26.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a601ec390 0x7f0a6006fe10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:26.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 --2- 172.21.15.120:0/1152338285 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a4c003b70 0x7f0a6006f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:26.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 -- 172.21.15.120:0/1152338285 >> 172.21.15.120:0/1152338285 conn(0x7f0a6006cb70 msgr2=0x7f0a6015a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:26.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 -- 172.21.15.120:0/1152338285 shutdown_connections 2024-09-10T10:27:26.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:26.545+0000 7f0a67842640 1 -- 172.21.15.120:0/1152338285 wait complete. 2024-09-10T10:27:27.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:26:41.113510+0000', 'modified': '2024-09-10T10:27:20.935087+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4306, 'mds_1': 4303, 'mds_2': 4288, 'mds_3': 4284, 'mds_4': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'c', 'rank': 4, 'incarnation': 17, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.120:6836/2787145148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6835', 'nonce': 2787145148}, {'type': 'v1', 'addr': '172.21.15.120:6836', 'nonce': 2787145148}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4284': {'gid': 4284, 'name': 'e', 'rank': 3, 'incarnation': 15, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.120:6839/1710396335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1710396335}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1710396335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4288': {'gid': 4288, 'name': 'a', 'rank': 2, 'incarnation': 13, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.120:6837/3208935627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 3208935627}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 3208935627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4303': {'gid': 4303, 'name': 'd', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.134:6834/231686968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 231686968}, {'type': 'v1', 'addr': '172.21.15.134:6834', 'nonce': 231686968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4306': {'gid': 4306, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6835/1434442356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6833', 'nonce': 1434442356}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1434442356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4306, 'qdb_cluster': [4306, 4303, 4288, 4284]} 2024-09-10T10:27:27.161 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4281:up:creating 2024-09-10T10:27:28.162 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:27:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.271+0000 7f150e1e4640 1 Processor -- start 2024-09-10T10:27:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.271+0000 7f150e1e4640 1 -- start start 2024-09-10T10:27:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f150e1e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080c6980 0x7f15080cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f150e1e4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f15080cb410 0x7f15080cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f150e1e4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150815b1a0 con 0x7f15080c7770 2024-09-10T10:27:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f150e1e4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f150815b9e0 con 0x7f15080cb410 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f150e1e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f150815b420 con 0x7f15080c6980 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f1507fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f15080cb410 0x7f15080cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f15077fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080c6980 0x7f15080cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f1506ffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f15080c7770 0x7f150815aa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:39160/0 (socket says 172.21.15.120:39160) 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f1507fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f15080cb410 0x7f15080cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:33362/0 (socket says 172.21.15.120:33362) 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f1506ffd640 1 -- 172.21.15.120:0/1850093232 learned_addr learned my addr 172.21.15.120:0/1850093232 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f1507fff640 1 -- 172.21.15.120:0/1850093232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080c6980 msgr2=0x7f15080cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f1507fff640 1 --2- 172.21.15.120:0/1850093232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080c6980 0x7f15080cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f1507fff640 1 -- 172.21.15.120:0/1850093232 >> v1:172.21.15.134:6789/0 conn(0x7f15080c7770 legacy=0x7f150815aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f1507fff640 1 -- 172.21.15.120:0/1850093232 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150815b700 con 0x7f15080cb410 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f15067fc640 1 -- 172.21.15.120:0/1850093232 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1783400453 0 0) 0x7f14fc001110 con 0x7f15080c7770 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.272+0000 7f15077fe640 1 --2- 172.21.15.120:0/1850093232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080c6980 0x7f15080cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.273+0000 7f1507fff640 1 --2- 172.21.15.120:0/1850093232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080cb410 0x7f15080cbfe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f15000097a0 tx=0x7f150002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eddde757cf8aad91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.273+0000 7f15067fc640 1 -- 172.21.15.120:0/1850093232 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500037070 con 0x7f15080cb410 2024-09-10T10:27:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.273+0000 7f15067fc640 1 -- 172.21.15.120:0/1850093232 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500032440 con 0x7f15080cb410 2024-09-10T10:27:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.273+0000 7f15067fc640 1 -- 172.21.15.120:0/1850093232 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500032810 con 0x7f15080cb410 2024-09-10T10:27:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.274+0000 7f150e1e4640 1 -- 172.21.15.120:0/1850093232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080cb410 msgr2=0x7f15080cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.274+0000 7f150e1e4640 1 --2- 172.21.15.120:0/1850093232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080cb410 0x7f15080cbfe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f15000097a0 tx=0x7f150002a180 comp rx=0 tx=0).stop 2024-09-10T10:27:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.274+0000 7f150e1e4640 1 -- 172.21.15.120:0/1850093232 shutdown_connections 2024-09-10T10:27:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.274+0000 7f150e1e4640 1 --2- 172.21.15.120:0/1850093232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080cb410 0x7f15080cbfe0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.274+0000 7f150e1e4640 1 --2- 172.21.15.120:0/1850093232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080c6980 0x7f15080cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.274+0000 7f150e1e4640 1 -- 172.21.15.120:0/1850093232 >> 172.21.15.120:0/1850093232 conn(0x7f150806cb70 msgr2=0x7f150806e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.274+0000 7f150e1e4640 1 -- 172.21.15.120:0/1850093232 shutdown_connections 2024-09-10T10:27:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.274+0000 7f150e1e4640 1 -- 172.21.15.120:0/1850093232 wait complete. 2024-09-10T10:27:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.275+0000 7f150e1e4640 1 Processor -- start 2024-09-10T10:27:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.275+0000 7f150e1e4640 1 -- start start 2024-09-10T10:27:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f150e1e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c6980 0x7f150815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f150e1e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15080c7770 0x7f1508154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f150e1e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080cb410 0x7f15081556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f150e1e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15080cd210 con 0x7f15080c7770 2024-09-10T10:27:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f150e1e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15080ccf30 con 0x7f15080c6980 2024-09-10T10:27:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f150e1e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f15080d0fe0 con 0x7f15080cb410 2024-09-10T10:27:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f1506ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15080c7770 0x7f1508154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f1506ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15080c7770 0x7f1508154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37912/0 (socket says 172.21.15.120:37912) 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f1506ffd640 1 -- 172.21.15.120:0/2458868163 learned_addr learned my addr 172.21.15.120:0/2458868163 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f15077fe640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c6980 0x7f150815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.276+0000 7f1507fff640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080cb410 0x7f15081556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f1506ffd640 1 -- 172.21.15.120:0/2458868163 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080cb410 msgr2=0x7f15081556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f1506ffd640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080cb410 0x7f15081556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f1506ffd640 1 -- 172.21.15.120:0/2458868163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c6980 msgr2=0x7f150815a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f1506ffd640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c6980 0x7f150815a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f1506ffd640 1 -- 172.21.15.120:0/2458868163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15081f01b0 con 0x7f15080c7770 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f1506ffd640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15080c7770 0x7f1508154810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00c440 tx=0x7f14fc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f14effff640 1 -- 172.21.15.120:0/2458868163 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc018070 con 0x7f15080c7770 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f14effff640 1 -- 172.21.15.120:0/2458868163 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14fc0074e0 con 0x7f15080c7770 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f150e1e4640 1 -- 172.21.15.120:0/2458868163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15081f0490 con 0x7f15080c7770 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f14effff640 1 -- 172.21.15.120:0/2458868163 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc004a20 con 0x7f15080c7770 2024-09-10T10:27:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.277+0000 7f150e1e4640 1 -- 172.21.15.120:0/2458868163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15081f0a50 con 0x7f15080c7770 2024-09-10T10:27:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.278+0000 7f150e1e4640 1 -- 172.21.15.120:0/2458868163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15080cd730 con 0x7f15080c7770 2024-09-10T10:27:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.278+0000 7f14effff640 1 -- 172.21.15.120:0/2458868163 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14fc01b070 con 0x7f15080c7770 2024-09-10T10:27:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.279+0000 7f14effff640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14f406be30 0x7f14f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.279+0000 7f14effff640 1 -- 172.21.15.120:0/2458868163 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f14fc08a3a0 con 0x7f15080c7770 2024-09-10T10:27:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.279+0000 7f15077fe640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14f406be30 0x7f14f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.279+0000 7f15077fe640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14f406be30 0x7f14f406e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009bd0 tx=0x7f14f800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.283+0000 7f14effff640 1 -- 172.21.15.120:0/2458868163 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f150815b1a0 con 0x7f15080c7770 2024-09-10T10:27:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.485+0000 7f150e1e4640 1 -- 172.21.15.120:0/2458868163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1508155400 con 0x7f15080c7770 2024-09-10T10:27:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.486+0000 7f14effff640 1 -- 172.21.15.120:0/2458868163 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6962 (secure 0 0 0) 0x7f14fc058390 con 0x7f15080c7770 2024-09-10T10:27:28.487 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:28.487 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":18,"btime":"2024-09-10T10:27:26:951215+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:26:41.113510+0000","modified":"2024-09-10T10:27:26.951208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4303,"mds_2":4288,"mds_3":4284,"mds_4":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":4,"incarnation":17,"state":"up:active","state_seq":14,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":13,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4281,4284,4288,4303,4306]},"id":1}]} 2024-09-10T10:27:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 18 2024-09-10T10:27:28.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 -- 172.21.15.120:0/2458868163 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14f406be30 msgr2=0x7f14f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:28.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14f406be30 0x7f14f406e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009bd0 tx=0x7f14f800b040 comp rx=0 tx=0).stop 2024-09-10T10:27:28.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 -- 172.21.15.120:0/2458868163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15080c7770 msgr2=0x7f1508154810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:28.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15080c7770 0x7f1508154810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00c440 tx=0x7f14fc00c900 comp rx=0 tx=0).stop 2024-09-10T10:27:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 -- 172.21.15.120:0/2458868163 shutdown_connections 2024-09-10T10:27:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f14f406be30 0x7f14f406e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15080cb410 0x7f15081556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15080c7770 0x7f1508154810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 --2- 172.21.15.120:0/2458868163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c6980 0x7f150815a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 -- 172.21.15.120:0/2458868163 >> 172.21.15.120:0/2458868163 conn(0x7f150806cb70 msgr2=0x7f15080d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 -- 172.21.15.120:0/2458868163 shutdown_connections 2024-09-10T10:27:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:28.488+0000 7f150e1e4640 1 -- 172.21.15.120:0/2458868163 wait complete. 2024-09-10T10:27:29.105 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:26:41.113510+0000', 'modified': '2024-09-10T10:27:26.951208+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4306, 'mds_1': 4303, 'mds_2': 4288, 'mds_3': 4284, 'mds_4': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'c', 'rank': 4, 'incarnation': 17, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.120:6836/2787145148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6835', 'nonce': 2787145148}, {'type': 'v1', 'addr': '172.21.15.120:6836', 'nonce': 2787145148}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4284': {'gid': 4284, 'name': 'e', 'rank': 3, 'incarnation': 15, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.120:6839/1710396335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1710396335}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1710396335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4288': {'gid': 4288, 'name': 'a', 'rank': 2, 'incarnation': 13, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.120:6837/3208935627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 3208935627}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 3208935627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4303': {'gid': 4303, 'name': 'd', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.134:6834/231686968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 231686968}, {'type': 'v1', 'addr': '172.21.15.134:6834', 'nonce': 231686968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4306': {'gid': 4306, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6835/1434442356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6833', 'nonce': 1434442356}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1434442356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4306, 'qdb_cluster': [4281, 4284, 4288, 4303, 4306]} 2024-09-10T10:27:29.105 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T10:27:29.105 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:27:29.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.205+0000 7faeeb723640 1 Processor -- start 2024-09-10T10:27:29.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.206+0000 7faeeb723640 1 -- start start 2024-09-10T10:27:29.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.206+0000 7faeeb723640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7faee40c0760 0x7faee406cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.206+0000 7faeeb723640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee415ad60 0x7faee415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.206+0000 7faeeb723640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee40c7b40 con 0x7faee4154e40 2024-09-10T10:27:29.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.206+0000 7faeeb723640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7faee415b730 con 0x7faee40c0760 2024-09-10T10:27:29.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.206+0000 7faeeb723640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faee40c8ba0 con 0x7faee415ad60 2024-09-10T10:27:29.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.207+0000 7faee8cbe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7faee40c0760 0x7faee406cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.207+0000 7faee8cbe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7faee40c0760 0x7faee406cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37916/0 (socket says 172.21.15.120:37916) 2024-09-10T10:27:29.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.207+0000 7faee8cbe640 1 -- 172.21.15.120:0/3045307805 learned_addr learned my addr 172.21.15.120:0/3045307805 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:29.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.207+0000 7faee9cc0640 1 --2- 172.21.15.120:0/3045307805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee415ad60 0x7faee415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.207+0000 7faee8cbe640 1 -- 172.21.15.120:0/3045307805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee415ad60 msgr2=0x7faee415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:29.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.207+0000 7faee8cbe640 1 --2- 172.21.15.120:0/3045307805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee415ad60 0x7faee415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.207+0000 7faee8cbe640 1 -- 172.21.15.120:0/3045307805 >> v1:172.21.15.134:6789/0 conn(0x7faee4154e40 legacy=0x7faee406ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:29.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.207+0000 7faee8cbe640 1 -- 172.21.15.120:0/3045307805 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee40c8e80 con 0x7faee40c0760 2024-09-10T10:27:29.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.207+0000 7faee9cc0640 1 --2- 172.21.15.120:0/3045307805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee415ad60 0x7faee415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:29.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faee8cbe640 1 --2- 172.21.15.120:0/3045307805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 0x7faee406cf90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faed4009870 tx=0x7faed402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6480712b470b66c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:29.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faedbfff640 1 -- 172.21.15.120:0/3045307805 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4037070 con 0x7faee40c0760 2024-09-10T10:27:29.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faedbfff640 1 -- 172.21.15.120:0/3045307805 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed4032440 con 0x7faee40c0760 2024-09-10T10:27:29.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faedbfff640 1 -- 172.21.15.120:0/3045307805 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4032810 con 0x7faee40c0760 2024-09-10T10:27:29.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faeeb723640 1 -- 172.21.15.120:0/3045307805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 msgr2=0x7faee406cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:29.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faeeb723640 1 --2- 172.21.15.120:0/3045307805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 0x7faee406cf90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faed4009870 tx=0x7faed402a270 comp rx=0 tx=0).stop 2024-09-10T10:27:29.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faeeb723640 1 -- 172.21.15.120:0/3045307805 shutdown_connections 2024-09-10T10:27:29.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faeeb723640 1 --2- 172.21.15.120:0/3045307805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee415ad60 0x7faee415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faeeb723640 1 --2- 172.21.15.120:0/3045307805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 0x7faee406cf90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faeeb723640 1 -- 172.21.15.120:0/3045307805 >> 172.21.15.120:0/3045307805 conn(0x7faee4067950 msgr2=0x7faee4067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:29.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faeeb723640 1 -- 172.21.15.120:0/3045307805 shutdown_connections 2024-09-10T10:27:29.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.208+0000 7faeeb723640 1 -- 172.21.15.120:0/3045307805 wait complete. 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faeeb723640 1 Processor -- start 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faeeb723640 1 -- start start 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faeeb723640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 0x7faee415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faeeb723640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee4154e40 0x7faee40c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faeeb723640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee415ad60 0x7faee40bdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faeeb723640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faee406bd30 con 0x7faee40c0760 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faeeb723640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faee40c82e0 con 0x7faee415ad60 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faeeb723640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faee40c8620 con 0x7faee4154e40 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faee94bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 0x7faee415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faee94bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 0x7faee415dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37932/0 (socket says 172.21.15.120:37932) 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.210+0000 7faee94bf640 1 -- 172.21.15.120:0/1340075331 learned_addr learned my addr 172.21.15.120:0/1340075331 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:29.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faee9cc0640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee415ad60 0x7faee40bdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faee8cbe640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee4154e40 0x7faee40c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faee94bf640 1 -- 172.21.15.120:0/1340075331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee4154e40 msgr2=0x7faee40c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faee94bf640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee4154e40 0x7faee40c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faee94bf640 1 -- 172.21.15.120:0/1340075331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee415ad60 msgr2=0x7faee40bdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faee94bf640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee415ad60 0x7faee40bdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faee94bf640 1 -- 172.21.15.120:0/1340075331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee415f030 con 0x7faee40c0760 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faee94bf640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 0x7faee415dda0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faee000c430 tx=0x7faee000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faed9ffb640 1 -- 172.21.15.120:0/1340075331 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee00052d0 con 0x7faee40c0760 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faed9ffb640 1 -- 172.21.15.120:0/1340075331 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee0005490 con 0x7faee40c0760 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faed9ffb640 1 -- 172.21.15.120:0/1340075331 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0007260 con 0x7faee40c0760 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faeeb723640 1 -- 172.21.15.120:0/1340075331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee415ea70 con 0x7faee40c0760 2024-09-10T10:27:29.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.211+0000 7faeeb723640 1 -- 172.21.15.120:0/1340075331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee40bd3f0 con 0x7faee40c0760 2024-09-10T10:27:29.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.212+0000 7faed9ffb640 1 -- 172.21.15.120:0/1340075331 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faee0007400 con 0x7faee40c0760 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.213+0000 7faed9ffb640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7faec806be30 0x7faec806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.213+0000 7faed9ffb640 1 -- 172.21.15.120:0/1340075331 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7faee008a760 con 0x7faee40c0760 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.213+0000 7faee8cbe640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7faec806be30 0x7faec806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.213+0000 7faee8cbe640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7faec806be30 0x7faec806e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faed4034000 tx=0x7faed402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.214+0000 7faeee4ba740 1 Processor -- start 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.214+0000 7faeee4ba740 1 -- start start 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb42519e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4216f80 0x564bb4251f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4219b30 0x564bb4215a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faeee4ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564bb417a520 con 0x564bb4216f80 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faeee4ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564bb41223e0 con 0x564bb4219b30 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faeee4ba740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x564bb41764e0 con 0x564bb4219750 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee8cbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4216f80 0x564bb4251f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee8cbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4216f80 0x564bb4251f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37948/0 (socket says 172.21.15.120:37948) 2024-09-10T10:27:29.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee8cbe640 1 -- 172.21.15.120:0/871655288 learned_addr learned my addr 172.21.15.120:0/871655288 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee94bf640 1 --2- 172.21.15.120:0/871655288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb42519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee9cc0640 1 --2- 172.21.15.120:0/871655288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4219b30 0x564bb4215a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee8cbe640 1 -- 172.21.15.120:0/871655288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 msgr2=0x564bb42519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee8cbe640 1 --2- 172.21.15.120:0/871655288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb42519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee8cbe640 1 -- 172.21.15.120:0/871655288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4219b30 msgr2=0x564bb4215a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee8cbe640 1 --2- 172.21.15.120:0/871655288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4219b30 0x564bb4215a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee8cbe640 1 -- 172.21.15.120:0/871655288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb413d2f0 con 0x564bb4216f80 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faee8cbe640 1 --2- 172.21.15.120:0/871655288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4216f80 0x564bb4251f20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faed4000c00 tx=0x7faed402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46588650a2535386 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faec6ffd640 1 -- 172.21.15.120:0/871655288 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed404c070 con 0x564bb4216f80 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faec6ffd640 1 -- 172.21.15.120:0/871655288 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed4032d90 con 0x564bb4216f80 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.215+0000 7faec6ffd640 1 -- 172.21.15.120:0/871655288 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4037040 con 0x564bb4216f80 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.216+0000 7faeee4ba740 1 -- 172.21.15.120:0/871655288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4216f80 msgr2=0x564bb4251f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.216+0000 7faeee4ba740 1 --2- 172.21.15.120:0/871655288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4216f80 0x564bb4251f20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faed4000c00 tx=0x7faed402c000 comp rx=0 tx=0).stop 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.216+0000 7faeee4ba740 1 -- 172.21.15.120:0/871655288 shutdown_connections 2024-09-10T10:27:29.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.216+0000 7faeee4ba740 1 --2- 172.21.15.120:0/871655288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4219b30 0x564bb4215a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.216+0000 7faeee4ba740 1 --2- 172.21.15.120:0/871655288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4216f80 0x564bb4251f20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.216+0000 7faeee4ba740 1 --2- 172.21.15.120:0/871655288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb42519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.216+0000 7faeee4ba740 1 -- 172.21.15.120:0/871655288 >> 172.21.15.120:0/871655288 conn(0x564bb4217e70 msgr2=0x564bb4218270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:29.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.216+0000 7faeee4ba740 1 -- 172.21.15.120:0/871655288 shutdown_connections 2024-09-10T10:27:29.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.216+0000 7faeee4ba740 1 -- 172.21.15.120:0/871655288 wait complete. 2024-09-10T10:27:29.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.217+0000 7faeee4ba740 1 -- start start 2024-09-10T10:27:29.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.217+0000 7faeee4ba740 1 Processor -- start 2024-09-10T10:27:29.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4216f80 0x564bb424aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb423b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4239e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faeee4ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564bb4218fe0 con 0x564bb4219b30 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faeee4ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564bb4103450 con 0x564bb4216f80 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faeee4ba740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x564bb4251550 con 0x564bb4219750 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faeee4ba740 10 client.0 _ll_register_callbacks cb 0x564bb3e8cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee9cc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4239e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee9cc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4239e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37954/0 (socket says 172.21.15.120:37954) 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee9cc0640 1 -- 172.21.15.120:0/378367846 learned_addr learned my addr 172.21.15.120:0/378367846 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee8cbe640 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb423b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee94bf640 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4216f80 0x564bb424aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee9cc0640 1 -- 172.21.15.120:0/378367846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 msgr2=0x564bb423b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee9cc0640 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb423b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee9cc0640 1 -- 172.21.15.120:0/378367846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4216f80 msgr2=0x564bb424aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee9cc0640 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4216f80 0x564bb424aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee9cc0640 1 -- 172.21.15.120:0/378367846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb3e8d2d0 con 0x564bb4219b30 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faee9cc0640 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4239e50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faedc004660 tx=0x7faedc007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875e4ddbe31c6bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faebffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faebffff640 1 -- 172.21.15.120:0/378367846 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc01a070 con 0x564bb4219b30 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faeee4ba740 1 -- 172.21.15.120:0/378367846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faed4032d90 con 0x564bb4219b30 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faebffff640 1 -- 172.21.15.120:0/378367846 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7faedc016430 con 0x564bb4219b30 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faebffff640 1 -- 172.21.15.120:0/378367846 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7faedc025050 con 0x564bb4219b30 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faeee4ba740 10 client.4797 fetch_fsmap learned FSMap version 18 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.218+0000 7faeee4ba740 1 -- 172.21.15.120:0/378367846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x564bb3f576b0 con 0x564bb4219b30 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faebffff640 1 -- 172.21.15.120:0/378367846 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed4032d90 con 0x564bb4219b30 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faebffff640 1 -- 172.21.15.120:0/378367846 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 18) ==== 4800+0+0 (secure 0 0 0) 0x7faedc028020 con 0x564bb4219b30 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faeee4ba740 10 client.4797 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:27:29.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faeee4ba740 10 client.4797 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:27:29.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faeee4ba740 20 client.4797 populate_metadata read hostname 'smithi120' 2024-09-10T10:27:29.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faeee4ba740 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.120:6835/2787145148,v1:172.21.15.120:6836/2787145148] conn(0x564bb426cd40 0x564bb4277850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:29.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faeee4ba740 4 client.4797 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:27:29.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faeee4ba740 1 -- 172.21.15.120:0/378367846 --> [v2:172.21.15.120:6835/2787145148,v1:172.21.15.120:6836/2787145148] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bb4187c50 con 0x564bb426cd40 2024-09-10T10:27:29.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faee94bf640 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.120:6835/2787145148,v1:172.21.15.120:6836/2787145148] conn(0x564bb426cd40 0x564bb4277850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:29.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faebffff640 10 client.4797 ms_handle_connect on v2:172.21.15.120:6835/2787145148 2024-09-10T10:27:29.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:29.219+0000 7faee94bf640 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.120:6835/2787145148,v1:172.21.15.120:6836/2787145148] conn(0x564bb426cd40 0x564bb4277850 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9fa8a3f272a6eae9 server_cookie=f0b1f186a20b6d68 in_seq=0 out_seq=0 2024-09-10T10:27:31.766 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-09-10T10:27:31.765+0000 7f890ecec640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-10T10:27:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.955+0000 7faebffff640 1 -- 172.21.15.120:0/378367846 <== mds.4 v2:172.21.15.120:6835/2787145148 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bb4187c50 con 0x564bb426cd40 2024-09-10T10:27:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.955+0000 7faebffff640 10 client.4797 handle_command_reply: tid=0 2024-09-10T10:27:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.955+0000 7faeee4ba740 1 client.4797 shutdown 2024-09-10T10:27:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 1 -- 172.21.15.120:0/378367846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 msgr2=0x564bb4239e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4239e50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faedc004660 tx=0x7faedc007700 comp rx=0 tx=0).stop 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 1 -- 172.21.15.120:0/378367846 shutdown_connections 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.120:6835/2787145148,v1:172.21.15.120:6836/2787145148] conn(0x564bb426cd40 0x564bb4277850 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4239e50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb423b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 1 --2- 172.21.15.120:0/378367846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4216f80 0x564bb424aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 1 -- 172.21.15.120:0/378367846 >> 172.21.15.120:0/378367846 conn(0x564bb4217e70 msgr2=0x564bb4252590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faebffff640 0 client.4797 ms_handle_reset on v2:172.21.15.120:6835/2787145148 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 1 -- 172.21.15.120:0/378367846 shutdown_connections 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 1 -- 172.21.15.120:0/378367846 wait complete. 2024-09-10T10:27:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.956+0000 7faeee4ba740 20 client.4797 trim_cache size 0 max 16384 2024-09-10T10:27:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.976+0000 7faeee4ba740 1 Processor -- start 2024-09-10T10:27:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.976+0000 7faeee4ba740 1 -- start start 2024-09-10T10:27:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4216f80 0x564bb4213ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4219750 0x564bb4271d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4272240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faeee4ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faee41f50e0 con 0x564bb4219b30 2024-09-10T10:27:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faeee4ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564bb423efe0 con 0x564bb4219750 2024-09-10T10:27:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faeee4ba740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x564bb423c640 con 0x564bb4216f80 2024-09-10T10:27:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faee9cc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4272240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faee9cc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4272240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37960/0 (socket says 172.21.15.120:37960) 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faee9cc0640 1 -- 172.21.15.120:0/922269908 learned_addr learned my addr 172.21.15.120:0/922269908 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faee8cbe640 1 --2- 172.21.15.120:0/922269908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4219750 0x564bb4271d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faee94bf640 1 --2- 172.21.15.120:0/922269908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4216f80 0x564bb4213ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faee9cc0640 1 -- 172.21.15.120:0/922269908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4216f80 msgr2=0x564bb4213ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faee9cc0640 1 --2- 172.21.15.120:0/922269908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4216f80 0x564bb4213ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faee9cc0640 1 -- 172.21.15.120:0/922269908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4219750 msgr2=0x564bb4271d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faee9cc0640 1 --2- 172.21.15.120:0/922269908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4219750 0x564bb4271d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.977+0000 7faee9cc0640 1 -- 172.21.15.120:0/922269908 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb42562f0 con 0x564bb4219b30 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.978+0000 7faee9cc0640 1 --2- 172.21.15.120:0/922269908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4272240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faedc009ea0 tx=0x7faedc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2134d8cba6173f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.978+0000 7faec67fc640 1 -- 172.21.15.120:0/922269908 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc01a070 con 0x564bb4219b30 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.978+0000 7faec67fc640 1 -- 172.21.15.120:0/922269908 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faedc003e50 con 0x564bb4219b30 2024-09-10T10:27:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.978+0000 7faec67fc640 1 -- 172.21.15.120:0/922269908 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc01f440 con 0x564bb4219b30 2024-09-10T10:27:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 -- 172.21.15.120:0/922269908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 msgr2=0x564bb4272240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 --2- 172.21.15.120:0/922269908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4272240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faedc009ea0 tx=0x7faedc010000 comp rx=0 tx=0).stop 2024-09-10T10:27:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 -- 172.21.15.120:0/922269908 shutdown_connections 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 --2- 172.21.15.120:0/922269908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb4272240 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 --2- 172.21.15.120:0/922269908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4219750 0x564bb4271d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 --2- 172.21.15.120:0/922269908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4216f80 0x564bb4213ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 -- 172.21.15.120:0/922269908 >> 172.21.15.120:0/922269908 conn(0x564bb4217e70 msgr2=0x564bb4217380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 -- 172.21.15.120:0/922269908 shutdown_connections 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 -- 172.21.15.120:0/922269908 wait complete. 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 -- start start 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.979+0000 7faeee4ba740 1 Processor -- start 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.980+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4216f80 0x564bb4236ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.980+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb425ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.980+0000 7faeee4ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb425b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.980+0000 7faeee4ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564bb423fbe0 con 0x564bb4219b30 2024-09-10T10:27:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.980+0000 7faeee4ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564bb424c500 con 0x564bb4216f80 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.980+0000 7faeee4ba740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x564bb4240270 con 0x564bb4219750 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.980+0000 7faeee4ba740 10 client.0 _ll_register_callbacks cb 0x7faedc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee9cc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb425b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee9cc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb425b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37968/0 (socket says 172.21.15.120:37968) 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee9cc0640 1 -- 172.21.15.120:0/3165204098 learned_addr learned my addr 172.21.15.120:0/3165204098 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee94bf640 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4216f80 0x564bb4236ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee8cbe640 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb425ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee9cc0640 1 -- 172.21.15.120:0/3165204098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 msgr2=0x564bb425ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee9cc0640 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb425ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee9cc0640 1 -- 172.21.15.120:0/3165204098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4216f80 msgr2=0x564bb4236ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee9cc0640 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4216f80 0x564bb4236ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee9cc0640 1 -- 172.21.15.120:0/3165204098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb420d470 con 0x564bb4219b30 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee8cbe640 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb425ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faee9cc0640 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb425b260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faedc022040 tx=0x7faedc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01081b6d729ec48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faebffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faebffff640 1 -- 172.21.15.120:0/3165204098 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc037070 con 0x564bb4219b30 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faebffff640 1 -- 172.21.15.120:0/3165204098 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7faedc01fb60 con 0x564bb4219b30 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.981+0000 7faeee4ba740 1 -- 172.21.15.120:0/3165204098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faedc003e50 con 0x564bb4219b30 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faebffff640 1 -- 172.21.15.120:0/3165204098 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7faedc042050 con 0x564bb4219b30 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faeee4ba740 10 client.4803 fetch_fsmap learned FSMap version 18 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faeee4ba740 1 -- 172.21.15.120:0/3165204098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x564bb4272770 con 0x564bb4219b30 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faebffff640 1 -- 172.21.15.120:0/3165204098 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faedc003e50 con 0x564bb4219b30 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faebffff640 1 -- 172.21.15.120:0/3165204098 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 18) ==== 4800+0+0 (secure 0 0 0) 0x7faedc045020 con 0x564bb4219b30 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faeee4ba740 10 client.4803 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faeee4ba740 10 client.4803 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faeee4ba740 20 client.4803 populate_metadata read hostname 'smithi120' 2024-09-10T10:27:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faeee4ba740 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.120:6835/2787145148,v1:172.21.15.120:6836/2787145148] conn(0x564bb42e0f60 0x564bb42e3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faeee4ba740 4 client.4803 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faeee4ba740 1 -- 172.21.15.120:0/3165204098 --> [v2:172.21.15.120:6835/2787145148,v1:172.21.15.120:6836/2787145148] -- command(tid 0: {"prefix": "status"}) -- 0x564bb4181fa0 con 0x564bb42e0f60 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.982+0000 7faee94bf640 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.120:6835/2787145148,v1:172.21.15.120:6836/2787145148] conn(0x564bb42e0f60 0x564bb42e3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faebffff640 10 client.4803 ms_handle_connect on v2:172.21.15.120:6835/2787145148 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faee94bf640 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.120:6835/2787145148,v1:172.21.15.120:6836/2787145148] conn(0x564bb42e0f60 0x564bb42e3350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7233b6a3796a3326 server_cookie=f612a83d2d7fe8d3 in_seq=0 out_seq=0 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faebffff640 1 -- 172.21.15.120:0/3165204098 <== mds.4 v2:172.21.15.120:6835/2787145148 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x564bb4181fa0 con 0x564bb42e0f60 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faebffff640 10 client.4803 handle_command_reply: tid=0 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 client.4803 shutdown 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 -- 172.21.15.120:0/3165204098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 msgr2=0x564bb425b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb425b260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faedc022040 tx=0x7faedc004a40 comp rx=0 tx=0).stop 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 -- 172.21.15.120:0/3165204098 shutdown_connections 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.120:6835/2787145148,v1:172.21.15.120:6836/2787145148] conn(0x564bb42e0f60 0x564bb42e3350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564bb4219b30 0x564bb425b260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564bb4219750 0x564bb425ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 --2- 172.21.15.120:0/3165204098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564bb4216f80 0x564bb4236ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faebffff640 0 client.4803 ms_handle_reset on v2:172.21.15.120:6835/2787145148 2024-09-10T10:27:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 -- 172.21.15.120:0/3165204098 >> 172.21.15.120:0/3165204098 conn(0x564bb4217e70 msgr2=0x564bb4217380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 -- 172.21.15.120:0/3165204098 shutdown_connections 2024-09-10T10:27:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 1 -- 172.21.15.120:0/3165204098 wait complete. 2024-09-10T10:27:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.983+0000 7faeee4ba740 20 client.4803 trim_cache size 0 max 16384 2024-09-10T10:27:31.991 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:27:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 4, 2024-09-10T10:27:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 4281, 2024-09-10T10:27:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-10T10:27:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-10T10:27:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-10T10:27:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 11.047428713, 2024-09-10T10:27:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 18, 2024-09-10T10:27:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 28, 2024-09-10T10:27:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 28, 2024-09-10T10:27:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 56.235282445000003 2024-09-10T10:27:31.992 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 -- 172.21.15.120:0/1340075331 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7faec806be30 msgr2=0x7faec806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7faec806be30 0x7faec806e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faed4034000 tx=0x7faed402e000 comp rx=0 tx=0).stop 2024-09-10T10:27:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 -- 172.21.15.120:0/1340075331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 msgr2=0x7faee415dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 0x7faee415dda0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faee000c430 tx=0x7faee000c8f0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 -- 172.21.15.120:0/1340075331 shutdown_connections 2024-09-10T10:27:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7faec806be30 0x7faec806e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee415ad60 0x7faee40bdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee4154e40 0x7faee40c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 --2- 172.21.15.120:0/1340075331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faee40c0760 0x7faee415dda0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 -- 172.21.15.120:0/1340075331 >> 172.21.15.120:0/1340075331 conn(0x7faee4067950 msgr2=0x7faee4154000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 -- 172.21.15.120:0/1340075331 shutdown_connections 2024-09-10T10:27:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:31.984+0000 7faeeb723640 1 -- 172.21.15.120:0/1340075331 wait complete. 2024-09-10T10:27:31.999 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-09-10T10:27:32.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.103+0000 7fae07cf1640 1 Processor -- start 2024-09-10T10:27:32.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.103+0000 7fae07cf1640 1 -- start start 2024-09-10T10:27:32.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae07cf1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae00157ef0 0x7fae000c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae07cf1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fae00072e00 0x7fae000732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae07cf1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0015afa0 con 0x7fae001582d0 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae07cf1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fae00072970 con 0x7fae00072e00 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae07cf1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fae000737f0 con 0x7fae00157ef0 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae0628e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fae00072e00 0x7fae000732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae0628e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fae00072e00 0x7fae000732b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37978/0 (socket says 172.21.15.120:37978) 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae0628e640 1 -- 172.21.15.120:0/4235477057 learned_addr learned my addr 172.21.15.120:0/4235477057 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae05a8d640 1 --2- 172.21.15.120:0/4235477057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae00157ef0 0x7fae000c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae0628e640 1 -- 172.21.15.120:0/4235477057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae00157ef0 msgr2=0x7fae000c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae0628e640 1 --2- 172.21.15.120:0/4235477057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae00157ef0 0x7fae000c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae0628e640 1 -- 172.21.15.120:0/4235477057 >> v1:172.21.15.134:6789/0 conn(0x7fae001582d0 legacy=0x7fae000c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.104+0000 7fae0628e640 1 -- 172.21.15.120:0/4235477057 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0015b280 con 0x7fae00072e00 2024-09-10T10:27:32.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.105+0000 7fae0628e640 1 --2- 172.21.15.120:0/4235477057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00072e00 0x7fae000732b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009e50 tx=0x7fadf802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169ebfa5c5f5b0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.105+0000 7fae04a8b640 1 -- 172.21.15.120:0/4235477057 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8036070 con 0x7fae00072e00 2024-09-10T10:27:32.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.105+0000 7fae04a8b640 1 -- 172.21.15.120:0/4235477057 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf8031600 con 0x7fae00072e00 2024-09-10T10:27:32.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.105+0000 7fae04a8b640 1 -- 172.21.15.120:0/4235477057 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf80319d0 con 0x7fae00072e00 2024-09-10T10:27:32.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.106+0000 7fae07cf1640 1 -- 172.21.15.120:0/4235477057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00072e00 msgr2=0x7fae000732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.106+0000 7fae07cf1640 1 --2- 172.21.15.120:0/4235477057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00072e00 0x7fae000732b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009e50 tx=0x7fadf802a7a0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.106+0000 7fae07cf1640 1 -- 172.21.15.120:0/4235477057 shutdown_connections 2024-09-10T10:27:32.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.106+0000 7fae07cf1640 1 --2- 172.21.15.120:0/4235477057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00072e00 0x7fae000732b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.106+0000 7fae07cf1640 1 --2- 172.21.15.120:0/4235477057 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae00157ef0 0x7fae000c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.106+0000 7fae07cf1640 1 -- 172.21.15.120:0/4235477057 >> 172.21.15.120:0/4235477057 conn(0x7fae000703e0 msgr2=0x7fae000707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.106+0000 7fae07cf1640 1 -- 172.21.15.120:0/4235477057 shutdown_connections 2024-09-10T10:27:32.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.106+0000 7fae07cf1640 1 -- 172.21.15.120:0/4235477057 wait complete. 2024-09-10T10:27:32.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.107+0000 7fae07cf1640 1 Processor -- start 2024-09-10T10:27:32.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.107+0000 7fae07cf1640 1 -- start start 2024-09-10T10:27:32.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae07cf1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae00072e00 0x7fae00153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae07cf1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00157ef0 0x7fae001f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae07cf1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae001582d0 0x7fae00150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae07cf1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae000c7540 con 0x7fae00157ef0 2024-09-10T10:27:32.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae07cf1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae000caa80 con 0x7fae001582d0 2024-09-10T10:27:32.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae07cf1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fae0015b840 con 0x7fae00072e00 2024-09-10T10:27:32.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae0528c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00157ef0 0x7fae001f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae0528c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00157ef0 0x7fae001f44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37994/0 (socket says 172.21.15.120:37994) 2024-09-10T10:27:32.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae0528c640 1 -- 172.21.15.120:0/605020937 learned_addr learned my addr 172.21.15.120:0/605020937 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae05a8d640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae00072e00 0x7fae00153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae0628e640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae001582d0 0x7fae00150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae0528c640 1 -- 172.21.15.120:0/605020937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae00072e00 msgr2=0x7fae00153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae0528c640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae00072e00 0x7fae00153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae0528c640 1 -- 172.21.15.120:0/605020937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae001582d0 msgr2=0x7fae00150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae0528c640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae001582d0 0x7fae00150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae0528c640 1 -- 172.21.15.120:0/605020937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae001e1bd0 con 0x7fae00157ef0 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae0528c640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00157ef0 0x7fae001f44c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c440 tx=0x7fadf0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fadf67fc640 1 -- 172.21.15.120:0/605020937 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf00052d0 con 0x7fae00157ef0 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fadf67fc640 1 -- 172.21.15.120:0/605020937 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf0005470 con 0x7fae00157ef0 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fadf67fc640 1 -- 172.21.15.120:0/605020937 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0007260 con 0x7fae00157ef0 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.108+0000 7fae07cf1640 1 -- 172.21.15.120:0/605020937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae001f50f0 con 0x7fae00157ef0 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.109+0000 7fae07cf1640 1 -- 172.21.15.120:0/605020937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae001e3020 con 0x7fae00157ef0 2024-09-10T10:27:32.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.110+0000 7fadf67fc640 1 -- 172.21.15.120:0/605020937 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadf0007400 con 0x7fae00157ef0 2024-09-10T10:27:32.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.110+0000 7fadf67fc640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fade406be60 0x7fade406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.110+0000 7fadf67fc640 1 -- 172.21.15.120:0/605020937 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fadf008a780 con 0x7fae00157ef0 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.110+0000 7fae05a8d640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fade406be60 0x7fade406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.111+0000 7fae05a8d640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fade406be60 0x7fade406e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fadfc006fd0 tx=0x7fadfc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.111+0000 7fae0aa8d740 1 Processor -- start 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae0aa8d740 1 -- start start 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b1f80 0x56487b2ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4b30 0x56487b2b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56487b215520 con 0x56487b2b4750 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56487b1bd3e0 con 0x56487b2b4b30 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56487b2114e0 con 0x56487b2b1f80 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae05a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae05a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ec9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38008/0 (socket says 172.21.15.120:38008) 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae05a8d640 1 -- 172.21.15.120:0/4018947548 learned_addr learned my addr 172.21.15.120:0/4018947548 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae0528c640 1 --2- 172.21.15.120:0/4018947548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b1f80 0x56487b2ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.112+0000 7fae0628e640 1 --2- 172.21.15.120:0/4018947548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4b30 0x56487b2b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.113+0000 7fae05a8d640 1 -- 172.21.15.120:0/4018947548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b1f80 msgr2=0x56487b2ecf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.113+0000 7fae05a8d640 1 --2- 172.21.15.120:0/4018947548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b1f80 0x56487b2ecf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.113+0000 7fae05a8d640 1 -- 172.21.15.120:0/4018947548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4b30 msgr2=0x56487b2b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.113+0000 7fae05a8d640 1 --2- 172.21.15.120:0/4018947548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4b30 0x56487b2b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.113+0000 7fae05a8d640 1 -- 172.21.15.120:0/4018947548 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56487b1d82f0 con 0x56487b2b4750 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.113+0000 7fae05a8d640 1 --2- 172.21.15.120:0/4018947548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ec9e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fadfc004240 tx=0x7fadfc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e39021a88985ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.113+0000 7fade37fe640 1 -- 172.21.15.120:0/4018947548 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc027070 con 0x56487b2b4750 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.113+0000 7fade37fe640 1 -- 172.21.15.120:0/4018947548 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc003820 con 0x56487b2b4750 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.113+0000 7fade37fe640 1 -- 172.21.15.120:0/4018947548 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc003bd0 con 0x56487b2b4750 2024-09-10T10:27:32.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 -- 172.21.15.120:0/4018947548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 msgr2=0x56487b2ec9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/4018947548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ec9e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fadfc004240 tx=0x7fadfc013000 comp rx=0 tx=0).stop 2024-09-10T10:27:32.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 -- 172.21.15.120:0/4018947548 shutdown_connections 2024-09-10T10:27:32.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/4018947548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4b30 0x56487b2b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/4018947548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b1f80 0x56487b2ecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/4018947548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ec9e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 -- 172.21.15.120:0/4018947548 >> 172.21.15.120:0/4018947548 conn(0x56487b2b2e70 msgr2=0x56487b2b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 -- 172.21.15.120:0/4018947548 shutdown_connections 2024-09-10T10:27:32.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 -- 172.21.15.120:0/4018947548 wait complete. 2024-09-10T10:27:32.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 -- start start 2024-09-10T10:27:32.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.114+0000 7fae0aa8d740 1 Processor -- start 2024-09-10T10:27:32.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2dd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4750 0x56487b2dc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b4b30 0x56487b2d4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56487b2b3fe0 con 0x56487b2b1f80 2024-09-10T10:27:32.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56487b19e450 con 0x56487b2b4750 2024-09-10T10:27:32.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56487b2ec550 con 0x56487b2b4b30 2024-09-10T10:27:32.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae0aa8d740 10 client.0 _ll_register_callbacks cb 0x56487af27ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:27:32.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae05a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2dd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae05a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2dd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38022/0 (socket says 172.21.15.120:38022) 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae05a8d640 1 -- 172.21.15.120:0/3418647400 learned_addr learned my addr 172.21.15.120:0/3418647400 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.115+0000 7fae0528c640 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4750 0x56487b2dc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae0628e640 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b4b30 0x56487b2d4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae05a8d640 1 -- 172.21.15.120:0/3418647400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b4b30 msgr2=0x56487b2d4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae05a8d640 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b4b30 0x56487b2d4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae05a8d640 1 -- 172.21.15.120:0/3418647400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4750 msgr2=0x56487b2dc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae05a8d640 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4750 0x56487b2dc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae05a8d640 1 -- 172.21.15.120:0/3418647400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56487af282d0 con 0x56487b2b1f80 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae05a8d640 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2dd470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadfc025040 tx=0x7fadfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc797d1ba6264c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fade0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fade0ff9640 1 -- 172.21.15.120:0/3418647400 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc030060 con 0x56487b2b1f80 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fade0ff9640 1 -- 172.21.15.120:0/3418647400 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fadfc016750 con 0x56487b2b1f80 2024-09-10T10:27:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3418647400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadfc003820 con 0x56487b2b1f80 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fade0ff9640 1 -- 172.21.15.120:0/3418647400 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fadfc027070 con 0x56487b2b1f80 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae0aa8d740 10 client.4815 fetch_fsmap learned FSMap version 18 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3418647400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x56487aff26b0 con 0x56487b2b1f80 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fade0ff9640 1 -- 172.21.15.120:0/3418647400 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc003820 con 0x56487b2b1f80 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fade0ff9640 1 -- 172.21.15.120:0/3418647400 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 18) ==== 4800+0+0 (secure 0 0 0) 0x7fadfc039080 con 0x56487b2b1f80 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae0aa8d740 10 client.4815 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae0aa8d740 10 client.4815 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae0aa8d740 20 client.4815 populate_metadata read hostname 'smithi120' 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.120:6838/1710396335,v1:172.21.15.120:6839/1710396335] conn(0x56487b307c20 0x56487b312850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae0aa8d740 4 client.4815 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.116+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3418647400 --> [v2:172.21.15.120:6838/1710396335,v1:172.21.15.120:6839/1710396335] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56487b222c50 con 0x56487b307c20 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.117+0000 7fae0528c640 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.120:6838/1710396335,v1:172.21.15.120:6839/1710396335] conn(0x56487b307c20 0x56487b312850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.117+0000 7fae0528c640 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.120:6838/1710396335,v1:172.21.15.120:6839/1710396335] conn(0x56487b307c20 0x56487b312850 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b28ece3ab0d250c1 server_cookie=44d3050de096ab98 in_seq=0 out_seq=0 2024-09-10T10:27:32.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.117+0000 7fade0ff9640 10 client.4815 ms_handle_connect on v2:172.21.15.120:6838/1710396335 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.117+0000 7fade0ff9640 1 -- 172.21.15.120:0/3418647400 <== mds.3 v2:172.21.15.120:6838/1710396335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56487b222c50 con 0x56487b307c20 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.117+0000 7fade0ff9640 10 client.4815 handle_command_reply: tid=0 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.117+0000 7fae0aa8d740 1 client.4815 shutdown 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3418647400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 msgr2=0x56487b2dd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2dd470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadfc025040 tx=0x7fadfc004730 comp rx=0 tx=0).stop 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3418647400 shutdown_connections 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.120:6838/1710396335,v1:172.21.15.120:6839/1710396335] conn(0x56487b307c20 0x56487b312850 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b4b30 0x56487b2d4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fade0ff9640 0 client.4815 ms_handle_reset on v2:172.21.15.120:6838/1710396335 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4750 0x56487b2dc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/3418647400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2dd470 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3418647400 >> 172.21.15.120:0/3418647400 conn(0x56487b2b2e70 msgr2=0x56487b2ed4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3418647400 shutdown_connections 2024-09-10T10:27:32.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3418647400 wait complete. 2024-09-10T10:27:32.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.118+0000 7fae0aa8d740 20 client.4815 trim_cache size 0 max 16384 2024-09-10T10:27:32.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0aa8d740 1 Processor -- start 2024-09-10T10:27:32.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0aa8d740 1 -- start start 2024-09-10T10:27:32.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b1f80 0x56487b2dc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4b30 0x56487b2de4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae001f15f0 con 0x56487b2b4750 2024-09-10T10:27:32.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56487b304800 con 0x56487b2b4b30 2024-09-10T10:27:32.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56487b2ddbc0 con 0x56487b2b1f80 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0528c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0528c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ddf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38024/0 (socket says 172.21.15.120:38024) 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.136+0000 7fae0528c640 1 -- 172.21.15.120:0/3895820534 learned_addr learned my addr 172.21.15.120:0/3895820534 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.137+0000 7fae0528c640 1 -- 172.21.15.120:0/3895820534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b1f80 msgr2=0x56487b2dc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.137+0000 7fae0628e640 1 --2- 172.21.15.120:0/3895820534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4b30 0x56487b2de4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.137+0000 7fae0528c640 1 --2- 172.21.15.120:0/3895820534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b1f80 0x56487b2dc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.137+0000 7fae0528c640 1 -- 172.21.15.120:0/3895820534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4b30 msgr2=0x56487b2de4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.137+0000 7fae0528c640 1 --2- 172.21.15.120:0/3895820534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4b30 0x56487b2de4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.137+0000 7fae0528c640 1 -- 172.21.15.120:0/3895820534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56487b2b1db0 con 0x56487b2b4750 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.137+0000 7fae0528c640 1 --2- 172.21.15.120:0/3895820534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ddf70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c410 tx=0x7fadf0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a510c87e4d7524ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.137+0000 7fade2ffd640 1 -- 172.21.15.120:0/3895820534 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0054cc0 con 0x56487b2b4750 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.137+0000 7fade2ffd640 1 -- 172.21.15.120:0/3895820534 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf0057aa0 con 0x56487b2b4750 2024-09-10T10:27:32.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.137+0000 7fade2ffd640 1 -- 172.21.15.120:0/3895820534 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0054e60 con 0x56487b2b4750 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3895820534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 msgr2=0x56487b2ddf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/3895820534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ddf70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c410 tx=0x7fadf0000bc0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3895820534 shutdown_connections 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/3895820534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4b30 0x56487b2de4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/3895820534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b4750 0x56487b2ddf70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/3895820534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b1f80 0x56487b2dc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3895820534 >> 172.21.15.120:0/3895820534 conn(0x56487b2b2e70 msgr2=0x56487b2b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3895820534 shutdown_connections 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 -- 172.21.15.120:0/3895820534 wait complete. 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 -- start start 2024-09-10T10:27:32.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.138+0000 7fae0aa8d740 1 Processor -- start 2024-09-10T10:27:32.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4750 0x56487b2b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae0aa8d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b4b30 0x56487b36ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56487b2dea00 con 0x56487b2b1f80 2024-09-10T10:27:32.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56487b2e4b40 con 0x56487b2b4750 2024-09-10T10:27:32.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae0aa8d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56487b2d5510 con 0x56487b2b4b30 2024-09-10T10:27:32.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae0aa8d740 10 client.0 _ll_register_callbacks cb 0x7fadfc025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:27:32.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae05a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae05a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2d9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38030/0 (socket says 172.21.15.120:38030) 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae05a8d640 1 -- 172.21.15.120:0/1636927628 learned_addr learned my addr 172.21.15.120:0/1636927628 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae0628e640 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b4b30 0x56487b36ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae05a8d640 1 -- 172.21.15.120:0/1636927628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b4b30 msgr2=0x56487b36ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae05a8d640 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b4b30 0x56487b36ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.139+0000 7fae0528c640 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4750 0x56487b2b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae05a8d640 1 -- 172.21.15.120:0/1636927628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4750 msgr2=0x56487b2b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae05a8d640 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4750 0x56487b2b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae05a8d640 1 -- 172.21.15.120:0/1636927628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56487b30d730 con 0x56487b2b1f80 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae05a8d640 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2d9810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fadfc016f30 tx=0x7fadfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7bac321adfb712 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fade0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fade0ff9640 1 -- 172.21.15.120:0/1636927628 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc030070 con 0x56487b2b1f80 2024-09-10T10:27:32.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fade0ff9640 1 -- 172.21.15.120:0/1636927628 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fadfc031460 con 0x56487b2b1f80 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae0aa8d740 1 -- 172.21.15.120:0/1636927628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf0057aa0 con 0x56487b2b1f80 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fade0ff9640 1 -- 172.21.15.120:0/1636927628 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fadfc044050 con 0x56487b2b1f80 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae0aa8d740 10 client.4821 fetch_fsmap learned FSMap version 18 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae0aa8d740 1 -- 172.21.15.120:0/1636927628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x56487b2b4f70 con 0x56487b2b1f80 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fade0ff9640 1 -- 172.21.15.120:0/1636927628 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf0057aa0 con 0x56487b2b1f80 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fade0ff9640 1 -- 172.21.15.120:0/1636927628 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 18) ==== 4800+0+0 (secure 0 0 0) 0x7fadfc047020 con 0x56487b2b1f80 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae0aa8d740 10 client.4821 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae0aa8d740 10 client.4821 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae0aa8d740 20 client.4821 populate_metadata read hostname 'smithi120' 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.120:6838/1710396335,v1:172.21.15.120:6839/1710396335] conn(0x56487b375aa0 0x56487b37e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae0aa8d740 4 client.4821 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae0aa8d740 1 -- 172.21.15.120:0/1636927628 --> [v2:172.21.15.120:6838/1710396335,v1:172.21.15.120:6839/1710396335] -- command(tid 0: {"prefix": "status"}) -- 0x56487b21cfa0 con 0x56487b375aa0 2024-09-10T10:27:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.140+0000 7fae0528c640 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.120:6838/1710396335,v1:172.21.15.120:6839/1710396335] conn(0x56487b375aa0 0x56487b37e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0528c640 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.120:6838/1710396335,v1:172.21.15.120:6839/1710396335] conn(0x56487b375aa0 0x56487b37e540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=398c17935291d51 server_cookie=405b3a8ec957214d in_seq=0 out_seq=0 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fade0ff9640 10 client.4821 ms_handle_connect on v2:172.21.15.120:6838/1710396335 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fade0ff9640 1 -- 172.21.15.120:0/1636927628 <== mds.3 v2:172.21.15.120:6838/1710396335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56487b21cfa0 con 0x56487b375aa0 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fade0ff9640 10 client.4821 handle_command_reply: tid=0 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 client.4821 shutdown 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 -- 172.21.15.120:0/1636927628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 msgr2=0x56487b2d9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2d9810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fadfc016f30 tx=0x7fadfc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 -- 172.21.15.120:0/1636927628 shutdown_connections 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.120:6838/1710396335,v1:172.21.15.120:6839/1710396335] conn(0x56487b375aa0 0x56487b37e540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56487b2b4b30 0x56487b36ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56487b2b4750 0x56487b2b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 --2- 172.21.15.120:0/1636927628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487b2b1f80 0x56487b2d9810 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fade0ff9640 0 client.4821 ms_handle_reset on v2:172.21.15.120:6838/1710396335 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 -- 172.21.15.120:0/1636927628 >> 172.21.15.120:0/1636927628 conn(0x56487b2b2e70 msgr2=0x56487b2ddf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 -- 172.21.15.120:0/1636927628 shutdown_connections 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 1 -- 172.21.15.120:0/1636927628 wait complete. 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.141+0000 7fae0aa8d740 20 client.4821 trim_cache size 0 max 16384 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 3, 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 4284, 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-10T10:27:32.150 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-10T10:27:32.151 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-10T10:27:32.151 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 17.225292472, 2024-09-10T10:27:32.151 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 18, 2024-09-10T10:27:32.151 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 28, 2024-09-10T10:27:32.151 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 28, 2024-09-10T10:27:32.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 56.388890076000003 2024-09-10T10:27:32.151 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:32.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 -- 172.21.15.120:0/605020937 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fade406be60 msgr2=0x7fade406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fade406be60 0x7fade406e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fadfc006fd0 tx=0x7fadfc008040 comp rx=0 tx=0).stop 2024-09-10T10:27:32.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 -- 172.21.15.120:0/605020937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00157ef0 msgr2=0x7fae001f44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00157ef0 0x7fae001f44c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c440 tx=0x7fadf0001e70 comp rx=0 tx=0).stop 2024-09-10T10:27:32.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 -- 172.21.15.120:0/605020937 shutdown_connections 2024-09-10T10:27:32.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fade406be60 0x7fade406e320 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae001582d0 0x7fae00150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae00157ef0 0x7fae001f44c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 --2- 172.21.15.120:0/605020937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae00072e00 0x7fae00153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 -- 172.21.15.120:0/605020937 >> 172.21.15.120:0/605020937 conn(0x7fae000703e0 msgr2=0x7fae00155ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 -- 172.21.15.120:0/605020937 shutdown_connections 2024-09-10T10:27:32.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.142+0000 7fae07cf1640 1 -- 172.21.15.120:0/605020937 wait complete. 2024-09-10T10:27:32.155 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:27:32.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.252+0000 7f91179c6640 1 Processor -- start 2024-09-10T10:27:32.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.252+0000 7f91179c6640 1 -- start start 2024-09-10T10:27:32.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f91179c6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f91100c6410 0x7f9110158040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f91179c6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f91100c5280 0x7f91100c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f91179c6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9110151e70 con 0x7f9110151a90 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f91179c6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f91100c8130 con 0x7f91100c5280 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f91179c6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f91100c8410 con 0x7f91100c6410 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f9115f63640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f91100c5280 0x7f91100c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f9115f63640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f91100c5280 0x7f91100c56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38042/0 (socket says 172.21.15.120:38042) 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f9115f63640 1 -- 172.21.15.120:0/3654235837 learned_addr learned my addr 172.21.15.120:0/3654235837 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f9114f61640 1 --2- 172.21.15.120:0/3654235837 >> v2:172.21.15.134:3300/0 conn(0x7f91100c6410 0x7f9110158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f9115f63640 1 -- 172.21.15.120:0/3654235837 >> v2:172.21.15.134:3300/0 conn(0x7f91100c6410 msgr2=0x7f9110158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f9115f63640 1 --2- 172.21.15.120:0/3654235837 >> v2:172.21.15.134:3300/0 conn(0x7f91100c6410 0x7f9110158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f9115f63640 1 -- 172.21.15.120:0/3654235837 >> v1:172.21.15.134:6789/0 conn(0x7f9110151a90 legacy=0x7f91100c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.253+0000 7f9115f63640 1 -- 172.21.15.120:0/3654235837 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91100c86f0 con 0x7f91100c5280 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.254+0000 7f9115f63640 1 --2- 172.21.15.120:0/3654235837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c5280 0x7f91100c56f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f911006ccb0 tx=0x7f910802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dadf695399a11e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.254+0000 7f910ffff640 1 -- 172.21.15.120:0/3654235837 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108037070 con 0x7f91100c5280 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.254+0000 7f910ffff640 1 -- 172.21.15.120:0/3654235837 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9108032440 con 0x7f91100c5280 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.254+0000 7f910ffff640 1 -- 172.21.15.120:0/3654235837 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108032810 con 0x7f91100c5280 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.255+0000 7f91179c6640 1 -- 172.21.15.120:0/3654235837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c5280 msgr2=0x7f91100c56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.255+0000 7f91179c6640 1 --2- 172.21.15.120:0/3654235837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c5280 0x7f91100c56f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f911006ccb0 tx=0x7f910802a270 comp rx=0 tx=0).stop 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.255+0000 7f91179c6640 1 -- 172.21.15.120:0/3654235837 shutdown_connections 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.255+0000 7f91179c6640 1 --2- 172.21.15.120:0/3654235837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c5280 0x7f91100c56f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.255+0000 7f91179c6640 1 --2- 172.21.15.120:0/3654235837 >> v2:172.21.15.134:3300/0 conn(0x7f91100c6410 0x7f9110158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.255+0000 7f91179c6640 1 -- 172.21.15.120:0/3654235837 >> 172.21.15.120:0/3654235837 conn(0x7f911006f3c0 msgr2=0x7f911006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.255+0000 7f91179c6640 1 -- 172.21.15.120:0/3654235837 shutdown_connections 2024-09-10T10:27:32.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.255+0000 7f91179c6640 1 -- 172.21.15.120:0/3654235837 wait complete. 2024-09-10T10:27:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.256+0000 7f91179c6640 1 Processor -- start 2024-09-10T10:27:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.256+0000 7f91179c6640 1 -- start start 2024-09-10T10:27:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f91179c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91100c5280 0x7f91101e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f91179c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c6410 0x7f9110155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f91179c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9110151a90 0x7f9110152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f91179c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9110157b20 con 0x7f91100c6410 2024-09-10T10:27:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f91179c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9110157840 con 0x7f9110151a90 2024-09-10T10:27:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f91179c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9110158580 con 0x7f91100c5280 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9114f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c6410 0x7f9110155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9114f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c6410 0x7f9110155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38054/0 (socket says 172.21.15.120:38054) 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9114f61640 1 -- 172.21.15.120:0/3085866792 learned_addr learned my addr 172.21.15.120:0/3085866792 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9115f63640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9110151a90 0x7f9110152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9115762640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91100c5280 0x7f91101e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9114f61640 1 -- 172.21.15.120:0/3085866792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91100c5280 msgr2=0x7f91101e3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9114f61640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91100c5280 0x7f91101e3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9114f61640 1 -- 172.21.15.120:0/3085866792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9110151a90 msgr2=0x7f9110152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9114f61640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9110151a90 0x7f9110152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9114f61640 1 -- 172.21.15.120:0/3085866792 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91101e58d0 con 0x7f91100c6410 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9115762640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91100c5280 0x7f91101e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f9114f61640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c6410 0x7f9110155600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f910400d800 tx=0x7f910400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f910dffb640 1 -- 172.21.15.120:0/3085866792 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104004020 con 0x7f91100c6410 2024-09-10T10:27:32.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f910dffb640 1 -- 172.21.15.120:0/3085866792 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91040041c0 con 0x7f91100c6410 2024-09-10T10:27:32.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f910dffb640 1 -- 172.21.15.120:0/3085866792 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104004390 con 0x7f91100c6410 2024-09-10T10:27:32.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.257+0000 7f91179c6640 1 -- 172.21.15.120:0/3085866792 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91101e4d00 con 0x7f91100c6410 2024-09-10T10:27:32.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.258+0000 7f91179c6640 1 -- 172.21.15.120:0/3085866792 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91101e5560 con 0x7f91100c6410 2024-09-10T10:27:32.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.259+0000 7f910dffb640 1 -- 172.21.15.120:0/3085866792 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f910401b020 con 0x7f91100c6410 2024-09-10T10:27:32.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.260+0000 7f910dffb640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f90fc06be30 0x7f90fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.260+0000 7f9115762640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f90fc06be30 0x7f90fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.260+0000 7f910dffb640 1 -- 172.21.15.120:0/3085866792 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f910408afe0 con 0x7f91100c6410 2024-09-10T10:27:32.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.260+0000 7f9115762640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f90fc06be30 0x7f90fc06e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9100001d60 tx=0x7f9100009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f911a837740 1 Processor -- start 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f911a837740 1 -- start start 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f911a837740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9750 0x5565ebe119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f911a837740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd6f80 0x5565ebe11f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f911a837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd9b30 0x5565ebdd5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f911a837740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565ebd3a520 con 0x5565ebdd6f80 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f911a837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565ebce23e0 con 0x5565ebdd9b30 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f911a837740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5565ebd364e0 con 0x5565ebdd9750 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f9114f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd6f80 0x5565ebe11f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f9114f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd6f80 0x5565ebe11f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38064/0 (socket says 172.21.15.120:38064) 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.261+0000 7f9114f61640 1 -- 172.21.15.120:0/2308804280 learned_addr learned my addr 172.21.15.120:0/2308804280 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f9115762640 1 --2- 172.21.15.120:0/2308804280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9750 0x5565ebe119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f9114f61640 1 -- 172.21.15.120:0/2308804280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9750 msgr2=0x5565ebe119e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f9115f63640 1 --2- 172.21.15.120:0/2308804280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd9b30 0x5565ebdd5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f9114f61640 1 --2- 172.21.15.120:0/2308804280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9750 0x5565ebe119e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f9114f61640 1 -- 172.21.15.120:0/2308804280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd9b30 msgr2=0x5565ebdd5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f9114f61640 1 --2- 172.21.15.120:0/2308804280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd9b30 0x5565ebdd5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f9114f61640 1 -- 172.21.15.120:0/2308804280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565ebcfd2f0 con 0x5565ebdd6f80 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f9114f61640 1 --2- 172.21.15.120:0/2308804280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd6f80 0x5565ebe11f20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9104089040 tx=0x7f9104059550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e1137d7e0a8e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f90f6ffd640 1 -- 172.21.15.120:0/2308804280 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104087070 con 0x5565ebdd6f80 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f90f6ffd640 1 -- 172.21.15.120:0/2308804280 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910405b040 con 0x5565ebdd6f80 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.262+0000 7f90f6ffd640 1 -- 172.21.15.120:0/2308804280 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104061a30 con 0x5565ebdd6f80 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 -- 172.21.15.120:0/2308804280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd6f80 msgr2=0x5565ebe11f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 --2- 172.21.15.120:0/2308804280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd6f80 0x5565ebe11f20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9104089040 tx=0x7f9104059550 comp rx=0 tx=0).stop 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 -- 172.21.15.120:0/2308804280 shutdown_connections 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 --2- 172.21.15.120:0/2308804280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd9b30 0x5565ebdd5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 --2- 172.21.15.120:0/2308804280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd6f80 0x5565ebe11f20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 --2- 172.21.15.120:0/2308804280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9750 0x5565ebe119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 -- 172.21.15.120:0/2308804280 >> 172.21.15.120:0/2308804280 conn(0x5565ebdd7e70 msgr2=0x5565ebdd8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 -- 172.21.15.120:0/2308804280 shutdown_connections 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 -- 172.21.15.120:0/2308804280 wait complete. 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 -- start start 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.263+0000 7f911a837740 1 Processor -- start 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f911a837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 0x5565ebe0b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f911a837740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe09570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f911a837740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 0x5565ebe06150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f911a837740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565ebdd8fe0 con 0x5565ebdd9750 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f911a837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565ebcc3450 con 0x5565ebdd6f80 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f911a837740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5565ebe11550 con 0x5565ebdd9b30 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f911a837740 10 client.0 _ll_register_callbacks cb 0x5565eba4cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f9114f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe09570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f9114f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe09570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38068/0 (socket says 172.21.15.120:38068) 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f9114f61640 1 -- 172.21.15.120:0/1305360279 learned_addr learned my addr 172.21.15.120:0/1305360279 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f9115762640 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 0x5565ebe0b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f9115f63640 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 0x5565ebe06150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f9114f61640 1 -- 172.21.15.120:0/1305360279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 msgr2=0x5565ebe06150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f9114f61640 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 0x5565ebe06150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f9114f61640 1 -- 172.21.15.120:0/1305360279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 msgr2=0x5565ebe0b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f9114f61640 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 0x5565ebe0b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.264+0000 7f9114f61640 1 -- 172.21.15.120:0/1305360279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565eba4d2d0 con 0x5565ebdd9750 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f9114f61640 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe09570 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9104061500 tx=0x7f910405c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a669160f60acc538 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f90ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f90ebfff640 1 -- 172.21.15.120:0/1305360279 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104077060 con 0x5565ebdd9750 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f90ebfff640 1 -- 172.21.15.120:0/1305360279 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f91040554c0 con 0x5565ebdd9750 2024-09-10T10:27:32.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f911a837740 1 -- 172.21.15.120:0/1305360279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910405b040 con 0x5565ebdd9750 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f90ebfff640 1 -- 172.21.15.120:0/1305360279 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f910407e050 con 0x5565ebdd9750 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f911a837740 10 client.4833 fetch_fsmap learned FSMap version 18 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f911a837740 1 -- 172.21.15.120:0/1305360279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5565ebb176b0 con 0x5565ebdd9750 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f90ebfff640 1 -- 172.21.15.120:0/1305360279 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910405b040 con 0x5565ebdd9750 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f90ebfff640 1 -- 172.21.15.120:0/1305360279 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 18) ==== 4800+0+0 (secure 0 0 0) 0x7f9104081020 con 0x5565ebdd9750 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f911a837740 10 client.4833 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f911a837740 10 client.4833 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f911a837740 20 client.4833 populate_metadata read hostname 'smithi120' 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f911a837740 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.120:6834/3208935627,v1:172.21.15.120:6837/3208935627] conn(0x5565ebe2c700 0x5565ebe375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f911a837740 4 client.4833 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f911a837740 1 -- 172.21.15.120:0/1305360279 --> [v2:172.21.15.120:6834/3208935627,v1:172.21.15.120:6837/3208935627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565ebd47c50 con 0x5565ebe2c700 2024-09-10T10:27:32.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.265+0000 7f9115762640 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.120:6834/3208935627,v1:172.21.15.120:6837/3208935627] conn(0x5565ebe2c700 0x5565ebe375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f90ebfff640 10 client.4833 ms_handle_connect on v2:172.21.15.120:6834/3208935627 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f9115762640 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.120:6834/3208935627,v1:172.21.15.120:6837/3208935627] conn(0x5565ebe2c700 0x5565ebe375d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9af21f22afee5399 server_cookie=37b77d6d803a069a in_seq=0 out_seq=0 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f90ebfff640 1 -- 172.21.15.120:0/1305360279 <== mds.2 v2:172.21.15.120:6834/3208935627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565ebd47c50 con 0x5565ebe2c700 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f90ebfff640 10 client.4833 handle_command_reply: tid=0 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f911a837740 1 client.4833 shutdown 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f911a837740 1 -- 172.21.15.120:0/1305360279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 msgr2=0x5565ebe09570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f911a837740 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe09570 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9104061500 tx=0x7f910405c000 comp rx=0 tx=0).stop 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f911a837740 1 -- 172.21.15.120:0/1305360279 shutdown_connections 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f911a837740 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.120:6834/3208935627,v1:172.21.15.120:6837/3208935627] conn(0x5565ebe2c700 0x5565ebe375d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f911a837740 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 0x5565ebe06150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f911a837740 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe09570 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f911a837740 1 --2- 172.21.15.120:0/1305360279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 0x5565ebe0b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f90ebfff640 0 client.4833 ms_handle_reset on v2:172.21.15.120:6834/3208935627 2024-09-10T10:27:32.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.266+0000 7f911a837740 1 -- 172.21.15.120:0/1305360279 >> 172.21.15.120:0/1305360279 conn(0x5565ebdd7e70 msgr2=0x5565ebe124e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.267+0000 7f911a837740 1 -- 172.21.15.120:0/1305360279 shutdown_connections 2024-09-10T10:27:32.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.267+0000 7f911a837740 1 -- 172.21.15.120:0/1305360279 wait complete. 2024-09-10T10:27:32.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.267+0000 7f911a837740 20 client.4833 trim_cache size 0 max 16384 2024-09-10T10:27:32.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.284+0000 7f911a837740 1 Processor -- start 2024-09-10T10:27:32.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.284+0000 7f911a837740 1 -- start start 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f911a837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 0x5565ebe15800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f911a837740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebdd7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f911a837740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 0x5565ebdd78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f911a837740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91101fcb50 con 0x5565ebdd9750 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f911a837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565ebdd8fe0 con 0x5565ebdd6f80 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f911a837740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5565ebe29640 con 0x5565ebdd9b30 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9114f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebdd7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9114f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebdd7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38074/0 (socket says 172.21.15.120:38074) 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9114f61640 1 -- 172.21.15.120:0/1895658824 learned_addr learned my addr 172.21.15.120:0/1895658824 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9114f61640 1 -- 172.21.15.120:0/1895658824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 msgr2=0x5565ebdd78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9115f63640 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 0x5565ebdd78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9115762640 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 0x5565ebe15800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9114f61640 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 0x5565ebdd78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9114f61640 1 -- 172.21.15.120:0/1895658824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 msgr2=0x5565ebe15800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9114f61640 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 0x5565ebe15800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9114f61640 1 -- 172.21.15.120:0/1895658824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565ebdd5580 con 0x5565ebdd9750 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9115762640 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 0x5565ebe15800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9115f63640 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 0x5565ebdd78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f9114f61640 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebdd7360 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9104089440 tx=0x7f9104022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a001f1a9bf3c48d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f90f67fc640 1 -- 172.21.15.120:0/1895658824 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104087040 con 0x5565ebdd9750 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f90f67fc640 1 -- 172.21.15.120:0/1895658824 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910405b050 con 0x5565ebdd9750 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.285+0000 7f90f67fc640 1 -- 172.21.15.120:0/1895658824 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104056ef0 con 0x5565ebdd9750 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.286+0000 7f911a837740 1 -- 172.21.15.120:0/1895658824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 msgr2=0x5565ebdd7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.286+0000 7f911a837740 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebdd7360 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9104089440 tx=0x7f9104022000 comp rx=0 tx=0).stop 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.286+0000 7f911a837740 1 -- 172.21.15.120:0/1895658824 shutdown_connections 2024-09-10T10:27:32.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.286+0000 7f911a837740 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd9b30 0x5565ebdd78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.286+0000 7f911a837740 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebdd7360 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.286+0000 7f911a837740 1 --2- 172.21.15.120:0/1895658824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd6f80 0x5565ebe15800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.286+0000 7f911a837740 1 -- 172.21.15.120:0/1895658824 >> 172.21.15.120:0/1895658824 conn(0x5565ebdd7e70 msgr2=0x5565ebe22b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.286+0000 7f911a837740 1 -- 172.21.15.120:0/1895658824 shutdown_connections 2024-09-10T10:27:32.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.286+0000 7f911a837740 1 -- 172.21.15.120:0/1895658824 wait complete. 2024-09-10T10:27:32.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.286+0000 7f911a837740 1 -- start start 2024-09-10T10:27:32.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.287+0000 7f911a837740 1 Processor -- start 2024-09-10T10:27:32.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.287+0000 7f911a837740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd6f80 0x5565ebe99eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.287+0000 7f911a837740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe02070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.287+0000 7f911a837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd9b30 0x5565ebe08f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.287+0000 7f911a837740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565ebe0ab90 con 0x5565ebdd9750 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.287+0000 7f911a837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565ebdfe500 con 0x5565ebdd9b30 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.287+0000 7f911a837740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5565ebe06760 con 0x5565ebdd6f80 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.287+0000 7f911a837740 10 client.0 _ll_register_callbacks cb 0x7f9104085070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9114f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe02070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9114f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe02070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38084/0 (socket says 172.21.15.120:38084) 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9114f61640 1 -- 172.21.15.120:0/314172468 learned_addr learned my addr 172.21.15.120:0/314172468 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9115f63640 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd9b30 0x5565ebe08f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9115762640 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd6f80 0x5565ebe99eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9114f61640 1 -- 172.21.15.120:0/314172468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd6f80 msgr2=0x5565ebe99eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9114f61640 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd6f80 0x5565ebe99eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9114f61640 1 -- 172.21.15.120:0/314172468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd9b30 msgr2=0x5565ebe08f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9114f61640 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd9b30 0x5565ebe08f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9114f61640 1 -- 172.21.15.120:0/314172468 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565ebd45b70 con 0x5565ebdd9750 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f9114f61640 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe02070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9104059010 tx=0x7f9104055170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706c8cfb8b872ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f90ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f90ebfff640 1 -- 172.21.15.120:0/314172468 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104087040 con 0x5565ebdd9750 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f90ebfff640 1 -- 172.21.15.120:0/314172468 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f9104058540 con 0x5565ebdd9750 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f911a837740 1 -- 172.21.15.120:0/314172468 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910405b050 con 0x5565ebdd9750 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f90ebfff640 1 -- 172.21.15.120:0/314172468 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f910405f410 con 0x5565ebdd9750 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f911a837740 10 client.4839 fetch_fsmap learned FSMap version 18 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f911a837740 1 -- 172.21.15.120:0/314172468 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5565ebd45b70 con 0x5565ebdd9750 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f90ebfff640 1 -- 172.21.15.120:0/314172468 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910405b050 con 0x5565ebdd9750 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f90ebfff640 1 -- 172.21.15.120:0/314172468 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 18) ==== 4800+0+0 (secure 0 0 0) 0x7f9104081030 con 0x5565ebdd9750 2024-09-10T10:27:32.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f911a837740 10 client.4839 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f911a837740 10 client.4839 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f911a837740 20 client.4839 populate_metadata read hostname 'smithi120' 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f911a837740 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.120:6834/3208935627,v1:172.21.15.120:6837/3208935627] conn(0x5565ebe900e0 0x5565ebe904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f911a837740 4 client.4839 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.288+0000 7f911a837740 1 -- 172.21.15.120:0/314172468 --> [v2:172.21.15.120:6834/3208935627,v1:172.21.15.120:6837/3208935627] -- command(tid 0: {"prefix": "status"}) -- 0x5565ebd41fa0 con 0x5565ebe900e0 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f9115762640 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.120:6834/3208935627,v1:172.21.15.120:6837/3208935627] conn(0x5565ebe900e0 0x5565ebe904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f9115762640 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.120:6834/3208935627,v1:172.21.15.120:6837/3208935627] conn(0x5565ebe900e0 0x5565ebe904c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74f3b0e281f6c92e server_cookie=46e6a24d83b9e23a in_seq=0 out_seq=0 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f90ebfff640 10 client.4839 ms_handle_connect on v2:172.21.15.120:6834/3208935627 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f90ebfff640 1 -- 172.21.15.120:0/314172468 <== mds.2 v2:172.21.15.120:6834/3208935627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5565ebd41fa0 con 0x5565ebe900e0 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f90ebfff640 10 client.4839 handle_command_reply: tid=0 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 client.4839 shutdown 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 -- 172.21.15.120:0/314172468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 msgr2=0x5565ebe02070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe02070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9104059010 tx=0x7f9104055170 comp rx=0 tx=0).stop 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 -- 172.21.15.120:0/314172468 shutdown_connections 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.120:6834/3208935627,v1:172.21.15.120:6837/3208935627] conn(0x5565ebe900e0 0x5565ebe904c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565ebdd9b30 0x5565ebe08f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565ebdd9750 0x5565ebe02070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f90ebfff640 0 client.4839 ms_handle_reset on v2:172.21.15.120:6834/3208935627 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 --2- 172.21.15.120:0/314172468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5565ebdd6f80 0x5565ebe99eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 -- 172.21.15.120:0/314172468 >> 172.21.15.120:0/314172468 conn(0x5565ebdd7e70 msgr2=0x5565ebe22b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 -- 172.21.15.120:0/314172468 shutdown_connections 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 1 -- 172.21.15.120:0/314172468 wait complete. 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.289+0000 7f911a837740 20 client.4839 trim_cache size 0 max 16384 2024-09-10T10:27:32.298 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 2, 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 4288, 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 23.346902132, 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 18, 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 28, 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 28, 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 56.541517775999999 2024-09-10T10:27:32.299 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:32.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 -- 172.21.15.120:0/3085866792 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f90fc06be30 msgr2=0x7f90fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f90fc06be30 0x7f90fc06e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9100001d60 tx=0x7f9100009040 comp rx=0 tx=0).stop 2024-09-10T10:27:32.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 -- 172.21.15.120:0/3085866792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c6410 msgr2=0x7f9110155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c6410 0x7f9110155600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f910400d800 tx=0x7f910400dcc0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 -- 172.21.15.120:0/3085866792 shutdown_connections 2024-09-10T10:27:32.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f90fc06be30 0x7f90fc06e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9110151a90 0x7f9110152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91100c6410 0x7f9110155600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 --2- 172.21.15.120:0/3085866792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91100c5280 0x7f91101e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 -- 172.21.15.120:0/3085866792 >> 172.21.15.120:0/3085866792 conn(0x7f911006f3c0 msgr2=0x7f911006da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 -- 172.21.15.120:0/3085866792 shutdown_connections 2024-09-10T10:27:32.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.290+0000 7f91179c6640 1 -- 172.21.15.120:0/3085866792 wait complete. 2024-09-10T10:27:32.306 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T10:27:32.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.403+0000 7f3b0d65e640 1 Processor -- start 2024-09-10T10:27:32.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.403+0000 7f3b0d65e640 1 -- start start 2024-09-10T10:27:32.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b0d65e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b080cc0c0 0x7f3b0815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b0d65e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3b080cc4a0 0x7f3b08165430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b0d65e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b0815b1d0 con 0x7f3b080cb2f0 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b0d65e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3b080cbd00 con 0x7f3b080cc4a0 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b0d65e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0815ba70 con 0x7f3b080cc0c0 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3afe7fc640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f3b080cb2f0 0x7f3b0815e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:33548/0 (socket says 172.21.15.120:33548) 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3afe7fc640 1 -- 172.21.15.120:0/620543059 learned_addr learned my addr 172.21.15.120:0/620543059 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b077fe640 1 --2- 172.21.15.120:0/620543059 >> v2:172.21.15.134:3300/0 conn(0x7f3b080cc4a0 0x7f3b08165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b080cc0c0 0x7f3b0815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4729184 0 0) 0x7f3b0815b1d0 con 0x7f3b080cb2f0 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3af4003520 con 0x7f3b080cb2f0 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1704062973 0 0) 0x7f3af4003520 con 0x7f3b080cb2f0 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b080cc0c0 msgr2=0x7f3b0815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b080cc0c0 0x7f3b0815aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 >> v2:172.21.15.134:3300/0 conn(0x7f3b080cc4a0 msgr2=0x7f3b08165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 --2- 172.21.15.120:0/620543059 >> v2:172.21.15.134:3300/0 conn(0x7f3b080cc4a0 0x7f3b08165430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0815b4b0 con 0x7f3b080cb2f0 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b077fe640 1 --2- 172.21.15.120:0/620543059 >> v2:172.21.15.134:3300/0 conn(0x7f3b080cc4a0 0x7f3b08165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f3af0001670 con 0x7f3b080cb2f0 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 >> v1:172.21.15.120:6789/0 conn(0x7f3b080cb2f0 legacy=0x7f3b0815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af40039b0 0x7f3af4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.404+0000 7f3b067fc640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af4004800 0x7f3af4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b067fc640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b080cc0c0 0x7f3af40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b080cbd00 con 0x7f3af40039b0 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0815ba70 con 0x7f3af4004800 2024-09-10T10:27:32.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3af400b9c0 con 0x7f3b080cc0c0 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af40039b0 0x7f3af4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af4004800 0x7f3af4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b077fe640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b080cc0c0 0x7f3af40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b06ffd640 1 -- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b080cc0c0 msgr2=0x7f3af40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b080cc0c0 0x7f3af40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b06ffd640 1 -- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af4004800 msgr2=0x7f3af4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af4004800 0x7f3af4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3b06ffd640 1 -- 172.21.15.120:0/620543059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af4003520 con 0x7f3af40039b0 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.405+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af4004800 0x7f3af4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af40039b0 0x7f3af4003dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009ac0 tx=0x7f3af4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3735f55c1227bc86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8036070 con 0x7f3af40039b0 2024-09-10T10:27:32.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af8031400 con 0x7f3af40039b0 2024-09-10T10:27:32.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b067fc640 1 -- 172.21.15.120:0/620543059 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8009690 con 0x7f3af40039b0 2024-09-10T10:27:32.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b0d65e640 1 -- 172.21.15.120:0/620543059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af40039b0 msgr2=0x7f3af4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b0d65e640 1 --2- 172.21.15.120:0/620543059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af40039b0 0x7f3af4003dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009ac0 tx=0x7f3af4005830 comp rx=0 tx=0).stop 2024-09-10T10:27:32.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b077fe640 1 -- 172.21.15.120:0/620543059 reap_dead start 2024-09-10T10:27:32.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b0d65e640 1 -- 172.21.15.120:0/620543059 shutdown_connections 2024-09-10T10:27:32.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b0d65e640 1 -- 172.21.15.120:0/620543059 >> 172.21.15.120:0/620543059 conn(0x7f3b0806d560 msgr2=0x7f3b0806d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b0d65e640 1 -- 172.21.15.120:0/620543059 shutdown_connections 2024-09-10T10:27:32.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.406+0000 7f3b0d65e640 1 -- 172.21.15.120:0/620543059 wait complete. 2024-09-10T10:27:32.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.407+0000 7f3b0d65e640 1 Processor -- start 2024-09-10T10:27:32.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3b0d65e640 1 -- start start 2024-09-10T10:27:32.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3b0d65e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af40039b0 0x7f3b080b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3b0d65e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b080ca810 0x7f3b0815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3b0d65e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b0815c900 0x7f3b081f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3b0d65e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0815e1d0 con 0x7f3b080ca810 2024-09-10T10:27:32.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3b0d65e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0815e540 con 0x7f3af40039b0 2024-09-10T10:27:32.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3b0d65e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3b080cc8b0 con 0x7f3b0815c900 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3afe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b080ca810 0x7f3b0815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3afe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b080ca810 0x7f3b0815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38094/0 (socket says 172.21.15.120:38094) 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3afe7fc640 1 -- 172.21.15.120:0/2519411197 learned_addr learned my addr 172.21.15.120:0/2519411197 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.408+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af40039b0 0x7f3b080b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3b077fe640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b0815c900 0x7f3b081f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3afe7fc640 1 -- 172.21.15.120:0/2519411197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b0815c900 msgr2=0x7f3b081f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b0815c900 0x7f3b081f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3afe7fc640 1 -- 172.21.15.120:0/2519411197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af40039b0 msgr2=0x7f3b080b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af40039b0 0x7f3b080b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3afe7fc640 1 -- 172.21.15.120:0/2519411197 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0815d770 con 0x7f3b080ca810 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b080ca810 0x7f3b0815c3c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3af000e820 tx=0x7f3af000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3afffff640 1 -- 172.21.15.120:0/2519411197 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af000c870 con 0x7f3b080ca810 2024-09-10T10:27:32.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3afffff640 1 -- 172.21.15.120:0/2519411197 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af0018070 con 0x7f3b080ca810 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3b0d65e640 1 -- 172.21.15.120:0/2519411197 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0815da50 con 0x7f3b080ca810 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3afffff640 1 -- 172.21.15.120:0/2519411197 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af000cc10 con 0x7f3b080ca810 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.409+0000 7f3b0d65e640 1 -- 172.21.15.120:0/2519411197 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0815bea0 con 0x7f3b080ca810 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.410+0000 7f3afffff640 1 -- 172.21.15.120:0/2519411197 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3af001d460 con 0x7f3b080ca810 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.411+0000 7f3afffff640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3ae806be60 0x7f3ae806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.411+0000 7f3afffff640 1 -- 172.21.15.120:0/2519411197 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f3af008b090 con 0x7f3b080ca810 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.411+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3ae806be60 0x7f3ae806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.411+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3ae806be60 0x7f3ae806e320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3af8033000 tx=0x7f3af8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.412+0000 7f3b103fc740 1 Processor -- start 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.412+0000 7f3b103fc740 1 -- start start 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942fe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942fef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b103fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562d94227520 con 0x562d942c6b30 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b103fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562d941cf3e0 con 0x562d942c6750 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b103fc740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562d942234e0 con 0x562d942c3f80 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942c2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38098/0 (socket says 172.21.15.120:38098) 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b06ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942fe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b077fe640 1 -- 172.21.15.120:0/323186939 learned_addr learned my addr 172.21.15.120:0/323186939 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/323186939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942fef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b077fe640 1 -- 172.21.15.120:0/323186939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 msgr2=0x562d942fef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b077fe640 1 --2- 172.21.15.120:0/323186939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942fef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b077fe640 1 -- 172.21.15.120:0/323186939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 msgr2=0x562d942fe9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b077fe640 1 --2- 172.21.15.120:0/323186939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942fe9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b077fe640 1 -- 172.21.15.120:0/323186939 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d941ea2f0 con 0x562d942c6b30 2024-09-10T10:27:32.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3b077fe640 1 --2- 172.21.15.120:0/323186939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942c2a60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3b0808e390 tx=0x7f3b00009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b50695b6b42391 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3afcff9640 1 -- 172.21.15.120:0/323186939 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0001f070 con 0x562d942c6b30 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3afcff9640 1 -- 172.21.15.120:0/323186939 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b00003b90 con 0x562d942c6b30 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.413+0000 7f3afcff9640 1 -- 172.21.15.120:0/323186939 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00004030 con 0x562d942c6b30 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.414+0000 7f3b103fc740 1 -- 172.21.15.120:0/323186939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 msgr2=0x562d942c2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.414+0000 7f3b103fc740 1 --2- 172.21.15.120:0/323186939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942c2a60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3b0808e390 tx=0x7f3b00009750 comp rx=0 tx=0).stop 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.414+0000 7f3b103fc740 1 -- 172.21.15.120:0/323186939 shutdown_connections 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.414+0000 7f3b103fc740 1 --2- 172.21.15.120:0/323186939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942c2a60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.414+0000 7f3b103fc740 1 --2- 172.21.15.120:0/323186939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.414+0000 7f3b103fc740 1 --2- 172.21.15.120:0/323186939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942fe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.414+0000 7f3b103fc740 1 -- 172.21.15.120:0/323186939 >> 172.21.15.120:0/323186939 conn(0x562d942c4e70 msgr2=0x562d942c5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.414+0000 7f3b103fc740 1 -- 172.21.15.120:0/323186939 shutdown_connections 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.414+0000 7f3b103fc740 1 -- 172.21.15.120:0/323186939 wait complete. 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.415+0000 7f3b103fc740 1 -- start start 2024-09-10T10:27:32.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.415+0000 7f3b103fc740 1 Processor -- start 2024-09-10T10:27:32.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.415+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.415+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942e8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b103fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562d942c5fe0 con 0x562d942c6b30 2024-09-10T10:27:32.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b103fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562d941b0450 con 0x562d942c6750 2024-09-10T10:27:32.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b103fc740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562d942fe550 con 0x562d942c3f80 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b103fc740 10 client.0 _ll_register_callbacks cb 0x562d93f39ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38102/0 (socket says 172.21.15.120:38102) 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b077fe640 1 -- 172.21.15.120:0/894891285 learned_addr learned my addr 172.21.15.120:0/894891285 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942e8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b077fe640 1 -- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 msgr2=0x562d942eb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b077fe640 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942eb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b077fe640 1 -- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 msgr2=0x562d942e8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b077fe640 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942e8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b077fe640 1 -- 172.21.15.120:0/894891285 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d93f3a2d0 con 0x562d942c6b30 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b077fe640 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942f37c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3b00013b10 tx=0x7f3b00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154839519c359d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3ae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:27:32.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/894891285 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0001f040 con 0x562d942c6b30 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/894891285 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f3b00003d40 con 0x562d942c6b30 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b103fc740 1 -- 172.21.15.120:0/894891285 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b00003b90 con 0x562d942c6b30 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/894891285 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3b00032050 con 0x562d942c6b30 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b103fc740 10 client.4851 fetch_fsmap learned FSMap version 18 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3b103fc740 1 -- 172.21.15.120:0/894891285 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x562d940046b0 con 0x562d942c6b30 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.416+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/894891285 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b00003b90 con 0x562d942c6b30 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.417+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/894891285 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 18) ==== 4800+0+0 (secure 0 0 0) 0x7f3b00035020 con 0x562d942c6b30 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.417+0000 7f3b103fc740 10 client.4851 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.417+0000 7f3b103fc740 10 client.4851 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.417+0000 7f3b103fc740 20 client.4851 populate_metadata read hostname 'smithi120' 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.417+0000 7f3b103fc740 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:6832/231686968,v1:172.21.15.134:6834/231686968] conn(0x562d942e91e0 0x562d94324640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.417+0000 7f3b103fc740 4 client.4851 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:27:32.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.417+0000 7f3b103fc740 1 -- 172.21.15.120:0/894891285 --> [v2:172.21.15.134:6832/231686968,v1:172.21.15.134:6834/231686968] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d94234c50 con 0x562d942e91e0 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.417+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:6832/231686968,v1:172.21.15.134:6834/231686968] conn(0x562d942e91e0 0x562d94324640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.417+0000 7f3ae5ffb640 10 client.4851 ms_handle_connect on v2:172.21.15.134:6832/231686968 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.417+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:6832/231686968,v1:172.21.15.134:6834/231686968] conn(0x562d942e91e0 0x562d94324640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65f370010ab0941c server_cookie=b9899a2d8baea528 in_seq=0 out_seq=0 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/894891285 <== mds.1 v2:172.21.15.134:6832/231686968 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d94234c50 con 0x562d942e91e0 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3ae5ffb640 10 client.4851 handle_command_reply: tid=0 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3b103fc740 1 client.4851 shutdown 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3b103fc740 1 -- 172.21.15.120:0/894891285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 msgr2=0x562d942f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3b103fc740 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942f37c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3b00013b10 tx=0x7f3b00004a40 comp rx=0 tx=0).stop 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3b103fc740 1 -- 172.21.15.120:0/894891285 shutdown_connections 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3b103fc740 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:6832/231686968,v1:172.21.15.134:6834/231686968] conn(0x562d942e91e0 0x562d94324640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3b103fc740 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6b30 0x562d942f37c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3b103fc740 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3ae5ffb640 0 client.4851 ms_handle_reset on v2:172.21.15.134:6832/231686968 2024-09-10T10:27:32.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3b103fc740 1 --2- 172.21.15.120:0/894891285 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942eb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.418+0000 7f3b103fc740 1 -- 172.21.15.120:0/894891285 >> 172.21.15.120:0/894891285 conn(0x562d942c4e70 msgr2=0x562d942ed2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.419+0000 7f3b103fc740 1 -- 172.21.15.120:0/894891285 shutdown_connections 2024-09-10T10:27:32.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.419+0000 7f3b103fc740 1 -- 172.21.15.120:0/894891285 wait complete. 2024-09-10T10:27:32.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.419+0000 7f3b103fc740 20 client.4851 trim_cache size 0 max 16384 2024-09-10T10:27:32.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.436+0000 7f3b103fc740 1 Processor -- start 2024-09-10T10:27:32.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.436+0000 7f3b103fc740 1 -- start start 2024-09-10T10:27:32.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c3f80 0x562d942fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c6b30 0x562d942c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b103fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081fd000 con 0x562d942c3f80 2024-09-10T10:27:32.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b103fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562d942eef80 con 0x562d942c6750 2024-09-10T10:27:32.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b103fc740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562d942eb6f0 con 0x562d942c6b30 2024-09-10T10:27:32.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b06ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c3f80 0x562d942fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b06ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c3f80 0x562d942fa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38116/0 (socket says 172.21.15.120:38116) 2024-09-10T10:27:32.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b06ffd640 1 -- 172.21.15.120:0/4244172835 learned_addr learned my addr 172.21.15.120:0/4244172835 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b06ffd640 1 -- 172.21.15.120:0/4244172835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c6b30 msgr2=0x562d942c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b077fe640 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c6b30 0x562d942c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c6b30 0x562d942c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b06ffd640 1 -- 172.21.15.120:0/4244172835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 msgr2=0x562d942c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b06ffd640 1 -- 172.21.15.120:0/4244172835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d942bb8b0 con 0x562d942c3f80 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b077fe640 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c6b30 0x562d942c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.437+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c3f80 0x562d942fa270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009be0 tx=0x7f3af802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfa33e37ee2c563 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3ae7fff640 1 -- 172.21.15.120:0/4244172835 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8056070 con 0x562d942c3f80 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3ae7fff640 1 -- 172.21.15.120:0/4244172835 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af80039e0 con 0x562d942c3f80 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3ae7fff640 1 -- 172.21.15.120:0/4244172835 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8040040 con 0x562d942c3f80 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3b103fc740 1 -- 172.21.15.120:0/4244172835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c3f80 msgr2=0x562d942fa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3b103fc740 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c3f80 0x562d942fa270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009be0 tx=0x7f3af802e000 comp rx=0 tx=0).stop 2024-09-10T10:27:32.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3b103fc740 1 -- 172.21.15.120:0/4244172835 shutdown_connections 2024-09-10T10:27:32.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3b103fc740 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c6b30 0x562d942c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3b103fc740 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6750 0x562d942c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3b103fc740 1 --2- 172.21.15.120:0/4244172835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c3f80 0x562d942fa270 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3b103fc740 1 -- 172.21.15.120:0/4244172835 >> 172.21.15.120:0/4244172835 conn(0x562d942c4e70 msgr2=0x562d942bfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3b103fc740 1 -- 172.21.15.120:0/4244172835 shutdown_connections 2024-09-10T10:27:32.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.438+0000 7f3b103fc740 1 -- 172.21.15.120:0/4244172835 wait complete. 2024-09-10T10:27:32.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.439+0000 7f3b103fc740 1 -- start start 2024-09-10T10:27:32.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.439+0000 7f3b103fc740 1 Processor -- start 2024-09-10T10:27:32.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.439+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.439+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6750 0x562d942c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.439+0000 7f3b103fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6b30 0x562d94385c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.439+0000 7f3b103fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562d942e9610 con 0x562d942c6750 2024-09-10T10:27:32.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.439+0000 7f3b103fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562d942ff090 con 0x562d942c6b30 2024-09-10T10:27:32.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.439+0000 7f3b103fc740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562d94319000 con 0x562d942c3f80 2024-09-10T10:27:32.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.439+0000 7f3b103fc740 10 client.0 _ll_register_callbacks cb 0x7f3b0001d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:27:32.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3afe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6750 0x562d942c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3afe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6750 0x562d942c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38122/0 (socket says 172.21.15.120:38122) 2024-09-10T10:27:32.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3afe7fc640 1 -- 172.21.15.120:0/2999476349 learned_addr learned my addr 172.21.15.120:0/2999476349 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3b077fe640 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6b30 0x562d94385c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3afe7fc640 1 -- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 msgr2=0x562d942ea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942ea270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3afe7fc640 1 -- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6b30 msgr2=0x562d94385c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6b30 0x562d94385c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3afe7fc640 1 -- 172.21.15.120:0/2999476349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d942ea7b0 con 0x562d942c6750 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3afe7fc640 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6750 0x562d942c58a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3af0058c40 tx=0x7f3af0059780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82390c3cd5e9cab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3ae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/2999476349 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af00555b0 con 0x562d942c6750 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/2999476349 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f3af0061690 con 0x562d942c6750 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3b103fc740 1 -- 172.21.15.120:0/2999476349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af80039e0 con 0x562d942c6750 2024-09-10T10:27:32.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/2999476349 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3af0055750 con 0x562d942c6750 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3b103fc740 10 client.4857 fetch_fsmap learned FSMap version 18 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3b103fc740 1 -- 172.21.15.120:0/2999476349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x562d942f6a90 con 0x562d942c6750 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/2999476349 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af80039e0 con 0x562d942c6750 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.440+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/2999476349 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 18) ==== 4800+0+0 (secure 0 0 0) 0x7f3af0058390 con 0x562d942c6750 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 10 client.4857 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 10 client.4857 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 20 client.4857 populate_metadata read hostname 'smithi120' 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:6832/231686968,v1:172.21.15.134:6834/231686968] conn(0x562d9438fbc0 0x562d9438ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 4 client.4857 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 1 -- 172.21.15.120:0/2999476349 --> [v2:172.21.15.134:6832/231686968,v1:172.21.15.134:6834/231686968] -- command(tid 0: {"prefix": "status"}) -- 0x562d9422efa0 con 0x562d9438fbc0 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:6832/231686968,v1:172.21.15.134:6834/231686968] conn(0x562d9438fbc0 0x562d9438ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3ae5ffb640 10 client.4857 ms_handle_connect on v2:172.21.15.134:6832/231686968 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b06ffd640 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:6832/231686968,v1:172.21.15.134:6834/231686968] conn(0x562d9438fbc0 0x562d9438ffa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5bb7ccd990f836f server_cookie=ced0e6db2e8830ef in_seq=0 out_seq=0 2024-09-10T10:27:32.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3ae5ffb640 1 -- 172.21.15.120:0/2999476349 <== mds.1 v2:172.21.15.134:6832/231686968 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562d9422efa0 con 0x562d9438fbc0 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3ae5ffb640 10 client.4857 handle_command_reply: tid=0 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 1 client.4857 shutdown 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 1 -- 172.21.15.120:0/2999476349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6750 msgr2=0x562d942c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6750 0x562d942c58a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3af0058c40 tx=0x7f3af0059780 comp rx=0 tx=0).stop 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 1 -- 172.21.15.120:0/2999476349 shutdown_connections 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:6832/231686968,v1:172.21.15.134:6834/231686968] conn(0x562d9438fbc0 0x562d9438ffa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.441+0000 7f3b103fc740 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d942c6b30 0x562d94385c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b103fc740 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d942c6750 0x562d942c58a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3ae5ffb640 0 client.4857 ms_handle_reset on v2:172.21.15.134:6832/231686968 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b103fc740 1 --2- 172.21.15.120:0/2999476349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562d942c3f80 0x562d942ea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b103fc740 1 -- 172.21.15.120:0/2999476349 >> 172.21.15.120:0/2999476349 conn(0x562d942c4e70 msgr2=0x562d942fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b103fc740 1 -- 172.21.15.120:0/2999476349 shutdown_connections 2024-09-10T10:27:32.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b103fc740 1 -- 172.21.15.120:0/2999476349 wait complete. 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b103fc740 20 client.4857 trim_cache size 0 max 16384 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 1, 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 4303, 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 28.576022308999999, 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 18, 2024-09-10T10:27:32.451 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 28, 2024-09-10T10:27:32.452 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 28, 2024-09-10T10:27:32.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 56.673593275999998 2024-09-10T10:27:32.452 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 -- 172.21.15.120:0/2519411197 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3ae806be60 msgr2=0x7f3ae806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3ae806be60 0x7f3ae806e320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3af8033000 tx=0x7f3af8008030 comp rx=0 tx=0).stop 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 -- 172.21.15.120:0/2519411197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b080ca810 msgr2=0x7f3b0815c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b080ca810 0x7f3b0815c3c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3af000e820 tx=0x7f3af000ece0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 -- 172.21.15.120:0/2519411197 shutdown_connections 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3ae806be60 0x7f3ae806e320 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b0815c900 0x7f3b081f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b080ca810 0x7f3b0815c3c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 --2- 172.21.15.120:0/2519411197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af40039b0 0x7f3b080b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 -- 172.21.15.120:0/2519411197 >> 172.21.15.120:0/2519411197 conn(0x7f3b0806d560 msgr2=0x7f3b080d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 -- 172.21.15.120:0/2519411197 shutdown_connections 2024-09-10T10:27:32.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.442+0000 7f3b0d65e640 1 -- 172.21.15.120:0/2519411197 wait complete. 2024-09-10T10:27:32.455 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T10:27:32.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.552+0000 7f6bff8d0640 1 Processor -- start 2024-09-10T10:27:32.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.553+0000 7f6bff8d0640 1 -- start start 2024-09-10T10:27:32.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.553+0000 7f6bff8d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bf80ca7b0 0x7f6bf815e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.553+0000 7f6bff8d0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6bf80c7050 0x7f6bf815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.553+0000 7f6bff8d0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bf815ead0 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.553+0000 7f6bff8d0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f6bf80c9240 con 0x7f6bf80c7050 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.553+0000 7f6bff8d0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6bf815edb0 con 0x7f6bf80ca7b0 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.553+0000 7f6bfd66c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f6bf80ca3d0 0x7f6bf80c6940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:33562/0 (socket says 172.21.15.120:33562) 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.553+0000 7f6bfd66c640 1 -- 172.21.15.120:0/2038264480 learned_addr learned my addr 172.21.15.120:0/2038264480 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.553+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/2038264480 >> v2:172.21.15.120:3300/0 conn(0x7f6bf80c7050 0x7f6bf815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.554+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/2038264480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bf80ca7b0 0x7f6bf815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.554+0000 7f6bf7fff640 1 -- 172.21.15.120:0/2038264480 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3578384427 0 0) 0x7f6bf815ead0 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.554+0000 7f6bf7fff640 1 -- 172.21.15.120:0/2038264480 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6be0003520 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.554+0000 7f6bfde6d640 1 -- 172.21.15.120:0/2038264480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bf80ca7b0 msgr2=0x7f6bf815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.554+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/2038264480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bf80ca7b0 0x7f6bf815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.554+0000 7f6bfde6d640 1 -- 172.21.15.120:0/2038264480 >> v1:172.21.15.120:6789/0 conn(0x7f6bf80ca3d0 legacy=0x7f6bf80c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.554+0000 7f6bfde6d640 1 -- 172.21.15.120:0/2038264480 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf815f090 con 0x7f6bf80c7050 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/2038264480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80c7050 0x7f6bf815aec0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6be80097a0 tx=0x7f6be802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b54580a6f8de5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bf7fff640 1 -- 172.21.15.120:0/2038264480 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8037070 con 0x7f6bf80c7050 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bf7fff640 1 -- 172.21.15.120:0/2038264480 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be8032440 con 0x7f6bf80c7050 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bf7fff640 1 -- 172.21.15.120:0/2038264480 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8032810 con 0x7f6bf80c7050 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bff8d0640 1 -- 172.21.15.120:0/2038264480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80c7050 msgr2=0x7f6bf815aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bff8d0640 1 --2- 172.21.15.120:0/2038264480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80c7050 0x7f6bf815aec0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6be80097a0 tx=0x7f6be802a210 comp rx=0 tx=0).stop 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bff8d0640 1 -- 172.21.15.120:0/2038264480 shutdown_connections 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bff8d0640 1 --2- 172.21.15.120:0/2038264480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80c7050 0x7f6bf815aec0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bff8d0640 1 --2- 172.21.15.120:0/2038264480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bf80ca7b0 0x7f6bf815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bff8d0640 1 -- 172.21.15.120:0/2038264480 >> 172.21.15.120:0/2038264480 conn(0x7f6bf80cac70 msgr2=0x7f6bf80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bff8d0640 1 -- 172.21.15.120:0/2038264480 shutdown_connections 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.555+0000 7f6bff8d0640 1 -- 172.21.15.120:0/2038264480 wait complete. 2024-09-10T10:27:32.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.556+0000 7f6bff8d0640 1 Processor -- start 2024-09-10T10:27:32.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.556+0000 7f6bff8d0640 1 -- start start 2024-09-10T10:27:32.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bff8d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bf80c7050 0x7f6bf80c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bff8d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80ca3d0 0x7f6bf81f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bff8d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf80ca7b0 0x7f6bf8150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bff8d0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf80cba40 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bff8d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf80c6420 con 0x7f6bf80ca7b0 2024-09-10T10:27:32.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bff8d0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6bf80cbbc0 con 0x7f6bf80c7050 2024-09-10T10:27:32.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bfce6b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80ca3d0 0x7f6bf81f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bfce6b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80ca3d0 0x7f6bf81f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38138/0 (socket says 172.21.15.120:38138) 2024-09-10T10:27:32.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bfce6b640 1 -- 172.21.15.120:0/127597409 learned_addr learned my addr 172.21.15.120:0/127597409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bf80c7050 0x7f6bf80c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.557+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf80ca7b0 0x7f6bf8150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bfce6b640 1 -- 172.21.15.120:0/127597409 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bf80c7050 msgr2=0x7f6bf80c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bf80c7050 0x7f6bf80c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bfce6b640 1 -- 172.21.15.120:0/127597409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf80ca7b0 msgr2=0x7f6bf8150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf80ca7b0 0x7f6bf8150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bfce6b640 1 -- 172.21.15.120:0/127597409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf81f3f80 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf80ca7b0 0x7f6bf8150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80ca3d0 0x7f6bf81f4760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6bec007e60 tx=0x7f6bec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bf5ffb640 1 -- 172.21.15.120:0/127597409 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec014620 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bf5ffb640 1 -- 172.21.15.120:0/127597409 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bec019070 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bf5ffb640 1 -- 172.21.15.120:0/127597409 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec014a00 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bff8d0640 1 -- 172.21.15.120:0/127597409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf8158020 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.558+0000 7f6bff8d0640 1 -- 172.21.15.120:0/127597409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf81585e0 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.559+0000 7f6bf5ffb640 1 -- 172.21.15.120:0/127597409 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bec014ba0 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.560+0000 7f6bf5ffb640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6be406be60 0x7f6be406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.560+0000 7f6bf5ffb640 1 -- 172.21.15.120:0/127597409 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f6bec08b9b0 con 0x7f6bf80ca3d0 2024-09-10T10:27:32.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.560+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6be406be60 0x7f6be406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.560+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6be406be60 0x7f6be406e320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0005880 tx=0x7f6bf0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.561+0000 7f6c02668740 1 Processor -- start 2024-09-10T10:27:32.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.561+0000 7f6c02668740 1 -- start start 2024-09-10T10:27:32.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.561+0000 7f6c02668740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.561+0000 7f6c02668740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29586f80 0x557e295c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.561+0000 7f6c02668740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e29585a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.561+0000 7f6c02668740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e294ea520 con 0x557e29589b30 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.561+0000 7f6c02668740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557e294923e0 con 0x557e29589750 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.561+0000 7f6c02668740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x557e294e64e0 con 0x557e29586f80 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfde6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e29585a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfde6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e29585a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38142/0 (socket says 172.21.15.120:38142) 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfde6d640 1 -- 172.21.15.120:0/1936240339 learned_addr learned my addr 172.21.15.120:0/1936240339 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/1936240339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/1936240339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29586f80 0x557e295c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfde6d640 1 -- 172.21.15.120:0/1936240339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29586f80 msgr2=0x557e295c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/1936240339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29586f80 0x557e295c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfde6d640 1 -- 172.21.15.120:0/1936240339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 msgr2=0x557e295c19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/1936240339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295c19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfde6d640 1 -- 172.21.15.120:0/1936240339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e294ad2f0 con 0x557e29589b30 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/1936240339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e29585a60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009890 tx=0x7f6be802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885367bd16340b03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bdeffd640 1 -- 172.21.15.120:0/1936240339 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8037040 con 0x557e29589b30 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bdeffd640 1 -- 172.21.15.120:0/1936240339 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be803e660 con 0x557e29589b30 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.562+0000 7f6bdeffd640 1 -- 172.21.15.120:0/1936240339 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be803ea10 con 0x557e29589b30 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 -- 172.21.15.120:0/1936240339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 msgr2=0x557e29585a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 --2- 172.21.15.120:0/1936240339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e29585a60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009890 tx=0x7f6be802c000 comp rx=0 tx=0).stop 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 -- 172.21.15.120:0/1936240339 shutdown_connections 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 --2- 172.21.15.120:0/1936240339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e29585a60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 --2- 172.21.15.120:0/1936240339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29586f80 0x557e295c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 --2- 172.21.15.120:0/1936240339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295c19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 -- 172.21.15.120:0/1936240339 >> 172.21.15.120:0/1936240339 conn(0x557e29587e70 msgr2=0x557e29588270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 -- 172.21.15.120:0/1936240339 shutdown_connections 2024-09-10T10:27:32.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 -- 172.21.15.120:0/1936240339 wait complete. 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 -- start start 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.563+0000 7f6c02668740 1 Processor -- start 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.564+0000 7f6c02668740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29586f80 0x557e295afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.564+0000 7f6c02668740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.564+0000 7f6c02668740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e295bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.564+0000 7f6c02668740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e29588fe0 con 0x557e29589b30 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.564+0000 7f6c02668740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557e29473450 con 0x557e29589750 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.564+0000 7f6c02668740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x557e295c1550 con 0x557e29586f80 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.564+0000 7f6c02668740 10 client.0 _ll_register_callbacks cb 0x557e291fcac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.564+0000 7f6bfde6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e295bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.564+0000 7f6bfde6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e295bc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38152/0 (socket says 172.21.15.120:38152) 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.564+0000 7f6bfde6d640 1 -- 172.21.15.120:0/1700653022 learned_addr learned my addr 172.21.15.120:0/1700653022 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29586f80 0x557e295afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bfde6d640 1 -- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29586f80 msgr2=0x557e295afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29586f80 0x557e295afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bfde6d640 1 -- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 msgr2=0x557e295abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bfde6d640 1 -- 172.21.15.120:0/1700653022 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e291fd2d0 con 0x557e29589b30 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e295bc050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009510 tx=0x7f6be802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1df86fd2cb32f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bd3fff640 1 -- 172.21.15.120:0/1700653022 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8037070 con 0x557e29589b30 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bd3fff640 1 -- 172.21.15.120:0/1700653022 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f6be803ea20 con 0x557e29589b30 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6c02668740 1 -- 172.21.15.120:0/1700653022 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6be803e660 con 0x557e29589b30 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bd3fff640 1 -- 172.21.15.120:0/1700653022 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6be8050050 con 0x557e29589b30 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6c02668740 10 client.4872 fetch_fsmap learned FSMap version 18 2024-09-10T10:27:32.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6c02668740 1 -- 172.21.15.120:0/1700653022 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x557e292c76b0 con 0x557e29589b30 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.565+0000 7f6bd3fff640 1 -- 172.21.15.120:0/1700653022 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be803e660 con 0x557e29589b30 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.566+0000 7f6bd3fff640 1 -- 172.21.15.120:0/1700653022 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 18) ==== 4800+0+0 (secure 0 0 0) 0x7f6be8053020 con 0x557e29589b30 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.566+0000 7f6c02668740 10 client.4872 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.566+0000 7f6c02668740 10 client.4872 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.566+0000 7f6c02668740 20 client.4872 populate_metadata read hostname 'smithi120' 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.566+0000 7f6c02668740 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x557e295aa0e0 0x557e295864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.566+0000 7f6c02668740 4 client.4872 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.566+0000 7f6c02668740 1 -- 172.21.15.120:0/1700653022 --> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e294f7c50 con 0x557e295aa0e0 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.566+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x557e295aa0e0 0x557e295864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.566+0000 7f6bd3fff640 10 client.4872 ms_handle_connect on v2:172.21.15.134:6833/1434442356 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.566+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x557e295aa0e0 0x557e295864a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f63f4b2511c3766b server_cookie=f10e4fcb00fbbb14 in_seq=0 out_seq=0 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6bd3fff640 1 -- 172.21.15.120:0/1700653022 <== mds.0 v2:172.21.15.134:6833/1434442356 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e294f7c50 con 0x557e295aa0e0 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6bd3fff640 10 client.4872 handle_command_reply: tid=0 2024-09-10T10:27:32.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 client.4872 shutdown 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 -- 172.21.15.120:0/1700653022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 msgr2=0x557e295bc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e295bc050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009510 tx=0x7f6be802c070 comp rx=0 tx=0).stop 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 -- 172.21.15.120:0/1700653022 shutdown_connections 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x557e295aa0e0 0x557e295864a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29589b30 0x557e295bc050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6bd3fff640 0 client.4872 ms_handle_reset on v2:172.21.15.134:6833/1434442356 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 --2- 172.21.15.120:0/1700653022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29586f80 0x557e295afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 -- 172.21.15.120:0/1700653022 >> 172.21.15.120:0/1700653022 conn(0x557e29587e70 msgr2=0x557e295cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 -- 172.21.15.120:0/1700653022 shutdown_connections 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 1 -- 172.21.15.120:0/1700653022 wait complete. 2024-09-10T10:27:32.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.567+0000 7f6c02668740 20 client.4872 trim_cache size 0 max 16384 2024-09-10T10:27:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.585+0000 7f6c02668740 1 Processor -- start 2024-09-10T10:27:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.585+0000 7f6c02668740 1 -- start start 2024-09-10T10:27:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6c02668740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e29587550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6c02668740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295c5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6c02668740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589b30 0x557e295c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6c02668740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81571b0 con 0x557e29586f80 2024-09-10T10:27:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6c02668740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557e295af7e0 con 0x557e29589750 2024-09-10T10:27:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6c02668740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x557e295ac5b0 con 0x557e29589b30 2024-09-10T10:27:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfd66c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e29587550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfd66c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e29587550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38166/0 (socket says 172.21.15.120:38166) 2024-09-10T10:27:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfd66c640 1 -- 172.21.15.120:0/1830172161 learned_addr learned my addr 172.21.15.120:0/1830172161 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfd66c640 1 -- 172.21.15.120:0/1830172161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589b30 msgr2=0x557e295c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589b30 0x557e295c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589b30 0x557e295c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfd66c640 1 -- 172.21.15.120:0/1830172161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 msgr2=0x557e295c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfd66c640 1 -- 172.21.15.120:0/1830172161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e29585580 con 0x557e29586f80 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589b30 0x557e295c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e29587550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000fd90 tx=0x7f6bf0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a0b99cd373c7ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bde7fc640 1 -- 172.21.15.120:0/1830172161 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0014990 con 0x557e29586f80 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bde7fc640 1 -- 172.21.15.120:0/1830172161 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf0014b30 con 0x557e29586f80 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.586+0000 7f6bde7fc640 1 -- 172.21.15.120:0/1830172161 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0014ce0 con 0x557e29586f80 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 -- 172.21.15.120:0/1830172161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 msgr2=0x557e29587550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e29587550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000fd90 tx=0x7f6bf0012000 comp rx=0 tx=0).stop 2024-09-10T10:27:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 -- 172.21.15.120:0/1830172161 shutdown_connections 2024-09-10T10:27:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589b30 0x557e295c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589750 0x557e295c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 --2- 172.21.15.120:0/1830172161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e29587550 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 -- 172.21.15.120:0/1830172161 >> 172.21.15.120:0/1830172161 conn(0x557e29587e70 msgr2=0x557e295af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 -- 172.21.15.120:0/1830172161 shutdown_connections 2024-09-10T10:27:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 -- 172.21.15.120:0/1830172161 wait complete. 2024-09-10T10:27:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 -- start start 2024-09-10T10:27:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.587+0000 7f6c02668740 1 Processor -- start 2024-09-10T10:27:32.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.588+0000 7f6c02668740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e295acde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.588+0000 7f6c02668740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589750 0x557e29588440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.588+0000 7f6c02668740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589b30 0x557e29643ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.588+0000 7f6c02668740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e295a9f60 con 0x557e29586f80 2024-09-10T10:27:32.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.588+0000 7f6c02668740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557e295ae0b0 con 0x557e29589b30 2024-09-10T10:27:32.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.588+0000 7f6c02668740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x557e295aa610 con 0x557e29589750 2024-09-10T10:27:32.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.588+0000 7f6c02668740 10 client.0 _ll_register_callbacks cb 0x7f6bf00014d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:27:32.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.588+0000 7f6bfd66c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e295acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.588+0000 7f6bfd66c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e295acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38174/0 (socket says 172.21.15.120:38174) 2024-09-10T10:27:32.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.588+0000 7f6bfd66c640 1 -- 172.21.15.120:0/3927172087 learned_addr learned my addr 172.21.15.120:0/3927172087 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589750 0x557e29588440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bfde6d640 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589b30 0x557e29643ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bfd66c640 1 -- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589750 msgr2=0x557e29588440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589750 0x557e29588440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bfd66c640 1 -- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589b30 msgr2=0x557e29643ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589b30 0x557e29643ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bfd66c640 1 -- 172.21.15.120:0/3927172087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e295b95b0 con 0x557e29586f80 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589750 0x557e29588440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bfd66c640 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e295acde0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0000f60 tx=0x7f6bf0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719f1dc5443c6862 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bd3fff640 1 -- 172.21.15.120:0/3927172087 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf002ee30 con 0x557e29586f80 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bd3fff640 1 -- 172.21.15.120:0/3927172087 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f6bf00047e0 con 0x557e29586f80 2024-09-10T10:27:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6c02668740 1 -- 172.21.15.120:0/3927172087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf0014b30 con 0x557e29586f80 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bd3fff640 1 -- 172.21.15.120:0/3927172087 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6bf003d050 con 0x557e29586f80 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6c02668740 10 client.4878 fetch_fsmap learned FSMap version 18 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6c02668740 1 -- 172.21.15.120:0/3927172087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x557e295ad320 con 0x557e29586f80 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bd3fff640 1 -- 172.21.15.120:0/3927172087 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf0014b30 con 0x557e29586f80 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6bd3fff640 1 -- 172.21.15.120:0/3927172087 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 18) ==== 4800+0+0 (secure 0 0 0) 0x7f6bf0040020 con 0x557e29586f80 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6c02668740 10 client.4878 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6c02668740 10 client.4878 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6c02668740 20 client.4878 populate_metadata read hostname 'smithi120' 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6c02668740 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x557e295d5660 0x557e295d5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6c02668740 4 client.4878 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.589+0000 7f6c02668740 1 -- 172.21.15.120:0/3927172087 --> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] -- command(tid 0: {"prefix": "status"}) -- 0x557e294f1fa0 con 0x557e295d5660 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x557e295d5660 0x557e295d5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6bd3fff640 10 client.4878 ms_handle_connect on v2:172.21.15.134:6833/1434442356 2024-09-10T10:27:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6bfce6b640 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x557e295d5660 0x557e295d5a40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a819b87dc0dd9be server_cookie=26f1e193085a18bd in_seq=0 out_seq=0 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6bd3fff640 1 -- 172.21.15.120:0/3927172087 <== mds.0 v2:172.21.15.134:6833/1434442356 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x557e294f1fa0 con 0x557e295d5660 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6bd3fff640 10 client.4878 handle_command_reply: tid=0 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 client.4878 shutdown 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 -- 172.21.15.120:0/3927172087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 msgr2=0x557e295acde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e295acde0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0000f60 tx=0x7f6bf0000bc0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 -- 172.21.15.120:0/3927172087 shutdown_connections 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:6833/1434442356,v1:172.21.15.134:6835/1434442356] conn(0x557e295d5660 0x557e295d5a40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e29589b30 0x557e29643ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557e29589750 0x557e29588440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6bd3fff640 0 client.4878 ms_handle_reset on v2:172.21.15.134:6833/1434442356 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 --2- 172.21.15.120:0/3927172087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e29586f80 0x557e295acde0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 -- 172.21.15.120:0/3927172087 >> 172.21.15.120:0/3927172087 conn(0x557e29587e70 msgr2=0x557e295abad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 -- 172.21.15.120:0/3927172087 shutdown_connections 2024-09-10T10:27:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.590+0000 7f6c02668740 1 -- 172.21.15.120:0/3927172087 wait complete. 2024-09-10T10:27:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6c02668740 20 client.4878 trim_cache size 0 max 16384 2024-09-10T10:27:32.600 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:27:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 0, 2024-09-10T10:27:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 4306, 2024-09-10T10:27:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-10T10:27:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-10T10:27:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-10T10:27:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 50.816964894999998, 2024-09-10T10:27:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 18, 2024-09-10T10:27:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 28, 2024-09-10T10:27:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 28, 2024-09-10T10:27:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 56.822279041000002 2024-09-10T10:27:32.601 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 -- 172.21.15.120:0/127597409 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6be406be60 msgr2=0x7f6be406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6be406be60 0x7f6be406e320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0005880 tx=0x7f6bf0009040 comp rx=0 tx=0).stop 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 -- 172.21.15.120:0/127597409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80ca3d0 msgr2=0x7f6bf81f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80ca3d0 0x7f6bf81f4760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6bec007e60 tx=0x7f6bec00e4b0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 -- 172.21.15.120:0/127597409 shutdown_connections 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6be406be60 0x7f6be406e320 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bf80ca7b0 0x7f6bf8150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf80ca3d0 0x7f6bf81f4760 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 --2- 172.21.15.120:0/127597409 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bf80c7050 0x7f6bf80c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 -- 172.21.15.120:0/127597409 >> 172.21.15.120:0/127597409 conn(0x7f6bf80cac70 msgr2=0x7f6bf806ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 -- 172.21.15.120:0/127597409 shutdown_connections 2024-09-10T10:27:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:32.591+0000 7f6bff8d0640 1 -- 172.21.15.120:0/127597409 wait complete. 2024-09-10T10:27:32.605 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T10:27:32.817 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T10:27:32.817 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T10:27:32.818 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T10:27:32.818 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T10:27:32.833 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T10:27:32.834 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T10:27:32.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:32.834 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-09-10T10:27:32.850 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T10:27:32.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:32.850 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:27:32.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:32.918 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-09-10T10:27:33.026 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-10T10:27:33.027 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:27:33.027 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:27:33.027 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:27:33.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.125+0000 7fe542b9f640 1 Processor -- start 2024-09-10T10:27:33.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.126+0000 7fe542b9f640 1 -- start start 2024-09-10T10:27:33.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.126+0000 7fe542b9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 0x7fe53c1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.126+0000 7fe542b9f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe53c0c7b30 0x7fe53c151a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.126+0000 7fe542b9f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53c0c4890 con 0x7fe53c06cf90 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.126+0000 7fe542b9f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe53c0c8fc0 con 0x7fe53c0c7b30 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.126+0000 7fe542b9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c0c4630 con 0x7fe53c06d370 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.126+0000 7fe54113c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe53c0c7b30 0x7fe53c151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.127+0000 7fe54093b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fe53c06cf90 0x7fe53c0c9450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:33566/0 (socket says 172.21.15.120:33566) 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.127+0000 7fe54093b640 1 -- 172.21.15.120:0/2546989955 learned_addr learned my addr 172.21.15.120:0/2546989955 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.127+0000 7fe54113c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe53c0c7b30 0x7fe53c151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38186/0 (socket says 172.21.15.120:38186) 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.127+0000 7fe53bfff640 1 --2- 172.21.15.120:0/2546989955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 0x7fe53c1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.127+0000 7fe54113c640 1 -- 172.21.15.120:0/2546989955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 msgr2=0x7fe53c1514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.127+0000 7fe54113c640 1 --2- 172.21.15.120:0/2546989955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 0x7fe53c1514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.127+0000 7fe54113c640 1 -- 172.21.15.120:0/2546989955 >> v1:172.21.15.120:6789/0 conn(0x7fe53c06cf90 legacy=0x7fe53c0c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.127+0000 7fe54113c640 1 -- 172.21.15.120:0/2546989955 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c070d90 con 0x7fe53c0c7b30 2024-09-10T10:27:33.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.127+0000 7fe53b7fe640 1 -- 172.21.15.120:0/2546989955 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1541869118 0 0) 0x7fe53c0c4890 con 0x7fe53c06cf90 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.127+0000 7fe53bfff640 1 --2- 172.21.15.120:0/2546989955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 0x7fe53c1514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe54113c640 1 --2- 172.21.15.120:0/2546989955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c0c7b30 0x7fe53c151a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe53400ba40 tx=0x7fe53402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e6bef1506d1c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe53b7fe640 1 -- 172.21.15.120:0/2546989955 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534032750 con 0x7fe53c0c7b30 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe53b7fe640 1 -- 172.21.15.120:0/2546989955 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe534030050 con 0x7fe53c0c7b30 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe53b7fe640 1 -- 172.21.15.120:0/2546989955 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534032b10 con 0x7fe53c0c7b30 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe542b9f640 1 -- 172.21.15.120:0/2546989955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c0c7b30 msgr2=0x7fe53c151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe542b9f640 1 --2- 172.21.15.120:0/2546989955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c0c7b30 0x7fe53c151a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe53400ba40 tx=0x7fe53402c850 comp rx=0 tx=0).stop 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe542b9f640 1 -- 172.21.15.120:0/2546989955 shutdown_connections 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe542b9f640 1 --2- 172.21.15.120:0/2546989955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c0c7b30 0x7fe53c151a30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe542b9f640 1 --2- 172.21.15.120:0/2546989955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 0x7fe53c1514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe542b9f640 1 -- 172.21.15.120:0/2546989955 >> 172.21.15.120:0/2546989955 conn(0x7fe53c067d40 msgr2=0x7fe53c070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe542b9f640 1 -- 172.21.15.120:0/2546989955 shutdown_connections 2024-09-10T10:27:33.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.128+0000 7fe542b9f640 1 -- 172.21.15.120:0/2546989955 wait complete. 2024-09-10T10:27:33.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.129+0000 7fe542b9f640 1 Processor -- start 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.129+0000 7fe542b9f640 1 -- start start 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.130+0000 7fe542b9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c06cf90 0x7fe53c14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.130+0000 7fe542b9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 0x7fe53c14f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.130+0000 7fe542b9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c0c7b30 0x7fe53c1f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.130+0000 7fe542b9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c0c8660 con 0x7fe53c06cf90 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.130+0000 7fe542b9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c0c8380 con 0x7fe53c0c7b30 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.130+0000 7fe542b9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c0c8940 con 0x7fe53c06d370 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.130+0000 7fe54093b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c06cf90 0x7fe53c14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.130+0000 7fe54093b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c06cf90 0x7fe53c14dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38196/0 (socket says 172.21.15.120:38196) 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.130+0000 7fe54093b640 1 -- 172.21.15.120:0/517990996 learned_addr learned my addr 172.21.15.120:0/517990996 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe54113c640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c0c7b30 0x7fe53c1f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe53bfff640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 0x7fe53c14f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:33.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe54093b640 1 -- 172.21.15.120:0/517990996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 msgr2=0x7fe53c14f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe54093b640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 0x7fe53c14f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe54093b640 1 -- 172.21.15.120:0/517990996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c0c7b30 msgr2=0x7fe53c1f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe54093b640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c0c7b30 0x7fe53c1f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe54093b640 1 -- 172.21.15.120:0/517990996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c14d330 con 0x7fe53c06cf90 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe54113c640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c0c7b30 0x7fe53c1f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe53bfff640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 0x7fe53c14f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe54093b640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c06cf90 0x7fe53c14dc60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c8c0 tx=0x7fe52c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe5397fa640 1 -- 172.21.15.120:0/517990996 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c018070 con 0x7fe53c06cf90 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe5397fa640 1 -- 172.21.15.120:0/517990996 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52c00a640 con 0x7fe53c06cf90 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe542b9f640 1 -- 172.21.15.120:0/517990996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c14d5d0 con 0x7fe53c06cf90 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe5397fa640 1 -- 172.21.15.120:0/517990996 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00aa30 con 0x7fe53c06cf90 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.131+0000 7fe542b9f640 1 -- 172.21.15.120:0/517990996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c1f1740 con 0x7fe53c06cf90 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.132+0000 7fe542b9f640 1 -- 172.21.15.120:0/517990996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53c0cd080 con 0x7fe53c06cf90 2024-09-10T10:27:33.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.133+0000 7fe5397fa640 1 -- 172.21.15.120:0/517990996 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe52c01b070 con 0x7fe53c06cf90 2024-09-10T10:27:33.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.133+0000 7fe5397fa640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe52806be60 0x7fe52806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:33.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.133+0000 7fe53bfff640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe52806be60 0x7fe52806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:33.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.133+0000 7fe5397fa640 1 -- 172.21.15.120:0/517990996 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fe52c08a830 con 0x7fe53c06cf90 2024-09-10T10:27:33.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.134+0000 7fe53bfff640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe52806be60 0x7fe52806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe53c146a30 tx=0x7fe530008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:33.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.138+0000 7fe5397fa640 1 -- 172.21.15.120:0/517990996 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe52c058e70 con 0x7fe53c06cf90 2024-09-10T10:27:33.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.341+0000 7fe542b9f640 1 -- 172.21.15.120:0/517990996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe53c06c810 con 0x7fe53c06cf90 2024-09-10T10:27:33.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.341+0000 7fe5397fa640 1 -- 172.21.15.120:0/517990996 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+88 (secure 0 0 0) 0x7fe52c058770 con 0x7fe53c06cf90 2024-09-10T10:27:33.344 INFO:teuthology.orchestra.run.smithi120.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T10:27:33.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.343+0000 7fe542b9f640 1 -- 172.21.15.120:0/517990996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe52806be60 msgr2=0x7fe52806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:33.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.343+0000 7fe542b9f640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe52806be60 0x7fe52806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe53c146a30 tx=0x7fe530008040 comp rx=0 tx=0).stop 2024-09-10T10:27:33.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.343+0000 7fe542b9f640 1 -- 172.21.15.120:0/517990996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c06cf90 msgr2=0x7fe53c14dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:33.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.344+0000 7fe542b9f640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c06cf90 0x7fe53c14dc60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c8c0 tx=0x7fe52c00cd80 comp rx=0 tx=0).stop 2024-09-10T10:27:33.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.344+0000 7fe542b9f640 1 -- 172.21.15.120:0/517990996 shutdown_connections 2024-09-10T10:27:33.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.344+0000 7fe542b9f640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe52806be60 0x7fe52806e320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:33.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.344+0000 7fe542b9f640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe53c0c7b30 0x7fe53c1f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:33.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.344+0000 7fe542b9f640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe53c06d370 0x7fe53c14f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:33.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.344+0000 7fe542b9f640 1 --2- 172.21.15.120:0/517990996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe53c06cf90 0x7fe53c14dc60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:33.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.344+0000 7fe542b9f640 1 -- 172.21.15.120:0/517990996 >> 172.21.15.120:0/517990996 conn(0x7fe53c067d40 msgr2=0x7fe53c1512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:33.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.344+0000 7fe542b9f640 1 -- 172.21.15.120:0/517990996 shutdown_connections 2024-09-10T10:27:33.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33.344+0000 7fe542b9f640 1 -- 172.21.15.120:0/517990996 wait complete. 2024-09-10T10:27:33.357 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T10:27:33.357 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:27:33.357 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi120.front.sepia.ceph.com 2024-09-10T10:27:33.357 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:27:33.357 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:27:33.357 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T10:27:33.357 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:27:33.357 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:27:33.357 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:27:33.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:33.358 DEBUG:teuthology.orchestra.run.smithi120:> ip addr 2024-09-10T10:27:33.373 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:27:33.373 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:26 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f0 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:27 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f1 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8e brd ff:ff:ff:ff:ff:ff 2024-09-10T10:27:33.374 INFO:teuthology.orchestra.run.smithi120.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:27:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8f brd ff:ff:ff:ff:ff:ff 2024-09-10T10:27:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: inet 172.21.15.120/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T10:27:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft 42728sec preferred_lft 42728sec 2024-09-10T10:27:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::ec4:7aff:fe8f:478f/64 scope link noprefixroute 2024-09-10T10:27:33.375 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:27:33.375 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T10:27:33.375 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:27:33.375 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T10:27:33.375 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add name ceph-brx type bridge 2024-09-10T10:27:33.375 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr flush dev ceph-brx 2024-09-10T10:27:33.375 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set ceph-brx up 2024-09-10T10:27:33.376 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T10:27:33.376 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T10:27:33.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:27:33.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:27:33.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:33.512 DEBUG:teuthology.orchestra.run.smithi120:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:27:33.536 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-09-10T10:27:33.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:33.537 DEBUG:teuthology.orchestra.run.smithi120:> route 2024-09-10T10:27:33.594 INFO:teuthology.orchestra.run.smithi120.stdout:Kernel IP routing table 2024-09-10T10:27:33.595 INFO:teuthology.orchestra.run.smithi120.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:27:33.595 INFO:teuthology.orchestra.run.smithi120.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:27:33.595 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:27:33.595 INFO:teuthology.orchestra.run.smithi120.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:27:33.595 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:27:33.595 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T10:27:33.595 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:27:33.595 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:27:33.595 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:27:33.595 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T10:27:33.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:27:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:27:33.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:33.720 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-09-10T10:27:33.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:33.736 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list-id 2024-09-10T10:27:33.791 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:27:33.791 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T10:27:33.791 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:27:33.791 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T10:27:33.791 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T10:27:33.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:27:33.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:27:33.908 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T10:27:33.908 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:27:33.908 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T10:27:33.908 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T10:27:33.908 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T10:27:33.908 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T10:27:33.908 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T10:27:33.908 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T10:27:33.908 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T10:27:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:27:34.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:27:34.020 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:27:34.020 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T10:27:34.020 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set brx.0 up 2024-09-10T10:27:34.020 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T10:27:34.020 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T10:27:34.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:27:34.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:27:34.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:27:34.096 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:27:34.111 INFO:teuthology.orchestra.run.smithi120.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:27:34.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:27:34.111 DEBUG:teuthology.orchestra.run.smithi120:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:27:34.167 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:27:34.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:34.167 DEBUG:teuthology.orchestra.run.smithi120:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:27:34.250 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:27:34.251 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:27:34.251 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-09-10T10:27:34.397 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:27:34.397 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:27:34.397 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-09-10T10:27:34.397 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.120:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-10T10:27:34.397 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:27:34.415 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:27:34.415 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:27:34.415 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-09-10T10:27:34.415 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.120:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-10T10:27:34.416 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:27:34.416 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.120:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-10T10:27:34.416 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.120:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-10T10:27:34.416 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=7d0d0830-4370-41c3-9acc-b67b07024f68" will pass to kernel 2024-09-10T10:27:34.416 INFO:teuthology.orchestra.run.smithi120.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:27:34.416 INFO:teuthology.orchestra.run.smithi120.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:27:34.416 INFO:teuthology.orchestra.run.smithi120.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:27:34.416 INFO:teuthology.orchestra.run.smithi120.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:27:34.417 INFO:teuthology.orchestra.run.smithi120.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:27:34.418 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:27:34.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:27:34.418 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:27:34.483 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:27:34.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:27:34.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:27:34.580 INFO:teuthology.orchestra.run.smithi120.stdout:client4607 2024-09-10T10:27:34.581 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:27:34.581 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:27:34.597 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:27:34.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:27:34.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:27:34.692 INFO:teuthology.orchestra.run.smithi120.stdout:client4607 2024-09-10T10:27:34.693 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:27:34.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:27:34.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:27:34.748 INFO:teuthology.orchestra.run.smithi120.stdout:client4607 2024-09-10T10:27:34.749 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7d0d0830-4370-41c3-9acc-b67b07024f68.client4607/status') 2024-09-10T10:27:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:27:34.806 INFO:teuthology.orchestra.run.smithi120.stdout:instance: client.4607 (3)172.21.15.120:0/3199639053 2024-09-10T10:27:34.806 INFO:teuthology.orchestra.run.smithi120.stdout:blocklisted: false 2024-09-10T10:27:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records in 2024-09-10T10:27:34.807 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records out 2024-09-10T10:27:34.807 INFO:teuthology.orchestra.run.smithi120.stderr:71 bytes copied, 2.2237e-05 s, 3.2 MB/s 2024-09-10T10:27:34.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:27:34.813 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-10T10:27:34.846 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-10T10:27:34.847 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-rishabh-testing-20240909.165402-squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240909.165402-squid', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T10:27:34.847 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-10T10:27:34.847 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-10T10:27:34.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.947+0000 7f1cd9386640 1 Processor -- start 2024-09-10T10:27:34.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.947+0000 7f1cd9386640 1 -- start start 2024-09-10T10:27:34.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd9386640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd40cc480 0x7f1cd415e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:34.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd9386640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd415ba20 con 0x7f1cd40cc0a0 2024-09-10T10:27:34.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd9386640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd415aea0 con 0x7f1cd415eba0 2024-09-10T10:27:34.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd9386640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd415f050 con 0x7f1cd40cc480 2024-09-10T10:27:34.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd2ffd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f1cd40cc0a0 0x7f1cd40d0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:33580/0 (socket says 172.21.15.120:33580) 2024-09-10T10:27:34.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd2ffd640 1 -- 172.21.15.120:0/3720976322 learned_addr learned my addr 172.21.15.120:0/3720976322 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:34.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cca7fc640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd40cc480 0x7f1cd415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:34.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2753715325 0 0) 0x7f1cd415ba20 con 0x7f1cd40cc0a0 2024-09-10T10:27:34.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cc0003520 con 0x7f1cd40cc0a0 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 603197694 0 0) 0x7f1cc0003520 con 0x7f1cd40cc0a0 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd40cc480 msgr2=0x7f1cd415e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd27fc640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd40cc480 0x7f1cd415e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 >> v1:172.21.15.134:6789/0 conn(0x7f1cd415eba0 legacy=0x7f1cd4165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd415b180 con 0x7f1cd40cc0a0 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f1cc4001670 con 0x7f1cd40cc0a0 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.948+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 >> v1:172.21.15.120:6789/0 conn(0x7f1cd40cc0a0 legacy=0x7f1cd40d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cd27fc640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd40cc480 0x7f1cc00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cd27fc640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 0x7f1cc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cd27fc640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc0004d40 0x7f1cc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd415f050 con 0x7f1cc0003f20 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc000ba20 con 0x7f1cc0004d40 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc000bc20 con 0x7f1cd40cc480 2024-09-10T10:27:34.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cca7fc640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 0x7f1cc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cd2ffd640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd40cc480 0x7f1cc00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cd37fe640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc0004d40 0x7f1cc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cca7fc640 1 -- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd40cc480 msgr2=0x7f1cc00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cca7fc640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd40cc480 0x7f1cc00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cca7fc640 1 -- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc0004d40 msgr2=0x7f1cc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cca7fc640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc0004d40 0x7f1cc00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cca7fc640 1 -- 172.21.15.120:0/3720976322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc0003520 con 0x7f1cc0003f20 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.949+0000 7f1cd37fe640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc0004d40 0x7f1cc00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cca7fc640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 0x7f1cc0004800 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009a90 tx=0x7f1cc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e0cf3cbd0a31db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc036070 con 0x7f1cc0003f20 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cbc031400 con 0x7f1cc0003f20 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cd27fc640 1 -- 172.21.15.120:0/3720976322 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc009690 con 0x7f1cc0003f20 2024-09-10T10:27:34.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cd9386640 1 -- 172.21.15.120:0/3720976322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 msgr2=0x7f1cc0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:34.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cd9386640 1 --2- 172.21.15.120:0/3720976322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 0x7f1cc0004800 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009a90 tx=0x7f1cc0005830 comp rx=0 tx=0).stop 2024-09-10T10:27:34.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cd37fe640 1 -- 172.21.15.120:0/3720976322 reap_dead start 2024-09-10T10:27:34.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cd9386640 1 -- 172.21.15.120:0/3720976322 shutdown_connections 2024-09-10T10:27:34.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cd9386640 1 -- 172.21.15.120:0/3720976322 >> 172.21.15.120:0/3720976322 conn(0x7f1cd406d560 msgr2=0x7f1cd4073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:34.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cd9386640 1 -- 172.21.15.120:0/3720976322 shutdown_connections 2024-09-10T10:27:34.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.950+0000 7f1cd9386640 1 -- 172.21.15.120:0/3720976322 wait complete. 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.952+0000 7f1cd9386640 1 Processor -- start 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.952+0000 7f1cd9386640 1 -- start start 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.952+0000 7f1cd9386640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 0x7f1cd40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.952+0000 7f1cd9386640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd40ceb60 0x7f1cd40b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.952+0000 7f1cd9386640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd415c340 0x7f1cd4151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.952+0000 7f1cd9386640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd415b740 con 0x7f1cc0003f20 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.952+0000 7f1cd9386640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd415e1d0 con 0x7f1cd40ceb60 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.952+0000 7f1cd9386640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd40cc8a0 con 0x7f1cd415c340 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.952+0000 7f1cd2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 0x7f1cd40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 0x7f1cd40cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38208/0 (socket says 172.21.15.120:38208) 2024-09-10T10:27:34.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd2ffd640 1 -- 172.21.15.120:0/854878330 learned_addr learned my addr 172.21.15.120:0/854878330 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd37fe640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd415c340 0x7f1cd4151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cca7fc640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd40ceb60 0x7f1cd40b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd2ffd640 1 -- 172.21.15.120:0/854878330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd415c340 msgr2=0x7f1cd4151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd2ffd640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd415c340 0x7f1cd4151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd2ffd640 1 -- 172.21.15.120:0/854878330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd40ceb60 msgr2=0x7f1cd40b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd2ffd640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd40ceb60 0x7f1cd40b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd2ffd640 1 -- 172.21.15.120:0/854878330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd41f12a0 con 0x7f1cc0003f20 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd37fe640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd415c340 0x7f1cd4151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cca7fc640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd40ceb60 0x7f1cd40b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd2ffd640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 0x7f1cd40cfb10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4000f60 tx=0x7f1cc40077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1ccbfff640 1 -- 172.21.15.120:0/854878330 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400f9c0 con 0x7f1cc0003f20 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1ccbfff640 1 -- 172.21.15.120:0/854878330 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc4018070 con 0x7f1cc0003f20 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd9386640 1 -- 172.21.15.120:0/854878330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd41f1580 con 0x7f1cc0003f20 2024-09-10T10:27:34.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1ccbfff640 1 -- 172.21.15.120:0/854878330 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400fde0 con 0x7f1cc0003f20 2024-09-10T10:27:34.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.953+0000 7f1cd9386640 1 -- 172.21.15.120:0/854878330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd40ce640 con 0x7f1cc0003f20 2024-09-10T10:27:34.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.954+0000 7f1cd9386640 1 -- 172.21.15.120:0/854878330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd40b5b10 con 0x7f1cc0003f20 2024-09-10T10:27:34.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.955+0000 7f1ccbfff640 1 -- 172.21.15.120:0/854878330 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1cc401d460 con 0x7f1cc0003f20 2024-09-10T10:27:34.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.955+0000 7f1ccbfff640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1cb406be60 0x7f1cb406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:34.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.955+0000 7f1ccbfff640 1 -- 172.21.15.120:0/854878330 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1cc408b3d0 con 0x7f1cc0003f20 2024-09-10T10:27:34.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.955+0000 7f1cca7fc640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1cb406be60 0x7f1cb406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:34.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.956+0000 7f1cca7fc640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1cb406be60 0x7f1cb406e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc033000 tx=0x7f1cbc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:34.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:34.959+0000 7f1ccbfff640 1 -- 172.21.15.120:0/854878330 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1cc4059980 con 0x7f1cc0003f20 2024-09-10T10:27:35.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.127+0000 7f1cd9386640 1 -- 172.21.15.120:0/854878330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}] v 0) -- 0x7f1cd40cbde0 con 0x7f1cc0003f20 2024-09-10T10:27:35.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.130+0000 7f1ccbfff640 1 -- 172.21.15.120:0/854878330 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}]=0 v10)=0 v10) ==== 136+0+0 (secure 0 0 0) 0x7f1cc4059280 con 0x7f1cc0003f20 2024-09-10T10:27:35.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 -- 172.21.15.120:0/854878330 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1cb406be60 msgr2=0x7f1cb406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1cb406be60 0x7f1cb406e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc033000 tx=0x7f1cbc008030 comp rx=0 tx=0).stop 2024-09-10T10:27:35.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 -- 172.21.15.120:0/854878330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 msgr2=0x7f1cd40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 0x7f1cd40cfb10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4000f60 tx=0x7f1cc40077a0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 -- 172.21.15.120:0/854878330 shutdown_connections 2024-09-10T10:27:35.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1cb406be60 0x7f1cb406e320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd415c340 0x7f1cd4151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd40ceb60 0x7f1cd40b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 --2- 172.21.15.120:0/854878330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cc0003f20 0x7f1cd40cfb10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 -- 172.21.15.120:0/854878330 >> 172.21.15.120:0/854878330 conn(0x7f1cd406d560 msgr2=0x7f1cd41501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:35.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 -- 172.21.15.120:0/854878330 shutdown_connections 2024-09-10T10:27:35.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.133+0000 7f1cd9386640 1 -- 172.21.15.120:0/854878330 wait complete. 2024-09-10T10:27:35.146 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-10T10:27:35.147 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:27:35.147 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:27:35.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.243+0000 7f6990bfd640 1 Processor -- start 2024-09-10T10:27:35.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.243+0000 7f6990bfd640 1 -- start start 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6990bfd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f698c152c30 0x7f698c0ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6990bfd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f698c0ca7d0 0x7f698c06f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6990bfd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698c153010 con 0x7f698c0c3930 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6990bfd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f698c0741c0 con 0x7f698c0ca7d0 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6990bfd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f698c15cbc0 con 0x7f698c152c30 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6983fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f698c0ca7d0 0x7f698c06f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6983fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f698c0ca7d0 0x7f698c06f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38220/0 (socket says 172.21.15.120:38220) 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6983fff640 1 -- 172.21.15.120:0/3744956453 learned_addr learned my addr 172.21.15.120:0/3744956453 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f698a59c640 1 --2- 172.21.15.120:0/3744956453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f698c152c30 0x7f698c0ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6983fff640 1 -- 172.21.15.120:0/3744956453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f698c152c30 msgr2=0x7f698c0ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6983fff640 1 --2- 172.21.15.120:0/3744956453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f698c152c30 0x7f698c0ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6983fff640 1 -- 172.21.15.120:0/3744956453 >> v1:172.21.15.120:6789/0 conn(0x7f698c0c3930 legacy=0x7f698c158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6989d9b640 1 -- 172.21.15.120:0/3744956453 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4255639862 0 0) 0x7f698c153010 con 0x7f698c0c3930 2024-09-10T10:27:35.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.244+0000 7f6983fff640 1 -- 172.21.15.120:0/3744956453 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c0cacd0 con 0x7f698c0ca7d0 2024-09-10T10:27:35.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.245+0000 7f6983fff640 1 --2- 172.21.15.120:0/3744956453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c0ca7d0 0x7f698c06f670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6974009870 tx=0x7f697402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ea370d85e18229 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:35.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.245+0000 7f6989d9b640 1 -- 172.21.15.120:0/3744956453 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974037070 con 0x7f698c0ca7d0 2024-09-10T10:27:35.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.245+0000 7f6989d9b640 1 -- 172.21.15.120:0/3744956453 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697402ae50 con 0x7f698c0ca7d0 2024-09-10T10:27:35.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.245+0000 7f6989d9b640 1 -- 172.21.15.120:0/3744956453 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974032860 con 0x7f698c0ca7d0 2024-09-10T10:27:35.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.246+0000 7f6990bfd640 1 -- 172.21.15.120:0/3744956453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c0ca7d0 msgr2=0x7f698c06f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.246+0000 7f6990bfd640 1 --2- 172.21.15.120:0/3744956453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c0ca7d0 0x7f698c06f670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6974009870 tx=0x7f697402a300 comp rx=0 tx=0).stop 2024-09-10T10:27:35.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.246+0000 7f6990bfd640 1 -- 172.21.15.120:0/3744956453 shutdown_connections 2024-09-10T10:27:35.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.246+0000 7f6990bfd640 1 --2- 172.21.15.120:0/3744956453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c0ca7d0 0x7f698c06f670 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.246+0000 7f6990bfd640 1 --2- 172.21.15.120:0/3744956453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f698c152c30 0x7f698c0ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.246+0000 7f6990bfd640 1 -- 172.21.15.120:0/3744956453 >> 172.21.15.120:0/3744956453 conn(0x7f698c06bc10 msgr2=0x7f698c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:35.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.246+0000 7f6990bfd640 1 -- 172.21.15.120:0/3744956453 shutdown_connections 2024-09-10T10:27:35.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.246+0000 7f6990bfd640 1 -- 172.21.15.120:0/3744956453 wait complete. 2024-09-10T10:27:35.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.247+0000 7f6990bfd640 1 Processor -- start 2024-09-10T10:27:35.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.247+0000 7f6990bfd640 1 -- start start 2024-09-10T10:27:35.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f6990bfd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f698c0c3930 0x7f698c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f6990bfd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698c0ca7d0 0x7f698c1f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f6990bfd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c152c30 0x7f698c149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f6990bfd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f698c15c570 con 0x7f698c152c30 2024-09-10T10:27:35.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f6990bfd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f698c15c290 con 0x7f698c0ca7d0 2024-09-10T10:27:35.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f6990bfd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f698c15c850 con 0x7f698c0c3930 2024-09-10T10:27:35.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c152c30 0x7f698c149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c152c30 0x7f698c149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38228/0 (socket says 172.21.15.120:38228) 2024-09-10T10:27:35.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f698ad9d640 1 -- 172.21.15.120:0/3788640401 learned_addr learned my addr 172.21.15.120:0/3788640401 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f698a59c640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f698c0c3930 0x7f698c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f6983fff640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698c0ca7d0 0x7f698c1f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f698ad9d640 1 -- 172.21.15.120:0/3788640401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f698c0c3930 msgr2=0x7f698c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f698ad9d640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f698c0c3930 0x7f698c15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f698ad9d640 1 -- 172.21.15.120:0/3788640401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698c0ca7d0 msgr2=0x7f698c1f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f698ad9d640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698c0ca7d0 0x7f698c1f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f698ad9d640 1 -- 172.21.15.120:0/3788640401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c1ee580 con 0x7f698c152c30 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f698ad9d640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c152c30 0x7f698c149740 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f698400c7e0 tx=0x7f698400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.248+0000 7f6982ffd640 1 -- 172.21.15.120:0/3788640401 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984017070 con 0x7f698c152c30 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.249+0000 7f6982ffd640 1 -- 172.21.15.120:0/3788640401 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6984003dc0 con 0x7f698c152c30 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.249+0000 7f6990bfd640 1 -- 172.21.15.120:0/3788640401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c15b9a0 con 0x7f698c152c30 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.249+0000 7f6982ffd640 1 -- 172.21.15.120:0/3788640401 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69840052b0 con 0x7f698c152c30 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.249+0000 7f6990bfd640 1 -- 172.21.15.120:0/3788640401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c1eeb00 con 0x7f698c152c30 2024-09-10T10:27:35.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.250+0000 7f6990bfd640 1 -- 172.21.15.120:0/3788640401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c0ca290 con 0x7f698c152c30 2024-09-10T10:27:35.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.250+0000 7f6982ffd640 1 -- 172.21.15.120:0/3788640401 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f698401a070 con 0x7f698c152c30 2024-09-10T10:27:35.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.251+0000 7f6982ffd640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f696c06be30 0x7f696c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.251+0000 7f698a59c640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f696c06be30 0x7f696c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.251+0000 7f6982ffd640 1 -- 172.21.15.120:0/3788640401 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f698408a900 con 0x7f698c152c30 2024-09-10T10:27:35.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.251+0000 7f698a59c640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f696c06be30 0x7f696c06e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f697c0079a0 tx=0x7f697c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:35.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.255+0000 7f6982ffd640 1 -- 172.21.15.120:0/3788640401 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f69840587d0 con 0x7f698c152c30 2024-09-10T10:27:35.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.417+0000 7f6990bfd640 1 -- 172.21.15.120:0/3788640401 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f698c156340 con 0x7f696c06be30 2024-09-10T10:27:35.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.425+0000 7f6982ffd640 1 -- 172.21.15.120:0/3788640401 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332807 (secure 0 0 0) 0x7f698c156340 con 0x7f696c06be30 2024-09-10T10:27:35.430 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:27:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.428+0000 7f6990bfd640 1 -- 172.21.15.120:0/3788640401 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f696c06be30 msgr2=0x7f696c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f696c06be30 0x7f696c06e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f697c0079a0 tx=0x7f697c008040 comp rx=0 tx=0).stop 2024-09-10T10:27:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 -- 172.21.15.120:0/3788640401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c152c30 msgr2=0x7f698c149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c152c30 0x7f698c149740 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f698400c7e0 tx=0x7f698400cca0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 -- 172.21.15.120:0/3788640401 shutdown_connections 2024-09-10T10:27:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f696c06be30 0x7f696c06e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c152c30 0x7f698c149740 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698c0ca7d0 0x7f698c1f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 --2- 172.21.15.120:0/3788640401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f698c0c3930 0x7f698c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 -- 172.21.15.120:0/3788640401 >> 172.21.15.120:0/3788640401 conn(0x7f698c06bc10 msgr2=0x7f698c150450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 -- 172.21.15.120:0/3788640401 shutdown_connections 2024-09-10T10:27:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.429+0000 7f6990bfd640 1 -- 172.21.15.120:0/3788640401 wait complete. 2024-09-10T10:27:35.442 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-09-10T10:27:34.101968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600634,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":366,"num_write_kb":2482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218944,"kb_used_data":2752,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755776,"statfs":{"total":773094113280,"available":772869914624,"internally_reserved":0,"allocated":2818048,"data_stored":2210748,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2812,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.156632"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":41,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.948583+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:27:25.948583+0000","last_peered":"2024-09-10T10:27:25.948583+0000","last_clean":"2024-09-10T10:27:25.948583+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:27:25.948583+0000","last_undegraded":"2024-09-10T10:27:25.948583+0000","last_fullsized":"2024-09-10T10:27:25.948583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949805+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:27:23.949805+0000","last_peered":"2024-09-10T10:27:23.949805+0000","last_clean":"2024-09-10T10:27:23.949805+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:27:23.949805+0000","last_undegraded":"2024-09-10T10:27:23.949805+0000","last_fullsized":"2024-09-10T10:27:23.949805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948483+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:27:23.948483+0000","last_peered":"2024-09-10T10:27:23.948483+0000","last_clean":"2024-09-10T10:27:23.948483+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:27:23.948483+0000","last_undegraded":"2024-09-10T10:27:23.948483+0000","last_fullsized":"2024-09-10T10:27:23.948483+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949967+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:27:23.949967+0000","last_peered":"2024-09-10T10:27:23.949967+0000","last_clean":"2024-09-10T10:27:23.949967+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:27:23.949967+0000","last_undegraded":"2024-09-10T10:27:23.949967+0000","last_fullsized":"2024-09-10T10:27:23.949967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947811+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:27:23.947811+0000","last_peered":"2024-09-10T10:27:23.947811+0000","last_clean":"2024-09-10T10:27:23.947811+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:27:23.947811+0000","last_undegraded":"2024-09-10T10:27:23.947811+0000","last_fullsized":"2024-09-10T10:27:23.947811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947563+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:27:23.947563+0000","last_peered":"2024-09-10T10:27:23.947563+0000","last_clean":"2024-09-10T10:27:23.947563+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:27:23.947563+0000","last_undegraded":"2024-09-10T10:27:23.947563+0000","last_fullsized":"2024-09-10T10:27:23.947563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"28'11","reported_seq":39,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.948235+0000","last_change":"2024-09-10T10:26:40.796337+0000","last_active":"2024-09-10T10:27:25.948235+0000","last_peered":"2024-09-10T10:27:25.948235+0000","last_clean":"2024-09-10T10:27:25.948235+0000","last_became_active":"2024-09-10T10:26:40.796029+0000","last_became_peered":"2024-09-10T10:26:40.796029+0000","last_unstale":"2024-09-10T10:27:25.948235+0000","last_undegraded":"2024-09-10T10:27:25.948235+0000","last_fullsized":"2024-09-10T10:27:25.948235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949057+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:27:23.949057+0000","last_peered":"2024-09-10T10:27:23.949057+0000","last_clean":"2024-09-10T10:27:23.949057+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:27:23.949057+0000","last_undegraded":"2024-09-10T10:27:23.949057+0000","last_fullsized":"2024-09-10T10:27:23.949057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.955254+0000","last_change":"2024-09-10T10:27:23.955254+0000","last_active":"2024-09-10T10:27:23.955254+0000","last_peered":"2024-09-10T10:27:23.955254+0000","last_clean":"2024-09-10T10:27:23.955254+0000","last_became_active":"2024-09-10T10:27:23.954737+0000","last_became_peered":"2024-09-10T10:27:23.954737+0000","last_unstale":"2024-09-10T10:27:23.955254+0000","last_undegraded":"2024-09-10T10:27:23.955254+0000","last_fullsized":"2024-09-10T10:27:23.955254+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.946861+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:27:25.946861+0000","last_peered":"2024-09-10T10:27:25.946861+0000","last_clean":"2024-09-10T10:27:25.946861+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:27:25.946861+0000","last_undegraded":"2024-09-10T10:27:25.946861+0000","last_fullsized":"2024-09-10T10:27:25.946861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.957643+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:27:23.957643+0000","last_peered":"2024-09-10T10:27:23.957643+0000","last_clean":"2024-09-10T10:27:23.957643+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:27:23.957643+0000","last_undegraded":"2024-09-10T10:27:23.957643+0000","last_fullsized":"2024-09-10T10:27:23.957643+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947808+0000","last_change":"2024-09-10T10:26:45.816587+0000","last_active":"2024-09-10T10:27:23.947808+0000","last_peered":"2024-09-10T10:27:23.947808+0000","last_clean":"2024-09-10T10:27:23.947808+0000","last_became_active":"2024-09-10T10:26:45.816138+0000","last_became_peered":"2024-09-10T10:26:45.816138+0000","last_unstale":"2024-09-10T10:27:23.947808+0000","last_undegraded":"2024-09-10T10:27:23.947808+0000","last_fullsized":"2024-09-10T10:27:23.947808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952071+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:27:23.952071+0000","last_peered":"2024-09-10T10:27:23.952071+0000","last_clean":"2024-09-10T10:27:23.952071+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:27:23.952071+0000","last_undegraded":"2024-09-10T10:27:23.952071+0000","last_fullsized":"2024-09-10T10:27:23.952071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086759+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:27:24.086759+0000","last_peered":"2024-09-10T10:27:24.086759+0000","last_clean":"2024-09-10T10:27:24.086759+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:27:24.086759+0000","last_undegraded":"2024-09-10T10:27:24.086759+0000","last_fullsized":"2024-09-10T10:27:24.086759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.955276+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:27:23.955276+0000","last_peered":"2024-09-10T10:27:23.955276+0000","last_clean":"2024-09-10T10:27:23.955276+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:27:23.955276+0000","last_undegraded":"2024-09-10T10:27:23.955276+0000","last_fullsized":"2024-09-10T10:27:23.955276+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949460+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:27:23.949460+0000","last_peered":"2024-09-10T10:27:23.949460+0000","last_clean":"2024-09-10T10:27:23.949460+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:27:23.949460+0000","last_undegraded":"2024-09-10T10:27:23.949460+0000","last_fullsized":"2024-09-10T10:27:23.949460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'3","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949216+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:27:23.949216+0000","last_peered":"2024-09-10T10:27:23.949216+0000","last_clean":"2024-09-10T10:27:23.949216+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:27:23.949216+0000","last_undegraded":"2024-09-10T10:27:23.949216+0000","last_fullsized":"2024-09-10T10:27:23.949216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947768+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:27:23.947768+0000","last_peered":"2024-09-10T10:27:23.947768+0000","last_clean":"2024-09-10T10:27:23.947768+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:27:23.947768+0000","last_undegraded":"2024-09-10T10:27:23.947768+0000","last_fullsized":"2024-09-10T10:27:23.947768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949368+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:27:23.949368+0000","last_peered":"2024-09-10T10:27:23.949368+0000","last_clean":"2024-09-10T10:27:23.949368+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:27:23.949368+0000","last_undegraded":"2024-09-10T10:27:23.949368+0000","last_fullsized":"2024-09-10T10:27:23.949368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":35,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.946351+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:27:25.946351+0000","last_peered":"2024-09-10T10:27:25.946351+0000","last_clean":"2024-09-10T10:27:25.946351+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:27:25.946351+0000","last_undegraded":"2024-09-10T10:27:25.946351+0000","last_fullsized":"2024-09-10T10:27:25.946351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949704+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:27:23.949704+0000","last_peered":"2024-09-10T10:27:23.949704+0000","last_clean":"2024-09-10T10:27:23.949704+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:27:23.949704+0000","last_undegraded":"2024-09-10T10:27:23.949704+0000","last_fullsized":"2024-09-10T10:27:23.949704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948893+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:27:23.948893+0000","last_peered":"2024-09-10T10:27:23.948893+0000","last_clean":"2024-09-10T10:27:23.948893+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:27:23.948893+0000","last_undegraded":"2024-09-10T10:27:23.948893+0000","last_fullsized":"2024-09-10T10:27:23.948893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086339+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:27:24.086339+0000","last_peered":"2024-09-10T10:27:24.086339+0000","last_clean":"2024-09-10T10:27:24.086339+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:27:24.086339+0000","last_undegraded":"2024-09-10T10:27:24.086339+0000","last_fullsized":"2024-09-10T10:27:24.086339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947906+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:27:23.947906+0000","last_peered":"2024-09-10T10:27:23.947906+0000","last_clean":"2024-09-10T10:27:23.947906+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:27:23.947906+0000","last_undegraded":"2024-09-10T10:27:23.947906+0000","last_fullsized":"2024-09-10T10:27:23.947906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947699+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:27:23.947699+0000","last_peered":"2024-09-10T10:27:23.947699+0000","last_clean":"2024-09-10T10:27:23.947699+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:27:23.947699+0000","last_undegraded":"2024-09-10T10:27:23.947699+0000","last_fullsized":"2024-09-10T10:27:23.947699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'7","reported_seq":35,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.945274+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:27:25.945274+0000","last_peered":"2024-09-10T10:27:25.945274+0000","last_clean":"2024-09-10T10:27:25.945274+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:27:25.945274+0000","last_undegraded":"2024-09-10T10:27:25.945274+0000","last_fullsized":"2024-09-10T10:27:25.945274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947985+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:27:23.947985+0000","last_peered":"2024-09-10T10:27:23.947985+0000","last_clean":"2024-09-10T10:27:23.947985+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:27:23.947985+0000","last_undegraded":"2024-09-10T10:27:23.947985+0000","last_fullsized":"2024-09-10T10:27:23.947985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950344+0000","last_change":"2024-09-10T10:26:41.795085+0000","last_active":"2024-09-10T10:27:23.950344+0000","last_peered":"2024-09-10T10:27:23.950344+0000","last_clean":"2024-09-10T10:27:23.950344+0000","last_became_active":"2024-09-10T10:26:41.794819+0000","last_became_peered":"2024-09-10T10:26:41.794819+0000","last_unstale":"2024-09-10T10:27:23.950344+0000","last_undegraded":"2024-09-10T10:27:23.950344+0000","last_fullsized":"2024-09-10T10:27:23.950344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'7","reported_seq":35,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950702+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:27:23.950702+0000","last_peered":"2024-09-10T10:27:23.950702+0000","last_clean":"2024-09-10T10:27:23.950702+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:27:23.950702+0000","last_undegraded":"2024-09-10T10:27:23.950702+0000","last_fullsized":"2024-09-10T10:27:23.950702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948433+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:27:23.948433+0000","last_peered":"2024-09-10T10:27:23.948433+0000","last_clean":"2024-09-10T10:27:23.948433+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:27:23.948433+0000","last_undegraded":"2024-09-10T10:27:23.948433+0000","last_fullsized":"2024-09-10T10:27:23.948433+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949581+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:27:23.949581+0000","last_peered":"2024-09-10T10:27:23.949581+0000","last_clean":"2024-09-10T10:27:23.949581+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:27:23.949581+0000","last_undegraded":"2024-09-10T10:27:23.949581+0000","last_fullsized":"2024-09-10T10:27:23.949581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087107+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:27:24.087107+0000","last_peered":"2024-09-10T10:27:24.087107+0000","last_clean":"2024-09-10T10:27:24.087107+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:27:24.087107+0000","last_undegraded":"2024-09-10T10:27:24.087107+0000","last_fullsized":"2024-09-10T10:27:24.087107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947236+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:27:23.947236+0000","last_peered":"2024-09-10T10:27:23.947236+0000","last_clean":"2024-09-10T10:27:23.947236+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:27:23.947236+0000","last_undegraded":"2024-09-10T10:27:23.947236+0000","last_fullsized":"2024-09-10T10:27:23.947236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949745+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:27:23.949745+0000","last_peered":"2024-09-10T10:27:23.949745+0000","last_clean":"2024-09-10T10:27:23.949745+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:27:23.949745+0000","last_undegraded":"2024-09-10T10:27:23.949745+0000","last_fullsized":"2024-09-10T10:27:23.949745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949683+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:27:23.949683+0000","last_peered":"2024-09-10T10:27:23.949683+0000","last_clean":"2024-09-10T10:27:23.949683+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:27:23.949683+0000","last_undegraded":"2024-09-10T10:27:23.949683+0000","last_fullsized":"2024-09-10T10:27:23.949683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948820+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:27:23.948820+0000","last_peered":"2024-09-10T10:27:23.948820+0000","last_clean":"2024-09-10T10:27:23.948820+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:27:23.948820+0000","last_undegraded":"2024-09-10T10:27:23.948820+0000","last_fullsized":"2024-09-10T10:27:23.948820+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948546+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:27:23.948546+0000","last_peered":"2024-09-10T10:27:23.948546+0000","last_clean":"2024-09-10T10:27:23.948546+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:27:23.948546+0000","last_undegraded":"2024-09-10T10:27:23.948546+0000","last_fullsized":"2024-09-10T10:27:23.948546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.950835+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:27:25.950835+0000","last_peered":"2024-09-10T10:27:25.950835+0000","last_clean":"2024-09-10T10:27:25.950835+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:27:25.950835+0000","last_undegraded":"2024-09-10T10:27:25.950835+0000","last_fullsized":"2024-09-10T10:27:25.950835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949466+0000","last_change":"2024-09-10T10:26:45.814008+0000","last_active":"2024-09-10T10:27:23.949466+0000","last_peered":"2024-09-10T10:27:23.949466+0000","last_clean":"2024-09-10T10:27:23.949466+0000","last_became_active":"2024-09-10T10:26:45.813627+0000","last_became_peered":"2024-09-10T10:26:45.813627+0000","last_unstale":"2024-09-10T10:27:23.949466+0000","last_undegraded":"2024-09-10T10:27:23.949466+0000","last_fullsized":"2024-09-10T10:27:23.949466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948999+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:27:23.948999+0000","last_peered":"2024-09-10T10:27:23.948999+0000","last_clean":"2024-09-10T10:27:23.948999+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:27:23.948999+0000","last_undegraded":"2024-09-10T10:27:23.948999+0000","last_fullsized":"2024-09-10T10:27:23.948999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"28'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952325+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:27:23.952325+0000","last_peered":"2024-09-10T10:27:23.952325+0000","last_clean":"2024-09-10T10:27:23.952325+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:27:23.952325+0000","last_undegraded":"2024-09-10T10:27:23.952325+0000","last_fullsized":"2024-09-10T10:27:23.952325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949799+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:27:23.949799+0000","last_peered":"2024-09-10T10:27:23.949799+0000","last_clean":"2024-09-10T10:27:23.949799+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:27:23.949799+0000","last_undegraded":"2024-09-10T10:27:23.949799+0000","last_fullsized":"2024-09-10T10:27:23.949799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.946956+0000","last_change":"2024-09-10T10:26:41.799224+0000","last_active":"2024-09-10T10:27:23.946956+0000","last_peered":"2024-09-10T10:27:23.946956+0000","last_clean":"2024-09-10T10:27:23.946956+0000","last_became_active":"2024-09-10T10:26:41.798948+0000","last_became_peered":"2024-09-10T10:26:41.798948+0000","last_unstale":"2024-09-10T10:27:23.946956+0000","last_undegraded":"2024-09-10T10:27:23.946956+0000","last_fullsized":"2024-09-10T10:27:23.946956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086405+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:27:24.086405+0000","last_peered":"2024-09-10T10:27:24.086405+0000","last_clean":"2024-09-10T10:27:24.086405+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:27:24.086405+0000","last_undegraded":"2024-09-10T10:27:24.086405+0000","last_fullsized":"2024-09-10T10:27:24.086405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947499+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:27:23.947499+0000","last_peered":"2024-09-10T10:27:23.947499+0000","last_clean":"2024-09-10T10:27:23.947499+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:27:23.947499+0000","last_undegraded":"2024-09-10T10:27:23.947499+0000","last_fullsized":"2024-09-10T10:27:23.947499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950262+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:27:23.950262+0000","last_peered":"2024-09-10T10:27:23.950262+0000","last_clean":"2024-09-10T10:27:23.950262+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:27:23.950262+0000","last_undegraded":"2024-09-10T10:27:23.950262+0000","last_fullsized":"2024-09-10T10:27:23.950262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949493+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:27:23.949493+0000","last_peered":"2024-09-10T10:27:23.949493+0000","last_clean":"2024-09-10T10:27:23.949493+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:27:23.949493+0000","last_undegraded":"2024-09-10T10:27:23.949493+0000","last_fullsized":"2024-09-10T10:27:23.949493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":184,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086475+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:27:24.086475+0000","last_peered":"2024-09-10T10:27:24.086475+0000","last_clean":"2024-09-10T10:27:24.086475+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:27:24.086475+0000","last_undegraded":"2024-09-10T10:27:24.086475+0000","last_fullsized":"2024-09-10T10:27:24.086475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952629+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:27:23.952629+0000","last_peered":"2024-09-10T10:27:23.952629+0000","last_clean":"2024-09-10T10:27:23.952629+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:27:23.952629+0000","last_undegraded":"2024-09-10T10:27:23.952629+0000","last_fullsized":"2024-09-10T10:27:23.952629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086749+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:27:24.086749+0000","last_peered":"2024-09-10T10:27:24.086749+0000","last_clean":"2024-09-10T10:27:24.086749+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:27:24.086749+0000","last_undegraded":"2024-09-10T10:27:24.086749+0000","last_fullsized":"2024-09-10T10:27:24.086749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952500+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:27:23.952500+0000","last_peered":"2024-09-10T10:27:23.952500+0000","last_clean":"2024-09-10T10:27:23.952500+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:27:23.952500+0000","last_undegraded":"2024-09-10T10:27:23.952500+0000","last_fullsized":"2024-09-10T10:27:23.952500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952228+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:27:23.952228+0000","last_peered":"2024-09-10T10:27:23.952228+0000","last_clean":"2024-09-10T10:27:23.952228+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:27:23.952228+0000","last_undegraded":"2024-09-10T10:27:23.952228+0000","last_fullsized":"2024-09-10T10:27:23.952228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949116+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:27:23.949116+0000","last_peered":"2024-09-10T10:27:23.949116+0000","last_clean":"2024-09-10T10:27:23.949116+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:27:23.949116+0000","last_undegraded":"2024-09-10T10:27:23.949116+0000","last_fullsized":"2024-09-10T10:27:23.949116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950498+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:27:23.950498+0000","last_peered":"2024-09-10T10:27:23.950498+0000","last_clean":"2024-09-10T10:27:23.950498+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:27:23.950498+0000","last_undegraded":"2024-09-10T10:27:23.950498+0000","last_fullsized":"2024-09-10T10:27:23.950498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947427+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:27:23.947427+0000","last_peered":"2024-09-10T10:27:23.947427+0000","last_clean":"2024-09-10T10:27:23.947427+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:27:23.947427+0000","last_undegraded":"2024-09-10T10:27:23.947427+0000","last_fullsized":"2024-09-10T10:27:23.947427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087253+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:27:24.087253+0000","last_peered":"2024-09-10T10:27:24.087253+0000","last_clean":"2024-09-10T10:27:24.087253+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:27:24.087253+0000","last_undegraded":"2024-09-10T10:27:24.087253+0000","last_fullsized":"2024-09-10T10:27:24.087253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.944217+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:27:25.944217+0000","last_peered":"2024-09-10T10:27:25.944217+0000","last_clean":"2024-09-10T10:27:25.944217+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:27:25.944217+0000","last_undegraded":"2024-09-10T10:27:25.944217+0000","last_fullsized":"2024-09-10T10:27:25.944217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947488+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:27:23.947488+0000","last_peered":"2024-09-10T10:27:23.947488+0000","last_clean":"2024-09-10T10:27:23.947488+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:27:23.947488+0000","last_undegraded":"2024-09-10T10:27:23.947488+0000","last_fullsized":"2024-09-10T10:27:23.947488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087016+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:27:24.087016+0000","last_peered":"2024-09-10T10:27:24.087016+0000","last_clean":"2024-09-10T10:27:24.087016+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:27:24.087016+0000","last_undegraded":"2024-09-10T10:27:24.087016+0000","last_fullsized":"2024-09-10T10:27:24.087016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948188+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:27:23.948188+0000","last_peered":"2024-09-10T10:27:23.948188+0000","last_clean":"2024-09-10T10:27:23.948188+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:27:23.948188+0000","last_undegraded":"2024-09-10T10:27:23.948188+0000","last_fullsized":"2024-09-10T10:27:23.948188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947786+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:27:23.947786+0000","last_peered":"2024-09-10T10:27:23.947786+0000","last_clean":"2024-09-10T10:27:23.947786+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:27:23.947786+0000","last_undegraded":"2024-09-10T10:27:23.947786+0000","last_fullsized":"2024-09-10T10:27:23.947786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086641+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:27:24.086641+0000","last_peered":"2024-09-10T10:27:24.086641+0000","last_clean":"2024-09-10T10:27:24.086641+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:27:24.086641+0000","last_undegraded":"2024-09-10T10:27:24.086641+0000","last_fullsized":"2024-09-10T10:27:24.086641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"28'17","reported_seq":45,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952559+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:27:23.952559+0000","last_peered":"2024-09-10T10:27:23.952559+0000","last_clean":"2024-09-10T10:27:23.952559+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:27:23.952559+0000","last_undegraded":"2024-09-10T10:27:23.952559+0000","last_fullsized":"2024-09-10T10:27:23.952559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947405+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:27:23.947405+0000","last_peered":"2024-09-10T10:27:23.947405+0000","last_clean":"2024-09-10T10:27:23.947405+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:27:23.947405+0000","last_undegraded":"2024-09-10T10:27:23.947405+0000","last_fullsized":"2024-09-10T10:27:23.947405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952447+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:27:23.952447+0000","last_peered":"2024-09-10T10:27:23.952447+0000","last_clean":"2024-09-10T10:27:23.952447+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:27:23.952447+0000","last_undegraded":"2024-09-10T10:27:23.952447+0000","last_fullsized":"2024-09-10T10:27:23.952447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952119+0000","last_change":"2024-09-10T10:26:40.794739+0000","last_active":"2024-09-10T10:27:23.952119+0000","last_peered":"2024-09-10T10:27:23.952119+0000","last_clean":"2024-09-10T10:27:23.952119+0000","last_became_active":"2024-09-10T10:26:40.794430+0000","last_became_peered":"2024-09-10T10:26:40.794430+0000","last_unstale":"2024-09-10T10:27:23.952119+0000","last_undegraded":"2024-09-10T10:27:23.952119+0000","last_fullsized":"2024-09-10T10:27:23.952119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950206+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:27:23.950206+0000","last_peered":"2024-09-10T10:27:23.950206+0000","last_clean":"2024-09-10T10:27:23.950206+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:27:23.950206+0000","last_undegraded":"2024-09-10T10:27:23.950206+0000","last_fullsized":"2024-09-10T10:27:23.950206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086713+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:27:24.086713+0000","last_peered":"2024-09-10T10:27:24.086713+0000","last_clean":"2024-09-10T10:27:24.086713+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:27:24.086713+0000","last_undegraded":"2024-09-10T10:27:24.086713+0000","last_fullsized":"2024-09-10T10:27:24.086713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"28'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948645+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:27:23.948645+0000","last_peered":"2024-09-10T10:27:23.948645+0000","last_clean":"2024-09-10T10:27:23.948645+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:27:23.948645+0000","last_undegraded":"2024-09-10T10:27:23.948645+0000","last_fullsized":"2024-09-10T10:27:23.948645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949263+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:27:23.949263+0000","last_peered":"2024-09-10T10:27:23.949263+0000","last_clean":"2024-09-10T10:27:23.949263+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:27:23.949263+0000","last_undegraded":"2024-09-10T10:27:23.949263+0000","last_fullsized":"2024-09-10T10:27:23.949263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947085+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:27:23.947085+0000","last_peered":"2024-09-10T10:27:23.947085+0000","last_clean":"2024-09-10T10:27:23.947085+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:27:23.947085+0000","last_undegraded":"2024-09-10T10:27:23.947085+0000","last_fullsized":"2024-09-10T10:27:23.947085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":37,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.944224+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:27:25.944224+0000","last_peered":"2024-09-10T10:27:25.944224+0000","last_clean":"2024-09-10T10:27:25.944224+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:27:25.944224+0000","last_undegraded":"2024-09-10T10:27:25.944224+0000","last_fullsized":"2024-09-10T10:27:25.944224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949572+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:27:23.949572+0000","last_peered":"2024-09-10T10:27:23.949572+0000","last_clean":"2024-09-10T10:27:23.949572+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:27:23.949572+0000","last_undegraded":"2024-09-10T10:27:23.949572+0000","last_fullsized":"2024-09-10T10:27:23.949572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086767+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:27:24.086767+0000","last_peered":"2024-09-10T10:27:24.086767+0000","last_clean":"2024-09-10T10:27:24.086767+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:27:24.086767+0000","last_undegraded":"2024-09-10T10:27:24.086767+0000","last_fullsized":"2024-09-10T10:27:24.086767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"28'13","reported_seq":41,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.946768+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:27:25.946768+0000","last_peered":"2024-09-10T10:27:25.946768+0000","last_clean":"2024-09-10T10:27:25.946768+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:27:25.946768+0000","last_undegraded":"2024-09-10T10:27:25.946768+0000","last_fullsized":"2024-09-10T10:27:25.946768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948966+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:27:23.948966+0000","last_peered":"2024-09-10T10:27:23.948966+0000","last_clean":"2024-09-10T10:27:23.948966+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:27:23.948966+0000","last_undegraded":"2024-09-10T10:27:23.948966+0000","last_fullsized":"2024-09-10T10:27:23.948966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948445+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:27:23.948445+0000","last_peered":"2024-09-10T10:27:23.948445+0000","last_clean":"2024-09-10T10:27:23.948445+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:27:23.948445+0000","last_undegraded":"2024-09-10T10:27:23.948445+0000","last_fullsized":"2024-09-10T10:27:23.948445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950348+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:27:23.950348+0000","last_peered":"2024-09-10T10:27:23.950348+0000","last_clean":"2024-09-10T10:27:23.950348+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:27:23.950348+0000","last_undegraded":"2024-09-10T10:27:23.950348+0000","last_fullsized":"2024-09-10T10:27:23.950348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948304+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:27:23.948304+0000","last_peered":"2024-09-10T10:27:23.948304+0000","last_clean":"2024-09-10T10:27:23.948304+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:27:23.948304+0000","last_undegraded":"2024-09-10T10:27:23.948304+0000","last_fullsized":"2024-09-10T10:27:23.948304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947912+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:27:23.947912+0000","last_peered":"2024-09-10T10:27:23.947912+0000","last_clean":"2024-09-10T10:27:23.947912+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:27:23.947912+0000","last_undegraded":"2024-09-10T10:27:23.947912+0000","last_fullsized":"2024-09-10T10:27:23.947912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.953606+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:27:25.953606+0000","last_peered":"2024-09-10T10:27:25.953606+0000","last_clean":"2024-09-10T10:27:25.953606+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:27:25.953606+0000","last_undegraded":"2024-09-10T10:27:25.953606+0000","last_fullsized":"2024-09-10T10:27:25.953606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.958155+0000","last_change":"2024-09-10T10:27:23.958155+0000","last_active":"2024-09-10T10:27:23.958155+0000","last_peered":"2024-09-10T10:27:23.958155+0000","last_clean":"2024-09-10T10:27:23.958155+0000","last_became_active":"2024-09-10T10:27:23.957798+0000","last_became_peered":"2024-09-10T10:27:23.957798+0000","last_unstale":"2024-09-10T10:27:23.958155+0000","last_undegraded":"2024-09-10T10:27:23.958155+0000","last_fullsized":"2024-09-10T10:27:23.958155+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:17.103260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6,4],"acting":[3,2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.089036+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:27:24.089036+0000","last_peered":"2024-09-10T10:27:24.089036+0000","last_clean":"2024-09-10T10:27:24.089036+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:27:24.089036+0000","last_undegraded":"2024-09-10T10:27:24.089036+0000","last_fullsized":"2024-09-10T10:27:24.089036+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"26'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952989+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:27:23.952989+0000","last_peered":"2024-09-10T10:27:23.952989+0000","last_clean":"2024-09-10T10:27:23.952989+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:27:23.952989+0000","last_undegraded":"2024-09-10T10:27:23.952989+0000","last_fullsized":"2024-09-10T10:27:23.952989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948595+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:27:23.948595+0000","last_peered":"2024-09-10T10:27:23.948595+0000","last_clean":"2024-09-10T10:27:23.948595+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:27:23.948595+0000","last_undegraded":"2024-09-10T10:27:23.948595+0000","last_fullsized":"2024-09-10T10:27:23.948595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947308+0000","last_change":"2024-09-10T10:26:46.193656+0000","last_active":"2024-09-10T10:27:23.947308+0000","last_peered":"2024-09-10T10:27:23.947308+0000","last_clean":"2024-09-10T10:27:23.947308+0000","last_became_active":"2024-09-10T10:26:46.193109+0000","last_became_peered":"2024-09-10T10:26:46.193109+0000","last_unstale":"2024-09-10T10:27:23.947308+0000","last_undegraded":"2024-09-10T10:27:23.947308+0000","last_fullsized":"2024-09-10T10:27:23.947308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948758+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:27:23.948758+0000","last_peered":"2024-09-10T10:27:23.948758+0000","last_clean":"2024-09-10T10:27:23.948758+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:27:23.948758+0000","last_undegraded":"2024-09-10T10:27:23.948758+0000","last_fullsized":"2024-09-10T10:27:23.948758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948208+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:27:23.948208+0000","last_peered":"2024-09-10T10:27:23.948208+0000","last_clean":"2024-09-10T10:27:23.948208+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:27:23.948208+0000","last_undegraded":"2024-09-10T10:27:23.948208+0000","last_fullsized":"2024-09-10T10:27:23.948208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952876+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:27:23.952876+0000","last_peered":"2024-09-10T10:27:23.952876+0000","last_clean":"2024-09-10T10:27:23.952876+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:27:23.952876+0000","last_undegraded":"2024-09-10T10:27:23.952876+0000","last_fullsized":"2024-09-10T10:27:23.952876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949339+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:27:23.949339+0000","last_peered":"2024-09-10T10:27:23.949339+0000","last_clean":"2024-09-10T10:27:23.949339+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:27:23.949339+0000","last_undegraded":"2024-09-10T10:27:23.949339+0000","last_fullsized":"2024-09-10T10:27:23.949339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949149+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:27:23.949149+0000","last_peered":"2024-09-10T10:27:23.949149+0000","last_clean":"2024-09-10T10:27:23.949149+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:27:23.949149+0000","last_undegraded":"2024-09-10T10:27:23.949149+0000","last_fullsized":"2024-09-10T10:27:23.949149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":35,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952440+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:27:23.952440+0000","last_peered":"2024-09-10T10:27:23.952440+0000","last_clean":"2024-09-10T10:27:23.952440+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:27:23.952440+0000","last_undegraded":"2024-09-10T10:27:23.952440+0000","last_fullsized":"2024-09-10T10:27:23.952440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947612+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:27:23.947612+0000","last_peered":"2024-09-10T10:27:23.947612+0000","last_clean":"2024-09-10T10:27:23.947612+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:27:23.947612+0000","last_undegraded":"2024-09-10T10:27:23.947612+0000","last_fullsized":"2024-09-10T10:27:23.947612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952299+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:27:23.952299+0000","last_peered":"2024-09-10T10:27:23.952299+0000","last_clean":"2024-09-10T10:27:23.952299+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:27:23.952299+0000","last_undegraded":"2024-09-10T10:27:23.952299+0000","last_fullsized":"2024-09-10T10:27:23.952299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"26'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949520+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:27:23.949520+0000","last_peered":"2024-09-10T10:27:23.949520+0000","last_clean":"2024-09-10T10:27:23.949520+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:27:23.949520+0000","last_undegraded":"2024-09-10T10:27:23.949520+0000","last_fullsized":"2024-09-10T10:27:23.949520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948234+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:27:23.948234+0000","last_peered":"2024-09-10T10:27:23.948234+0000","last_clean":"2024-09-10T10:27:23.948234+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:27:23.948234+0000","last_undegraded":"2024-09-10T10:27:23.948234+0000","last_fullsized":"2024-09-10T10:27:23.948234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949778+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:27:23.949778+0000","last_peered":"2024-09-10T10:27:23.949778+0000","last_clean":"2024-09-10T10:27:23.949778+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:27:23.949778+0000","last_undegraded":"2024-09-10T10:27:23.949778+0000","last_fullsized":"2024-09-10T10:27:23.949778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952288+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:27:23.952288+0000","last_peered":"2024-09-10T10:27:23.952288+0000","last_clean":"2024-09-10T10:27:23.952288+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:27:23.952288+0000","last_undegraded":"2024-09-10T10:27:23.952288+0000","last_fullsized":"2024-09-10T10:27:23.952288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952248+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:27:23.952248+0000","last_peered":"2024-09-10T10:27:23.952248+0000","last_clean":"2024-09-10T10:27:23.952248+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:27:23.952248+0000","last_undegraded":"2024-09-10T10:27:23.952248+0000","last_fullsized":"2024-09-10T10:27:23.952248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948513+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:27:23.948513+0000","last_peered":"2024-09-10T10:27:23.948513+0000","last_clean":"2024-09-10T10:27:23.948513+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:27:23.948513+0000","last_undegraded":"2024-09-10T10:27:23.948513+0000","last_fullsized":"2024-09-10T10:27:23.948513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947525+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:27:23.947525+0000","last_peered":"2024-09-10T10:27:23.947525+0000","last_clean":"2024-09-10T10:27:23.947525+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:27:23.947525+0000","last_undegraded":"2024-09-10T10:27:23.947525+0000","last_fullsized":"2024-09-10T10:27:23.947525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949548+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:27:23.949548+0000","last_peered":"2024-09-10T10:27:23.949548+0000","last_clean":"2024-09-10T10:27:23.949548+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:27:23.949548+0000","last_undegraded":"2024-09-10T10:27:23.949548+0000","last_fullsized":"2024-09-10T10:27:23.949548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950189+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:27:23.950189+0000","last_peered":"2024-09-10T10:27:23.950189+0000","last_clean":"2024-09-10T10:27:23.950189+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:27:23.950189+0000","last_undegraded":"2024-09-10T10:27:23.950189+0000","last_fullsized":"2024-09-10T10:27:23.950189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952979+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:27:23.952979+0000","last_peered":"2024-09-10T10:27:23.952979+0000","last_clean":"2024-09-10T10:27:23.952979+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:27:23.952979+0000","last_undegraded":"2024-09-10T10:27:23.952979+0000","last_fullsized":"2024-09-10T10:27:23.952979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950307+0000","last_change":"2024-09-10T10:26:41.802903+0000","last_active":"2024-09-10T10:27:23.950307+0000","last_peered":"2024-09-10T10:27:23.950307+0000","last_clean":"2024-09-10T10:27:23.950307+0000","last_became_active":"2024-09-10T10:26:41.802586+0000","last_became_peered":"2024-09-10T10:26:41.802586+0000","last_unstale":"2024-09-10T10:27:23.950307+0000","last_undegraded":"2024-09-10T10:27:23.950307+0000","last_fullsized":"2024-09-10T10:27:23.950307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.958304+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:27:23.958304+0000","last_peered":"2024-09-10T10:27:23.958304+0000","last_clean":"2024-09-10T10:27:23.958304+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:27:23.958304+0000","last_undegraded":"2024-09-10T10:27:23.958304+0000","last_fullsized":"2024-09-10T10:27:23.958304+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949009+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:27:23.949009+0000","last_peered":"2024-09-10T10:27:23.949009+0000","last_clean":"2024-09-10T10:27:23.949009+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:27:23.949009+0000","last_undegraded":"2024-09-10T10:27:23.949009+0000","last_fullsized":"2024-09-10T10:27:23.949009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948486+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:27:23.948486+0000","last_peered":"2024-09-10T10:27:23.948486+0000","last_clean":"2024-09-10T10:27:23.948486+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:27:23.948486+0000","last_undegraded":"2024-09-10T10:27:23.948486+0000","last_fullsized":"2024-09-10T10:27:23.948486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947334+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:27:23.947334+0000","last_peered":"2024-09-10T10:27:23.947334+0000","last_clean":"2024-09-10T10:27:23.947334+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:27:23.947334+0000","last_undegraded":"2024-09-10T10:27:23.947334+0000","last_fullsized":"2024-09-10T10:27:23.947334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952736+0000","last_change":"2024-09-10T10:26:41.787435+0000","last_active":"2024-09-10T10:27:23.952736+0000","last_peered":"2024-09-10T10:27:23.952736+0000","last_clean":"2024-09-10T10:27:23.952736+0000","last_became_active":"2024-09-10T10:26:41.787083+0000","last_became_peered":"2024-09-10T10:26:41.787083+0000","last_unstale":"2024-09-10T10:27:23.952736+0000","last_undegraded":"2024-09-10T10:27:23.952736+0000","last_fullsized":"2024-09-10T10:27:23.952736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087002+0000","last_change":"2024-09-10T10:26:41.792587+0000","last_active":"2024-09-10T10:27:24.087002+0000","last_peered":"2024-09-10T10:27:24.087002+0000","last_clean":"2024-09-10T10:27:24.087002+0000","last_became_active":"2024-09-10T10:26:41.792320+0000","last_became_peered":"2024-09-10T10:26:41.792320+0000","last_unstale":"2024-09-10T10:27:24.087002+0000","last_undegraded":"2024-09-10T10:27:24.087002+0000","last_fullsized":"2024-09-10T10:27:24.087002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952485+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:27:23.952485+0000","last_peered":"2024-09-10T10:27:23.952485+0000","last_clean":"2024-09-10T10:27:23.952485+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:27:23.952485+0000","last_undegraded":"2024-09-10T10:27:23.952485+0000","last_fullsized":"2024-09-10T10:27:23.952485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949592+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:27:23.949592+0000","last_peered":"2024-09-10T10:27:23.949592+0000","last_clean":"2024-09-10T10:27:23.949592+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:27:23.949592+0000","last_undegraded":"2024-09-10T10:27:23.949592+0000","last_fullsized":"2024-09-10T10:27:23.949592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947607+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:27:23.947607+0000","last_peered":"2024-09-10T10:27:23.947607+0000","last_clean":"2024-09-10T10:27:23.947607+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:27:23.947607+0000","last_undegraded":"2024-09-10T10:27:23.947607+0000","last_fullsized":"2024-09-10T10:27:23.947607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948854+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:27:23.948854+0000","last_peered":"2024-09-10T10:27:23.948854+0000","last_clean":"2024-09-10T10:27:23.948854+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:27:23.948854+0000","last_undegraded":"2024-09-10T10:27:23.948854+0000","last_fullsized":"2024-09-10T10:27:23.948854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.946966+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:27:23.946966+0000","last_peered":"2024-09-10T10:27:23.946966+0000","last_clean":"2024-09-10T10:27:23.946966+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:27:23.946966+0000","last_undegraded":"2024-09-10T10:27:23.946966+0000","last_fullsized":"2024-09-10T10:27:23.946966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949436+0000","last_change":"2024-09-10T10:26:41.802117+0000","last_active":"2024-09-10T10:27:23.949436+0000","last_peered":"2024-09-10T10:27:23.949436+0000","last_clean":"2024-09-10T10:27:23.949436+0000","last_became_active":"2024-09-10T10:26:41.801798+0000","last_became_peered":"2024-09-10T10:26:41.801798+0000","last_unstale":"2024-09-10T10:27:23.949436+0000","last_undegraded":"2024-09-10T10:27:23.949436+0000","last_fullsized":"2024-09-10T10:27:23.949436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948184+0000","last_change":"2024-09-10T10:26:41.791156+0000","last_active":"2024-09-10T10:27:23.948184+0000","last_peered":"2024-09-10T10:27:23.948184+0000","last_clean":"2024-09-10T10:27:23.948184+0000","last_became_active":"2024-09-10T10:26:41.790893+0000","last_became_peered":"2024-09-10T10:26:41.790893+0000","last_unstale":"2024-09-10T10:27:23.948184+0000","last_undegraded":"2024-09-10T10:27:23.948184+0000","last_fullsized":"2024-09-10T10:27:23.948184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:46.923082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952710+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:27:23.952710+0000","last_peered":"2024-09-10T10:27:23.952710+0000","last_clean":"2024-09-10T10:27:23.952710+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:27:23.952710+0000","last_undegraded":"2024-09-10T10:27:23.952710+0000","last_fullsized":"2024-09-10T10:27:23.952710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.958305+0000","last_change":"2024-09-10T10:27:23.958305+0000","last_active":"2024-09-10T10:27:23.958305+0000","last_peered":"2024-09-10T10:27:23.958305+0000","last_clean":"2024-09-10T10:27:23.958305+0000","last_became_active":"2024-09-10T10:27:23.957954+0000","last_became_peered":"2024-09-10T10:27:23.957954+0000","last_unstale":"2024-09-10T10:27:23.958305+0000","last_undegraded":"2024-09-10T10:27:23.958305+0000","last_fullsized":"2024-09-10T10:27:23.958305+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:32.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087185+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:27:24.087185+0000","last_peered":"2024-09-10T10:27:24.087185+0000","last_clean":"2024-09-10T10:27:24.087185+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:27:24.087185+0000","last_undegraded":"2024-09-10T10:27:24.087185+0000","last_fullsized":"2024-09-10T10:27:24.087185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949288+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:27:23.949288+0000","last_peered":"2024-09-10T10:27:23.949288+0000","last_clean":"2024-09-10T10:27:23.949288+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:27:23.949288+0000","last_undegraded":"2024-09-10T10:27:23.949288+0000","last_fullsized":"2024-09-10T10:27:23.949288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086491+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:27:24.086491+0000","last_peered":"2024-09-10T10:27:24.086491+0000","last_clean":"2024-09-10T10:27:24.086491+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:27:24.086491+0000","last_undegraded":"2024-09-10T10:27:24.086491+0000","last_fullsized":"2024-09-10T10:27:24.086491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948415+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:27:23.948415+0000","last_peered":"2024-09-10T10:27:23.948415+0000","last_clean":"2024-09-10T10:27:23.948415+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:27:23.948415+0000","last_undegraded":"2024-09-10T10:27:23.948415+0000","last_fullsized":"2024-09-10T10:27:23.948415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086577+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:27:24.086577+0000","last_peered":"2024-09-10T10:27:24.086577+0000","last_clean":"2024-09-10T10:27:24.086577+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:27:24.086577+0000","last_undegraded":"2024-09-10T10:27:24.086577+0000","last_fullsized":"2024-09-10T10:27:24.086577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948970+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:27:23.948970+0000","last_peered":"2024-09-10T10:27:23.948970+0000","last_clean":"2024-09-10T10:27:23.948970+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:27:23.948970+0000","last_undegraded":"2024-09-10T10:27:23.948970+0000","last_fullsized":"2024-09-10T10:27:23.948970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947120+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:27:23.947120+0000","last_peered":"2024-09-10T10:27:23.947120+0000","last_clean":"2024-09-10T10:27:23.947120+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:27:23.947120+0000","last_undegraded":"2024-09-10T10:27:23.947120+0000","last_fullsized":"2024-09-10T10:27:23.947120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950607+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:27:23.950607+0000","last_peered":"2024-09-10T10:27:23.950607+0000","last_clean":"2024-09-10T10:27:23.950607+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:27:23.950607+0000","last_undegraded":"2024-09-10T10:27:23.950607+0000","last_fullsized":"2024-09-10T10:27:23.950607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087106+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:27:24.087106+0000","last_peered":"2024-09-10T10:27:24.087106+0000","last_clean":"2024-09-10T10:27:24.087106+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:27:24.087106+0000","last_undegraded":"2024-09-10T10:27:24.087106+0000","last_fullsized":"2024-09-10T10:27:24.087106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10266,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":207,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":20532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":718653,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":130819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":130117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":127515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":130853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":127717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":717427,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:27:35.447 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:27:35.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.578+0000 7f95bd077640 1 Processor -- start 2024-09-10T10:27:35.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.578+0000 7f95bd077640 1 -- start start 2024-09-10T10:27:35.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95bd077640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f95b80c6980 0x7f95b80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95bd077640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b80c7770 0x7f95b815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95bd077640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f95b80cb410 0x7f95b80cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95bd077640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f95b815b9e0 con 0x7f95b80cb410 2024-09-10T10:27:35.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95bd077640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f95b815b420 con 0x7f95b80c6980 2024-09-10T10:27:35.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95bd077640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f95b815b700 con 0x7f95b80c7770 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95b759e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f95b80cb410 0x7f95b80cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95b759e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f95b80cb410 0x7f95b80cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38244/0 (socket says 172.21.15.120:38244) 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95b759e640 1 -- 172.21.15.120:0/3608618396 learned_addr learned my addr 172.21.15.120:0/3608618396 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95b6d9d640 1 --2- 172.21.15.120:0/3608618396 >> v2:172.21.15.134:3300/0 conn(0x7f95b80c6980 0x7f95b80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95ae59c640 1 --2- 172.21.15.120:0/3608618396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b80c7770 0x7f95b815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95b759e640 1 -- 172.21.15.120:0/3608618396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b80c7770 msgr2=0x7f95b815aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95b759e640 1 --2- 172.21.15.120:0/3608618396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b80c7770 0x7f95b815aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95b759e640 1 -- 172.21.15.120:0/3608618396 >> v2:172.21.15.134:3300/0 conn(0x7f95b80c6980 msgr2=0x7f95b80cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95b759e640 1 --2- 172.21.15.120:0/3608618396 >> v2:172.21.15.134:3300/0 conn(0x7f95b80c6980 0x7f95b80cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.579+0000 7f95b759e640 1 -- 172.21.15.120:0/3608618396 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b80ccc50 con 0x7f95b80cb410 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.580+0000 7f95b759e640 1 --2- 172.21.15.120:0/3608618396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80cb410 0x7f95b80cbfe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95b80cde90 tx=0x7f95b002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d12c6eec33a9008 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.580+0000 7f95b659c640 1 -- 172.21.15.120:0/3608618396 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0036070 con 0x7f95b80cb410 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.580+0000 7f95b659c640 1 -- 172.21.15.120:0/3608618396 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b0031600 con 0x7f95b80cb410 2024-09-10T10:27:35.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.580+0000 7f95b659c640 1 -- 172.21.15.120:0/3608618396 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b00319d0 con 0x7f95b80cb410 2024-09-10T10:27:35.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.581+0000 7f95bd077640 1 -- 172.21.15.120:0/3608618396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80cb410 msgr2=0x7f95b80cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.581+0000 7f95bd077640 1 --2- 172.21.15.120:0/3608618396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80cb410 0x7f95b80cbfe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95b80cde90 tx=0x7f95b002a610 comp rx=0 tx=0).stop 2024-09-10T10:27:35.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.581+0000 7f95bd077640 1 -- 172.21.15.120:0/3608618396 shutdown_connections 2024-09-10T10:27:35.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.581+0000 7f95bd077640 1 --2- 172.21.15.120:0/3608618396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80cb410 0x7f95b80cbfe0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.581+0000 7f95bd077640 1 --2- 172.21.15.120:0/3608618396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b80c7770 0x7f95b815aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.581+0000 7f95bd077640 1 --2- 172.21.15.120:0/3608618396 >> v2:172.21.15.134:3300/0 conn(0x7f95b80c6980 0x7f95b80cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.581+0000 7f95bd077640 1 -- 172.21.15.120:0/3608618396 >> 172.21.15.120:0/3608618396 conn(0x7f95b806cb70 msgr2=0x7f95b806e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:35.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.581+0000 7f95bd077640 1 -- 172.21.15.120:0/3608618396 shutdown_connections 2024-09-10T10:27:35.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.581+0000 7f95bd077640 1 -- 172.21.15.120:0/3608618396 wait complete. 2024-09-10T10:27:35.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.582+0000 7f95bd077640 1 Processor -- start 2024-09-10T10:27:35.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.582+0000 7f95bd077640 1 -- start start 2024-09-10T10:27:35.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95bd077640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b80c6980 0x7f95b81e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95bd077640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80c7770 0x7f95b81e2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95bd077640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95b80cb410 0x7f95b81519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95bd077640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f95b80d0fe0 con 0x7f95b80c7770 2024-09-10T10:27:35.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95bd077640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95b80cd210 con 0x7f95b80cb410 2024-09-10T10:27:35.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95bd077640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f95b815afd0 con 0x7f95b80c6980 2024-09-10T10:27:35.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95ae59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80c7770 0x7f95b81e2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95ae59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80c7770 0x7f95b81e2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38246/0 (socket says 172.21.15.120:38246) 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95ae59c640 1 -- 172.21.15.120:0/3123352132 learned_addr learned my addr 172.21.15.120:0/3123352132 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95b6d9d640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b80c6980 0x7f95b81e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95b759e640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95b80cb410 0x7f95b81519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95ae59c640 1 -- 172.21.15.120:0/3123352132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b80c6980 msgr2=0x7f95b81e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95ae59c640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b80c6980 0x7f95b81e5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95ae59c640 1 -- 172.21.15.120:0/3123352132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95b80cb410 msgr2=0x7f95b81519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95ae59c640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95b80cb410 0x7f95b81519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.583+0000 7f95ae59c640 1 -- 172.21.15.120:0/3123352132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b81e37a0 con 0x7f95b80c7770 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.584+0000 7f95ae59c640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80c7770 0x7f95b81e2920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f95a000e9b0 tx=0x7f95a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.584+0000 7f95affff640 1 -- 172.21.15.120:0/3123352132 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0017070 con 0x7f95b80c7770 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.584+0000 7f95affff640 1 -- 172.21.15.120:0/3123352132 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a000c9a0 con 0x7f95b80c7770 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.584+0000 7f95bd077640 1 -- 172.21.15.120:0/3123352132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b81f0270 con 0x7f95b80c7770 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.584+0000 7f95affff640 1 -- 172.21.15.120:0/3123352132 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a000cd90 con 0x7f95b80c7770 2024-09-10T10:27:35.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.584+0000 7f95bd077640 1 -- 172.21.15.120:0/3123352132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b81f07d0 con 0x7f95b80c7770 2024-09-10T10:27:35.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.585+0000 7f95bd077640 1 -- 172.21.15.120:0/3123352132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b815aa90 con 0x7f95b80c7770 2024-09-10T10:27:35.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.585+0000 7f95affff640 1 -- 172.21.15.120:0/3123352132 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95a001a070 con 0x7f95b80c7770 2024-09-10T10:27:35.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.585+0000 7f95affff640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f959806be30 0x7f959806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.586+0000 7f95affff640 1 -- 172.21.15.120:0/3123352132 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f95a008b150 con 0x7f95b80c7770 2024-09-10T10:27:35.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.586+0000 7f95b6d9d640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f959806be30 0x7f959806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.586+0000 7f95b6d9d640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f959806be30 0x7f959806e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009ac0 tx=0x7f95a8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:35.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.590+0000 7f95affff640 1 -- 172.21.15.120:0/3123352132 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f95b81f0270 con 0x7f95b80c7770 2024-09-10T10:27:35.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.753+0000 7f95bd077640 1 -- 172.21.15.120:0/3123352132 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95b80cbdc0 con 0x7f959806be30 2024-09-10T10:27:35.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.760+0000 7f95affff640 1 -- 172.21.15.120:0/3123352132 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332807 (secure 0 0 0) 0x7f95b80cbdc0 con 0x7f959806be30 2024-09-10T10:27:35.766 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:35.768 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:27:35.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 -- 172.21.15.120:0/3123352132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f959806be30 msgr2=0x7f959806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f959806be30 0x7f959806e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009ac0 tx=0x7f95a8009190 comp rx=0 tx=0).stop 2024-09-10T10:27:35.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 -- 172.21.15.120:0/3123352132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80c7770 msgr2=0x7f95b81e2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80c7770 0x7f95b81e2920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f95a000e9b0 tx=0x7f95a000ee70 comp rx=0 tx=0).stop 2024-09-10T10:27:35.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 -- 172.21.15.120:0/3123352132 shutdown_connections 2024-09-10T10:27:35.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f959806be30 0x7f959806e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95b80cb410 0x7f95b81519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95b80c7770 0x7f95b81e2920 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 --2- 172.21.15.120:0/3123352132 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b80c6980 0x7f95b81e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 -- 172.21.15.120:0/3123352132 >> 172.21.15.120:0/3123352132 conn(0x7f95b806cb70 msgr2=0x7f95b80cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:35.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 -- 172.21.15.120:0/3123352132 shutdown_connections 2024-09-10T10:27:35.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.764+0000 7f95bd077640 1 -- 172.21.15.120:0/3123352132 wait complete. 2024-09-10T10:27:35.777 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-09-10T10:27:34.101968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600634,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":366,"num_write_kb":2482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218944,"kb_used_data":2752,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755776,"statfs":{"total":773094113280,"available":772869914624,"internally_reserved":0,"allocated":2818048,"data_stored":2210748,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2812,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.156632"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":41,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.948583+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:27:25.948583+0000","last_peered":"2024-09-10T10:27:25.948583+0000","last_clean":"2024-09-10T10:27:25.948583+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:27:25.948583+0000","last_undegraded":"2024-09-10T10:27:25.948583+0000","last_fullsized":"2024-09-10T10:27:25.948583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949805+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:27:23.949805+0000","last_peered":"2024-09-10T10:27:23.949805+0000","last_clean":"2024-09-10T10:27:23.949805+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:27:23.949805+0000","last_undegraded":"2024-09-10T10:27:23.949805+0000","last_fullsized":"2024-09-10T10:27:23.949805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948483+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:27:23.948483+0000","last_peered":"2024-09-10T10:27:23.948483+0000","last_clean":"2024-09-10T10:27:23.948483+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:27:23.948483+0000","last_undegraded":"2024-09-10T10:27:23.948483+0000","last_fullsized":"2024-09-10T10:27:23.948483+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949967+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:27:23.949967+0000","last_peered":"2024-09-10T10:27:23.949967+0000","last_clean":"2024-09-10T10:27:23.949967+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:27:23.949967+0000","last_undegraded":"2024-09-10T10:27:23.949967+0000","last_fullsized":"2024-09-10T10:27:23.949967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947811+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:27:23.947811+0000","last_peered":"2024-09-10T10:27:23.947811+0000","last_clean":"2024-09-10T10:27:23.947811+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:27:23.947811+0000","last_undegraded":"2024-09-10T10:27:23.947811+0000","last_fullsized":"2024-09-10T10:27:23.947811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947563+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:27:23.947563+0000","last_peered":"2024-09-10T10:27:23.947563+0000","last_clean":"2024-09-10T10:27:23.947563+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:27:23.947563+0000","last_undegraded":"2024-09-10T10:27:23.947563+0000","last_fullsized":"2024-09-10T10:27:23.947563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"28'11","reported_seq":39,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.948235+0000","last_change":"2024-09-10T10:26:40.796337+0000","last_active":"2024-09-10T10:27:25.948235+0000","last_peered":"2024-09-10T10:27:25.948235+0000","last_clean":"2024-09-10T10:27:25.948235+0000","last_became_active":"2024-09-10T10:26:40.796029+0000","last_became_peered":"2024-09-10T10:26:40.796029+0000","last_unstale":"2024-09-10T10:27:25.948235+0000","last_undegraded":"2024-09-10T10:27:25.948235+0000","last_fullsized":"2024-09-10T10:27:25.948235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949057+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:27:23.949057+0000","last_peered":"2024-09-10T10:27:23.949057+0000","last_clean":"2024-09-10T10:27:23.949057+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:27:23.949057+0000","last_undegraded":"2024-09-10T10:27:23.949057+0000","last_fullsized":"2024-09-10T10:27:23.949057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.955254+0000","last_change":"2024-09-10T10:27:23.955254+0000","last_active":"2024-09-10T10:27:23.955254+0000","last_peered":"2024-09-10T10:27:23.955254+0000","last_clean":"2024-09-10T10:27:23.955254+0000","last_became_active":"2024-09-10T10:27:23.954737+0000","last_became_peered":"2024-09-10T10:27:23.954737+0000","last_unstale":"2024-09-10T10:27:23.955254+0000","last_undegraded":"2024-09-10T10:27:23.955254+0000","last_fullsized":"2024-09-10T10:27:23.955254+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.946861+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:27:25.946861+0000","last_peered":"2024-09-10T10:27:25.946861+0000","last_clean":"2024-09-10T10:27:25.946861+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:27:25.946861+0000","last_undegraded":"2024-09-10T10:27:25.946861+0000","last_fullsized":"2024-09-10T10:27:25.946861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.957643+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:27:23.957643+0000","last_peered":"2024-09-10T10:27:23.957643+0000","last_clean":"2024-09-10T10:27:23.957643+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:27:23.957643+0000","last_undegraded":"2024-09-10T10:27:23.957643+0000","last_fullsized":"2024-09-10T10:27:23.957643+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947808+0000","last_change":"2024-09-10T10:26:45.816587+0000","last_active":"2024-09-10T10:27:23.947808+0000","last_peered":"2024-09-10T10:27:23.947808+0000","last_clean":"2024-09-10T10:27:23.947808+0000","last_became_active":"2024-09-10T10:26:45.816138+0000","last_became_peered":"2024-09-10T10:26:45.816138+0000","last_unstale":"2024-09-10T10:27:23.947808+0000","last_undegraded":"2024-09-10T10:27:23.947808+0000","last_fullsized":"2024-09-10T10:27:23.947808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952071+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:27:23.952071+0000","last_peered":"2024-09-10T10:27:23.952071+0000","last_clean":"2024-09-10T10:27:23.952071+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:27:23.952071+0000","last_undegraded":"2024-09-10T10:27:23.952071+0000","last_fullsized":"2024-09-10T10:27:23.952071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086759+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:27:24.086759+0000","last_peered":"2024-09-10T10:27:24.086759+0000","last_clean":"2024-09-10T10:27:24.086759+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:27:24.086759+0000","last_undegraded":"2024-09-10T10:27:24.086759+0000","last_fullsized":"2024-09-10T10:27:24.086759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.955276+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:27:23.955276+0000","last_peered":"2024-09-10T10:27:23.955276+0000","last_clean":"2024-09-10T10:27:23.955276+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:27:23.955276+0000","last_undegraded":"2024-09-10T10:27:23.955276+0000","last_fullsized":"2024-09-10T10:27:23.955276+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949460+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:27:23.949460+0000","last_peered":"2024-09-10T10:27:23.949460+0000","last_clean":"2024-09-10T10:27:23.949460+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:27:23.949460+0000","last_undegraded":"2024-09-10T10:27:23.949460+0000","last_fullsized":"2024-09-10T10:27:23.949460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'3","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949216+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:27:23.949216+0000","last_peered":"2024-09-10T10:27:23.949216+0000","last_clean":"2024-09-10T10:27:23.949216+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:27:23.949216+0000","last_undegraded":"2024-09-10T10:27:23.949216+0000","last_fullsized":"2024-09-10T10:27:23.949216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947768+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:27:23.947768+0000","last_peered":"2024-09-10T10:27:23.947768+0000","last_clean":"2024-09-10T10:27:23.947768+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:27:23.947768+0000","last_undegraded":"2024-09-10T10:27:23.947768+0000","last_fullsized":"2024-09-10T10:27:23.947768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949368+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:27:23.949368+0000","last_peered":"2024-09-10T10:27:23.949368+0000","last_clean":"2024-09-10T10:27:23.949368+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:27:23.949368+0000","last_undegraded":"2024-09-10T10:27:23.949368+0000","last_fullsized":"2024-09-10T10:27:23.949368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":35,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.946351+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:27:25.946351+0000","last_peered":"2024-09-10T10:27:25.946351+0000","last_clean":"2024-09-10T10:27:25.946351+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:27:25.946351+0000","last_undegraded":"2024-09-10T10:27:25.946351+0000","last_fullsized":"2024-09-10T10:27:25.946351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949704+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:27:23.949704+0000","last_peered":"2024-09-10T10:27:23.949704+0000","last_clean":"2024-09-10T10:27:23.949704+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:27:23.949704+0000","last_undegraded":"2024-09-10T10:27:23.949704+0000","last_fullsized":"2024-09-10T10:27:23.949704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948893+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:27:23.948893+0000","last_peered":"2024-09-10T10:27:23.948893+0000","last_clean":"2024-09-10T10:27:23.948893+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:27:23.948893+0000","last_undegraded":"2024-09-10T10:27:23.948893+0000","last_fullsized":"2024-09-10T10:27:23.948893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086339+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:27:24.086339+0000","last_peered":"2024-09-10T10:27:24.086339+0000","last_clean":"2024-09-10T10:27:24.086339+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:27:24.086339+0000","last_undegraded":"2024-09-10T10:27:24.086339+0000","last_fullsized":"2024-09-10T10:27:24.086339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947906+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:27:23.947906+0000","last_peered":"2024-09-10T10:27:23.947906+0000","last_clean":"2024-09-10T10:27:23.947906+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:27:23.947906+0000","last_undegraded":"2024-09-10T10:27:23.947906+0000","last_fullsized":"2024-09-10T10:27:23.947906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947699+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:27:23.947699+0000","last_peered":"2024-09-10T10:27:23.947699+0000","last_clean":"2024-09-10T10:27:23.947699+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:27:23.947699+0000","last_undegraded":"2024-09-10T10:27:23.947699+0000","last_fullsized":"2024-09-10T10:27:23.947699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'7","reported_seq":35,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.945274+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:27:25.945274+0000","last_peered":"2024-09-10T10:27:25.945274+0000","last_clean":"2024-09-10T10:27:25.945274+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:27:25.945274+0000","last_undegraded":"2024-09-10T10:27:25.945274+0000","last_fullsized":"2024-09-10T10:27:25.945274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947985+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:27:23.947985+0000","last_peered":"2024-09-10T10:27:23.947985+0000","last_clean":"2024-09-10T10:27:23.947985+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:27:23.947985+0000","last_undegraded":"2024-09-10T10:27:23.947985+0000","last_fullsized":"2024-09-10T10:27:23.947985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950344+0000","last_change":"2024-09-10T10:26:41.795085+0000","last_active":"2024-09-10T10:27:23.950344+0000","last_peered":"2024-09-10T10:27:23.950344+0000","last_clean":"2024-09-10T10:27:23.950344+0000","last_became_active":"2024-09-10T10:26:41.794819+0000","last_became_peered":"2024-09-10T10:26:41.794819+0000","last_unstale":"2024-09-10T10:27:23.950344+0000","last_undegraded":"2024-09-10T10:27:23.950344+0000","last_fullsized":"2024-09-10T10:27:23.950344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'7","reported_seq":35,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950702+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:27:23.950702+0000","last_peered":"2024-09-10T10:27:23.950702+0000","last_clean":"2024-09-10T10:27:23.950702+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:27:23.950702+0000","last_undegraded":"2024-09-10T10:27:23.950702+0000","last_fullsized":"2024-09-10T10:27:23.950702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948433+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:27:23.948433+0000","last_peered":"2024-09-10T10:27:23.948433+0000","last_clean":"2024-09-10T10:27:23.948433+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:27:23.948433+0000","last_undegraded":"2024-09-10T10:27:23.948433+0000","last_fullsized":"2024-09-10T10:27:23.948433+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949581+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:27:23.949581+0000","last_peered":"2024-09-10T10:27:23.949581+0000","last_clean":"2024-09-10T10:27:23.949581+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:27:23.949581+0000","last_undegraded":"2024-09-10T10:27:23.949581+0000","last_fullsized":"2024-09-10T10:27:23.949581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087107+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:27:24.087107+0000","last_peered":"2024-09-10T10:27:24.087107+0000","last_clean":"2024-09-10T10:27:24.087107+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:27:24.087107+0000","last_undegraded":"2024-09-10T10:27:24.087107+0000","last_fullsized":"2024-09-10T10:27:24.087107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947236+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:27:23.947236+0000","last_peered":"2024-09-10T10:27:23.947236+0000","last_clean":"2024-09-10T10:27:23.947236+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:27:23.947236+0000","last_undegraded":"2024-09-10T10:27:23.947236+0000","last_fullsized":"2024-09-10T10:27:23.947236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949745+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:27:23.949745+0000","last_peered":"2024-09-10T10:27:23.949745+0000","last_clean":"2024-09-10T10:27:23.949745+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:27:23.949745+0000","last_undegraded":"2024-09-10T10:27:23.949745+0000","last_fullsized":"2024-09-10T10:27:23.949745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949683+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:27:23.949683+0000","last_peered":"2024-09-10T10:27:23.949683+0000","last_clean":"2024-09-10T10:27:23.949683+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:27:23.949683+0000","last_undegraded":"2024-09-10T10:27:23.949683+0000","last_fullsized":"2024-09-10T10:27:23.949683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948820+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:27:23.948820+0000","last_peered":"2024-09-10T10:27:23.948820+0000","last_clean":"2024-09-10T10:27:23.948820+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:27:23.948820+0000","last_undegraded":"2024-09-10T10:27:23.948820+0000","last_fullsized":"2024-09-10T10:27:23.948820+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948546+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:27:23.948546+0000","last_peered":"2024-09-10T10:27:23.948546+0000","last_clean":"2024-09-10T10:27:23.948546+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:27:23.948546+0000","last_undegraded":"2024-09-10T10:27:23.948546+0000","last_fullsized":"2024-09-10T10:27:23.948546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.950835+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:27:25.950835+0000","last_peered":"2024-09-10T10:27:25.950835+0000","last_clean":"2024-09-10T10:27:25.950835+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:27:25.950835+0000","last_undegraded":"2024-09-10T10:27:25.950835+0000","last_fullsized":"2024-09-10T10:27:25.950835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949466+0000","last_change":"2024-09-10T10:26:45.814008+0000","last_active":"2024-09-10T10:27:23.949466+0000","last_peered":"2024-09-10T10:27:23.949466+0000","last_clean":"2024-09-10T10:27:23.949466+0000","last_became_active":"2024-09-10T10:26:45.813627+0000","last_became_peered":"2024-09-10T10:26:45.813627+0000","last_unstale":"2024-09-10T10:27:23.949466+0000","last_undegraded":"2024-09-10T10:27:23.949466+0000","last_fullsized":"2024-09-10T10:27:23.949466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948999+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:27:23.948999+0000","last_peered":"2024-09-10T10:27:23.948999+0000","last_clean":"2024-09-10T10:27:23.948999+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:27:23.948999+0000","last_undegraded":"2024-09-10T10:27:23.948999+0000","last_fullsized":"2024-09-10T10:27:23.948999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"28'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952325+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:27:23.952325+0000","last_peered":"2024-09-10T10:27:23.952325+0000","last_clean":"2024-09-10T10:27:23.952325+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:27:23.952325+0000","last_undegraded":"2024-09-10T10:27:23.952325+0000","last_fullsized":"2024-09-10T10:27:23.952325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949799+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:27:23.949799+0000","last_peered":"2024-09-10T10:27:23.949799+0000","last_clean":"2024-09-10T10:27:23.949799+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:27:23.949799+0000","last_undegraded":"2024-09-10T10:27:23.949799+0000","last_fullsized":"2024-09-10T10:27:23.949799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.946956+0000","last_change":"2024-09-10T10:26:41.799224+0000","last_active":"2024-09-10T10:27:23.946956+0000","last_peered":"2024-09-10T10:27:23.946956+0000","last_clean":"2024-09-10T10:27:23.946956+0000","last_became_active":"2024-09-10T10:26:41.798948+0000","last_became_peered":"2024-09-10T10:26:41.798948+0000","last_unstale":"2024-09-10T10:27:23.946956+0000","last_undegraded":"2024-09-10T10:27:23.946956+0000","last_fullsized":"2024-09-10T10:27:23.946956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086405+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:27:24.086405+0000","last_peered":"2024-09-10T10:27:24.086405+0000","last_clean":"2024-09-10T10:27:24.086405+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:27:24.086405+0000","last_undegraded":"2024-09-10T10:27:24.086405+0000","last_fullsized":"2024-09-10T10:27:24.086405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947499+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:27:23.947499+0000","last_peered":"2024-09-10T10:27:23.947499+0000","last_clean":"2024-09-10T10:27:23.947499+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:27:23.947499+0000","last_undegraded":"2024-09-10T10:27:23.947499+0000","last_fullsized":"2024-09-10T10:27:23.947499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950262+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:27:23.950262+0000","last_peered":"2024-09-10T10:27:23.950262+0000","last_clean":"2024-09-10T10:27:23.950262+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:27:23.950262+0000","last_undegraded":"2024-09-10T10:27:23.950262+0000","last_fullsized":"2024-09-10T10:27:23.950262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949493+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:27:23.949493+0000","last_peered":"2024-09-10T10:27:23.949493+0000","last_clean":"2024-09-10T10:27:23.949493+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:27:23.949493+0000","last_undegraded":"2024-09-10T10:27:23.949493+0000","last_fullsized":"2024-09-10T10:27:23.949493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":184,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086475+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:27:24.086475+0000","last_peered":"2024-09-10T10:27:24.086475+0000","last_clean":"2024-09-10T10:27:24.086475+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:27:24.086475+0000","last_undegraded":"2024-09-10T10:27:24.086475+0000","last_fullsized":"2024-09-10T10:27:24.086475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952629+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:27:23.952629+0000","last_peered":"2024-09-10T10:27:23.952629+0000","last_clean":"2024-09-10T10:27:23.952629+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:27:23.952629+0000","last_undegraded":"2024-09-10T10:27:23.952629+0000","last_fullsized":"2024-09-10T10:27:23.952629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086749+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:27:24.086749+0000","last_peered":"2024-09-10T10:27:24.086749+0000","last_clean":"2024-09-10T10:27:24.086749+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:27:24.086749+0000","last_undegraded":"2024-09-10T10:27:24.086749+0000","last_fullsized":"2024-09-10T10:27:24.086749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952500+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:27:23.952500+0000","last_peered":"2024-09-10T10:27:23.952500+0000","last_clean":"2024-09-10T10:27:23.952500+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:27:23.952500+0000","last_undegraded":"2024-09-10T10:27:23.952500+0000","last_fullsized":"2024-09-10T10:27:23.952500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952228+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:27:23.952228+0000","last_peered":"2024-09-10T10:27:23.952228+0000","last_clean":"2024-09-10T10:27:23.952228+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:27:23.952228+0000","last_undegraded":"2024-09-10T10:27:23.952228+0000","last_fullsized":"2024-09-10T10:27:23.952228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949116+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:27:23.949116+0000","last_peered":"2024-09-10T10:27:23.949116+0000","last_clean":"2024-09-10T10:27:23.949116+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:27:23.949116+0000","last_undegraded":"2024-09-10T10:27:23.949116+0000","last_fullsized":"2024-09-10T10:27:23.949116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950498+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:27:23.950498+0000","last_peered":"2024-09-10T10:27:23.950498+0000","last_clean":"2024-09-10T10:27:23.950498+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:27:23.950498+0000","last_undegraded":"2024-09-10T10:27:23.950498+0000","last_fullsized":"2024-09-10T10:27:23.950498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947427+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:27:23.947427+0000","last_peered":"2024-09-10T10:27:23.947427+0000","last_clean":"2024-09-10T10:27:23.947427+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:27:23.947427+0000","last_undegraded":"2024-09-10T10:27:23.947427+0000","last_fullsized":"2024-09-10T10:27:23.947427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087253+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:27:24.087253+0000","last_peered":"2024-09-10T10:27:24.087253+0000","last_clean":"2024-09-10T10:27:24.087253+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:27:24.087253+0000","last_undegraded":"2024-09-10T10:27:24.087253+0000","last_fullsized":"2024-09-10T10:27:24.087253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.944217+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:27:25.944217+0000","last_peered":"2024-09-10T10:27:25.944217+0000","last_clean":"2024-09-10T10:27:25.944217+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:27:25.944217+0000","last_undegraded":"2024-09-10T10:27:25.944217+0000","last_fullsized":"2024-09-10T10:27:25.944217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947488+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:27:23.947488+0000","last_peered":"2024-09-10T10:27:23.947488+0000","last_clean":"2024-09-10T10:27:23.947488+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:27:23.947488+0000","last_undegraded":"2024-09-10T10:27:23.947488+0000","last_fullsized":"2024-09-10T10:27:23.947488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087016+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:27:24.087016+0000","last_peered":"2024-09-10T10:27:24.087016+0000","last_clean":"2024-09-10T10:27:24.087016+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:27:24.087016+0000","last_undegraded":"2024-09-10T10:27:24.087016+0000","last_fullsized":"2024-09-10T10:27:24.087016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948188+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:27:23.948188+0000","last_peered":"2024-09-10T10:27:23.948188+0000","last_clean":"2024-09-10T10:27:23.948188+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:27:23.948188+0000","last_undegraded":"2024-09-10T10:27:23.948188+0000","last_fullsized":"2024-09-10T10:27:23.948188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947786+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:27:23.947786+0000","last_peered":"2024-09-10T10:27:23.947786+0000","last_clean":"2024-09-10T10:27:23.947786+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:27:23.947786+0000","last_undegraded":"2024-09-10T10:27:23.947786+0000","last_fullsized":"2024-09-10T10:27:23.947786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086641+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:27:24.086641+0000","last_peered":"2024-09-10T10:27:24.086641+0000","last_clean":"2024-09-10T10:27:24.086641+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:27:24.086641+0000","last_undegraded":"2024-09-10T10:27:24.086641+0000","last_fullsized":"2024-09-10T10:27:24.086641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"28'17","reported_seq":45,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952559+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:27:23.952559+0000","last_peered":"2024-09-10T10:27:23.952559+0000","last_clean":"2024-09-10T10:27:23.952559+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:27:23.952559+0000","last_undegraded":"2024-09-10T10:27:23.952559+0000","last_fullsized":"2024-09-10T10:27:23.952559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947405+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:27:23.947405+0000","last_peered":"2024-09-10T10:27:23.947405+0000","last_clean":"2024-09-10T10:27:23.947405+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:27:23.947405+0000","last_undegraded":"2024-09-10T10:27:23.947405+0000","last_fullsized":"2024-09-10T10:27:23.947405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952447+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:27:23.952447+0000","last_peered":"2024-09-10T10:27:23.952447+0000","last_clean":"2024-09-10T10:27:23.952447+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:27:23.952447+0000","last_undegraded":"2024-09-10T10:27:23.952447+0000","last_fullsized":"2024-09-10T10:27:23.952447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952119+0000","last_change":"2024-09-10T10:26:40.794739+0000","last_active":"2024-09-10T10:27:23.952119+0000","last_peered":"2024-09-10T10:27:23.952119+0000","last_clean":"2024-09-10T10:27:23.952119+0000","last_became_active":"2024-09-10T10:26:40.794430+0000","last_became_peered":"2024-09-10T10:26:40.794430+0000","last_unstale":"2024-09-10T10:27:23.952119+0000","last_undegraded":"2024-09-10T10:27:23.952119+0000","last_fullsized":"2024-09-10T10:27:23.952119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950206+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:27:23.950206+0000","last_peered":"2024-09-10T10:27:23.950206+0000","last_clean":"2024-09-10T10:27:23.950206+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:27:23.950206+0000","last_undegraded":"2024-09-10T10:27:23.950206+0000","last_fullsized":"2024-09-10T10:27:23.950206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086713+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:27:24.086713+0000","last_peered":"2024-09-10T10:27:24.086713+0000","last_clean":"2024-09-10T10:27:24.086713+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:27:24.086713+0000","last_undegraded":"2024-09-10T10:27:24.086713+0000","last_fullsized":"2024-09-10T10:27:24.086713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"28'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948645+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:27:23.948645+0000","last_peered":"2024-09-10T10:27:23.948645+0000","last_clean":"2024-09-10T10:27:23.948645+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:27:23.948645+0000","last_undegraded":"2024-09-10T10:27:23.948645+0000","last_fullsized":"2024-09-10T10:27:23.948645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949263+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:27:23.949263+0000","last_peered":"2024-09-10T10:27:23.949263+0000","last_clean":"2024-09-10T10:27:23.949263+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:27:23.949263+0000","last_undegraded":"2024-09-10T10:27:23.949263+0000","last_fullsized":"2024-09-10T10:27:23.949263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947085+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:27:23.947085+0000","last_peered":"2024-09-10T10:27:23.947085+0000","last_clean":"2024-09-10T10:27:23.947085+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:27:23.947085+0000","last_undegraded":"2024-09-10T10:27:23.947085+0000","last_fullsized":"2024-09-10T10:27:23.947085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":37,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.944224+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:27:25.944224+0000","last_peered":"2024-09-10T10:27:25.944224+0000","last_clean":"2024-09-10T10:27:25.944224+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:27:25.944224+0000","last_undegraded":"2024-09-10T10:27:25.944224+0000","last_fullsized":"2024-09-10T10:27:25.944224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949572+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:27:23.949572+0000","last_peered":"2024-09-10T10:27:23.949572+0000","last_clean":"2024-09-10T10:27:23.949572+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:27:23.949572+0000","last_undegraded":"2024-09-10T10:27:23.949572+0000","last_fullsized":"2024-09-10T10:27:23.949572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086767+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:27:24.086767+0000","last_peered":"2024-09-10T10:27:24.086767+0000","last_clean":"2024-09-10T10:27:24.086767+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:27:24.086767+0000","last_undegraded":"2024-09-10T10:27:24.086767+0000","last_fullsized":"2024-09-10T10:27:24.086767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"28'13","reported_seq":41,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.946768+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:27:25.946768+0000","last_peered":"2024-09-10T10:27:25.946768+0000","last_clean":"2024-09-10T10:27:25.946768+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:27:25.946768+0000","last_undegraded":"2024-09-10T10:27:25.946768+0000","last_fullsized":"2024-09-10T10:27:25.946768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948966+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:27:23.948966+0000","last_peered":"2024-09-10T10:27:23.948966+0000","last_clean":"2024-09-10T10:27:23.948966+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:27:23.948966+0000","last_undegraded":"2024-09-10T10:27:23.948966+0000","last_fullsized":"2024-09-10T10:27:23.948966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948445+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:27:23.948445+0000","last_peered":"2024-09-10T10:27:23.948445+0000","last_clean":"2024-09-10T10:27:23.948445+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:27:23.948445+0000","last_undegraded":"2024-09-10T10:27:23.948445+0000","last_fullsized":"2024-09-10T10:27:23.948445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950348+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:27:23.950348+0000","last_peered":"2024-09-10T10:27:23.950348+0000","last_clean":"2024-09-10T10:27:23.950348+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:27:23.950348+0000","last_undegraded":"2024-09-10T10:27:23.950348+0000","last_fullsized":"2024-09-10T10:27:23.950348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948304+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:27:23.948304+0000","last_peered":"2024-09-10T10:27:23.948304+0000","last_clean":"2024-09-10T10:27:23.948304+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:27:23.948304+0000","last_undegraded":"2024-09-10T10:27:23.948304+0000","last_fullsized":"2024-09-10T10:27:23.948304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947912+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:27:23.947912+0000","last_peered":"2024-09-10T10:27:23.947912+0000","last_clean":"2024-09-10T10:27:23.947912+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:27:23.947912+0000","last_undegraded":"2024-09-10T10:27:23.947912+0000","last_fullsized":"2024-09-10T10:27:23.947912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:25.953606+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:27:25.953606+0000","last_peered":"2024-09-10T10:27:25.953606+0000","last_clean":"2024-09-10T10:27:25.953606+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:27:25.953606+0000","last_undegraded":"2024-09-10T10:27:25.953606+0000","last_fullsized":"2024-09-10T10:27:25.953606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.958155+0000","last_change":"2024-09-10T10:27:23.958155+0000","last_active":"2024-09-10T10:27:23.958155+0000","last_peered":"2024-09-10T10:27:23.958155+0000","last_clean":"2024-09-10T10:27:23.958155+0000","last_became_active":"2024-09-10T10:27:23.957798+0000","last_became_peered":"2024-09-10T10:27:23.957798+0000","last_unstale":"2024-09-10T10:27:23.958155+0000","last_undegraded":"2024-09-10T10:27:23.958155+0000","last_fullsized":"2024-09-10T10:27:23.958155+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:17.103260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6,4],"acting":[3,2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.089036+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:27:24.089036+0000","last_peered":"2024-09-10T10:27:24.089036+0000","last_clean":"2024-09-10T10:27:24.089036+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:27:24.089036+0000","last_undegraded":"2024-09-10T10:27:24.089036+0000","last_fullsized":"2024-09-10T10:27:24.089036+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"26'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952989+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:27:23.952989+0000","last_peered":"2024-09-10T10:27:23.952989+0000","last_clean":"2024-09-10T10:27:23.952989+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:27:23.952989+0000","last_undegraded":"2024-09-10T10:27:23.952989+0000","last_fullsized":"2024-09-10T10:27:23.952989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948595+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:27:23.948595+0000","last_peered":"2024-09-10T10:27:23.948595+0000","last_clean":"2024-09-10T10:27:23.948595+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:27:23.948595+0000","last_undegraded":"2024-09-10T10:27:23.948595+0000","last_fullsized":"2024-09-10T10:27:23.948595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947308+0000","last_change":"2024-09-10T10:26:46.193656+0000","last_active":"2024-09-10T10:27:23.947308+0000","last_peered":"2024-09-10T10:27:23.947308+0000","last_clean":"2024-09-10T10:27:23.947308+0000","last_became_active":"2024-09-10T10:26:46.193109+0000","last_became_peered":"2024-09-10T10:26:46.193109+0000","last_unstale":"2024-09-10T10:27:23.947308+0000","last_undegraded":"2024-09-10T10:27:23.947308+0000","last_fullsized":"2024-09-10T10:27:23.947308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948758+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:27:23.948758+0000","last_peered":"2024-09-10T10:27:23.948758+0000","last_clean":"2024-09-10T10:27:23.948758+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:27:23.948758+0000","last_undegraded":"2024-09-10T10:27:23.948758+0000","last_fullsized":"2024-09-10T10:27:23.948758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948208+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:27:23.948208+0000","last_peered":"2024-09-10T10:27:23.948208+0000","last_clean":"2024-09-10T10:27:23.948208+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:27:23.948208+0000","last_undegraded":"2024-09-10T10:27:23.948208+0000","last_fullsized":"2024-09-10T10:27:23.948208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952876+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:27:23.952876+0000","last_peered":"2024-09-10T10:27:23.952876+0000","last_clean":"2024-09-10T10:27:23.952876+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:27:23.952876+0000","last_undegraded":"2024-09-10T10:27:23.952876+0000","last_fullsized":"2024-09-10T10:27:23.952876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949339+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:27:23.949339+0000","last_peered":"2024-09-10T10:27:23.949339+0000","last_clean":"2024-09-10T10:27:23.949339+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:27:23.949339+0000","last_undegraded":"2024-09-10T10:27:23.949339+0000","last_fullsized":"2024-09-10T10:27:23.949339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949149+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:27:23.949149+0000","last_peered":"2024-09-10T10:27:23.949149+0000","last_clean":"2024-09-10T10:27:23.949149+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:27:23.949149+0000","last_undegraded":"2024-09-10T10:27:23.949149+0000","last_fullsized":"2024-09-10T10:27:23.949149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":35,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952440+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:27:23.952440+0000","last_peered":"2024-09-10T10:27:23.952440+0000","last_clean":"2024-09-10T10:27:23.952440+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:27:23.952440+0000","last_undegraded":"2024-09-10T10:27:23.952440+0000","last_fullsized":"2024-09-10T10:27:23.952440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947612+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:27:23.947612+0000","last_peered":"2024-09-10T10:27:23.947612+0000","last_clean":"2024-09-10T10:27:23.947612+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:27:23.947612+0000","last_undegraded":"2024-09-10T10:27:23.947612+0000","last_fullsized":"2024-09-10T10:27:23.947612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952299+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:27:23.952299+0000","last_peered":"2024-09-10T10:27:23.952299+0000","last_clean":"2024-09-10T10:27:23.952299+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:27:23.952299+0000","last_undegraded":"2024-09-10T10:27:23.952299+0000","last_fullsized":"2024-09-10T10:27:23.952299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"26'5","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949520+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:27:23.949520+0000","last_peered":"2024-09-10T10:27:23.949520+0000","last_clean":"2024-09-10T10:27:23.949520+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:27:23.949520+0000","last_undegraded":"2024-09-10T10:27:23.949520+0000","last_fullsized":"2024-09-10T10:27:23.949520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948234+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:27:23.948234+0000","last_peered":"2024-09-10T10:27:23.948234+0000","last_clean":"2024-09-10T10:27:23.948234+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:27:23.948234+0000","last_undegraded":"2024-09-10T10:27:23.948234+0000","last_fullsized":"2024-09-10T10:27:23.948234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949778+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:27:23.949778+0000","last_peered":"2024-09-10T10:27:23.949778+0000","last_clean":"2024-09-10T10:27:23.949778+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:27:23.949778+0000","last_undegraded":"2024-09-10T10:27:23.949778+0000","last_fullsized":"2024-09-10T10:27:23.949778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952288+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:27:23.952288+0000","last_peered":"2024-09-10T10:27:23.952288+0000","last_clean":"2024-09-10T10:27:23.952288+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:27:23.952288+0000","last_undegraded":"2024-09-10T10:27:23.952288+0000","last_fullsized":"2024-09-10T10:27:23.952288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952248+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:27:23.952248+0000","last_peered":"2024-09-10T10:27:23.952248+0000","last_clean":"2024-09-10T10:27:23.952248+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:27:23.952248+0000","last_undegraded":"2024-09-10T10:27:23.952248+0000","last_fullsized":"2024-09-10T10:27:23.952248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948513+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:27:23.948513+0000","last_peered":"2024-09-10T10:27:23.948513+0000","last_clean":"2024-09-10T10:27:23.948513+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:27:23.948513+0000","last_undegraded":"2024-09-10T10:27:23.948513+0000","last_fullsized":"2024-09-10T10:27:23.948513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947525+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:27:23.947525+0000","last_peered":"2024-09-10T10:27:23.947525+0000","last_clean":"2024-09-10T10:27:23.947525+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:27:23.947525+0000","last_undegraded":"2024-09-10T10:27:23.947525+0000","last_fullsized":"2024-09-10T10:27:23.947525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949548+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:27:23.949548+0000","last_peered":"2024-09-10T10:27:23.949548+0000","last_clean":"2024-09-10T10:27:23.949548+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:27:23.949548+0000","last_undegraded":"2024-09-10T10:27:23.949548+0000","last_fullsized":"2024-09-10T10:27:23.949548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950189+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:27:23.950189+0000","last_peered":"2024-09-10T10:27:23.950189+0000","last_clean":"2024-09-10T10:27:23.950189+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:27:23.950189+0000","last_undegraded":"2024-09-10T10:27:23.950189+0000","last_fullsized":"2024-09-10T10:27:23.950189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952979+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:27:23.952979+0000","last_peered":"2024-09-10T10:27:23.952979+0000","last_clean":"2024-09-10T10:27:23.952979+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:27:23.952979+0000","last_undegraded":"2024-09-10T10:27:23.952979+0000","last_fullsized":"2024-09-10T10:27:23.952979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950307+0000","last_change":"2024-09-10T10:26:41.802903+0000","last_active":"2024-09-10T10:27:23.950307+0000","last_peered":"2024-09-10T10:27:23.950307+0000","last_clean":"2024-09-10T10:27:23.950307+0000","last_became_active":"2024-09-10T10:26:41.802586+0000","last_became_peered":"2024-09-10T10:26:41.802586+0000","last_unstale":"2024-09-10T10:27:23.950307+0000","last_undegraded":"2024-09-10T10:27:23.950307+0000","last_fullsized":"2024-09-10T10:27:23.950307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.958304+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:27:23.958304+0000","last_peered":"2024-09-10T10:27:23.958304+0000","last_clean":"2024-09-10T10:27:23.958304+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:27:23.958304+0000","last_undegraded":"2024-09-10T10:27:23.958304+0000","last_fullsized":"2024-09-10T10:27:23.958304+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949009+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:27:23.949009+0000","last_peered":"2024-09-10T10:27:23.949009+0000","last_clean":"2024-09-10T10:27:23.949009+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:27:23.949009+0000","last_undegraded":"2024-09-10T10:27:23.949009+0000","last_fullsized":"2024-09-10T10:27:23.949009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948486+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:27:23.948486+0000","last_peered":"2024-09-10T10:27:23.948486+0000","last_clean":"2024-09-10T10:27:23.948486+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:27:23.948486+0000","last_undegraded":"2024-09-10T10:27:23.948486+0000","last_fullsized":"2024-09-10T10:27:23.948486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947334+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:27:23.947334+0000","last_peered":"2024-09-10T10:27:23.947334+0000","last_clean":"2024-09-10T10:27:23.947334+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:27:23.947334+0000","last_undegraded":"2024-09-10T10:27:23.947334+0000","last_fullsized":"2024-09-10T10:27:23.947334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952736+0000","last_change":"2024-09-10T10:26:41.787435+0000","last_active":"2024-09-10T10:27:23.952736+0000","last_peered":"2024-09-10T10:27:23.952736+0000","last_clean":"2024-09-10T10:27:23.952736+0000","last_became_active":"2024-09-10T10:26:41.787083+0000","last_became_peered":"2024-09-10T10:26:41.787083+0000","last_unstale":"2024-09-10T10:27:23.952736+0000","last_undegraded":"2024-09-10T10:27:23.952736+0000","last_fullsized":"2024-09-10T10:27:23.952736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087002+0000","last_change":"2024-09-10T10:26:41.792587+0000","last_active":"2024-09-10T10:27:24.087002+0000","last_peered":"2024-09-10T10:27:24.087002+0000","last_clean":"2024-09-10T10:27:24.087002+0000","last_became_active":"2024-09-10T10:26:41.792320+0000","last_became_peered":"2024-09-10T10:26:41.792320+0000","last_unstale":"2024-09-10T10:27:24.087002+0000","last_undegraded":"2024-09-10T10:27:24.087002+0000","last_fullsized":"2024-09-10T10:27:24.087002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952485+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:27:23.952485+0000","last_peered":"2024-09-10T10:27:23.952485+0000","last_clean":"2024-09-10T10:27:23.952485+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:27:23.952485+0000","last_undegraded":"2024-09-10T10:27:23.952485+0000","last_fullsized":"2024-09-10T10:27:23.952485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949592+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:27:23.949592+0000","last_peered":"2024-09-10T10:27:23.949592+0000","last_clean":"2024-09-10T10:27:23.949592+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:27:23.949592+0000","last_undegraded":"2024-09-10T10:27:23.949592+0000","last_fullsized":"2024-09-10T10:27:23.949592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947607+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:27:23.947607+0000","last_peered":"2024-09-10T10:27:23.947607+0000","last_clean":"2024-09-10T10:27:23.947607+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:27:23.947607+0000","last_undegraded":"2024-09-10T10:27:23.947607+0000","last_fullsized":"2024-09-10T10:27:23.947607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948854+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:27:23.948854+0000","last_peered":"2024-09-10T10:27:23.948854+0000","last_clean":"2024-09-10T10:27:23.948854+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:27:23.948854+0000","last_undegraded":"2024-09-10T10:27:23.948854+0000","last_fullsized":"2024-09-10T10:27:23.948854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.946966+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:27:23.946966+0000","last_peered":"2024-09-10T10:27:23.946966+0000","last_clean":"2024-09-10T10:27:23.946966+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:27:23.946966+0000","last_undegraded":"2024-09-10T10:27:23.946966+0000","last_fullsized":"2024-09-10T10:27:23.946966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949436+0000","last_change":"2024-09-10T10:26:41.802117+0000","last_active":"2024-09-10T10:27:23.949436+0000","last_peered":"2024-09-10T10:27:23.949436+0000","last_clean":"2024-09-10T10:27:23.949436+0000","last_became_active":"2024-09-10T10:26:41.801798+0000","last_became_peered":"2024-09-10T10:26:41.801798+0000","last_unstale":"2024-09-10T10:27:23.949436+0000","last_undegraded":"2024-09-10T10:27:23.949436+0000","last_fullsized":"2024-09-10T10:27:23.949436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948184+0000","last_change":"2024-09-10T10:26:41.791156+0000","last_active":"2024-09-10T10:27:23.948184+0000","last_peered":"2024-09-10T10:27:23.948184+0000","last_clean":"2024-09-10T10:27:23.948184+0000","last_became_active":"2024-09-10T10:26:41.790893+0000","last_became_peered":"2024-09-10T10:26:41.790893+0000","last_unstale":"2024-09-10T10:27:23.948184+0000","last_undegraded":"2024-09-10T10:27:23.948184+0000","last_fullsized":"2024-09-10T10:27:23.948184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:46.923082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.952710+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:27:23.952710+0000","last_peered":"2024-09-10T10:27:23.952710+0000","last_clean":"2024-09-10T10:27:23.952710+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:27:23.952710+0000","last_undegraded":"2024-09-10T10:27:23.952710+0000","last_fullsized":"2024-09-10T10:27:23.952710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.958305+0000","last_change":"2024-09-10T10:27:23.958305+0000","last_active":"2024-09-10T10:27:23.958305+0000","last_peered":"2024-09-10T10:27:23.958305+0000","last_clean":"2024-09-10T10:27:23.958305+0000","last_became_active":"2024-09-10T10:27:23.957954+0000","last_became_peered":"2024-09-10T10:27:23.957954+0000","last_unstale":"2024-09-10T10:27:23.958305+0000","last_undegraded":"2024-09-10T10:27:23.958305+0000","last_fullsized":"2024-09-10T10:27:23.958305+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:32.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087185+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:27:24.087185+0000","last_peered":"2024-09-10T10:27:24.087185+0000","last_clean":"2024-09-10T10:27:24.087185+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:27:24.087185+0000","last_undegraded":"2024-09-10T10:27:24.087185+0000","last_fullsized":"2024-09-10T10:27:24.087185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.949288+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:27:23.949288+0000","last_peered":"2024-09-10T10:27:23.949288+0000","last_clean":"2024-09-10T10:27:23.949288+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:27:23.949288+0000","last_undegraded":"2024-09-10T10:27:23.949288+0000","last_fullsized":"2024-09-10T10:27:23.949288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086491+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:27:24.086491+0000","last_peered":"2024-09-10T10:27:24.086491+0000","last_clean":"2024-09-10T10:27:24.086491+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:27:24.086491+0000","last_undegraded":"2024-09-10T10:27:24.086491+0000","last_fullsized":"2024-09-10T10:27:24.086491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948415+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:27:23.948415+0000","last_peered":"2024-09-10T10:27:23.948415+0000","last_clean":"2024-09-10T10:27:23.948415+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:27:23.948415+0000","last_undegraded":"2024-09-10T10:27:23.948415+0000","last_fullsized":"2024-09-10T10:27:23.948415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.086577+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:27:24.086577+0000","last_peered":"2024-09-10T10:27:24.086577+0000","last_clean":"2024-09-10T10:27:24.086577+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:27:24.086577+0000","last_undegraded":"2024-09-10T10:27:24.086577+0000","last_fullsized":"2024-09-10T10:27:24.086577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.948970+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:27:23.948970+0000","last_peered":"2024-09-10T10:27:23.948970+0000","last_clean":"2024-09-10T10:27:23.948970+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:27:23.948970+0000","last_undegraded":"2024-09-10T10:27:23.948970+0000","last_fullsized":"2024-09-10T10:27:23.948970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.947120+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:27:23.947120+0000","last_peered":"2024-09-10T10:27:23.947120+0000","last_clean":"2024-09-10T10:27:23.947120+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:27:23.947120+0000","last_undegraded":"2024-09-10T10:27:23.947120+0000","last_fullsized":"2024-09-10T10:27:23.947120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:23.950607+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:27:23.950607+0000","last_peered":"2024-09-10T10:27:23.950607+0000","last_clean":"2024-09-10T10:27:23.950607+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:27:23.950607+0000","last_undegraded":"2024-09-10T10:27:23.950607+0000","last_fullsized":"2024-09-10T10:27:23.950607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":26,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-10T10:27:24.087106+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:27:24.087106+0000","last_peered":"2024-09-10T10:27:24.087106+0000","last_clean":"2024-09-10T10:27:24.087106+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:27:24.087106+0000","last_undegraded":"2024-09-10T10:27:24.087106+0000","last_fullsized":"2024-09-10T10:27:24.087106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10266,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":207,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":20532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":718653,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":130819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":130117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":127515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":130853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":127717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":717427,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:27:35.782 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:27:35.782 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-10T10:27:35.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.913+0000 7f2b6f7c4640 1 Processor -- start 2024-09-10T10:27:35.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.913+0000 7f2b6f7c4640 1 -- start start 2024-09-10T10:27:35.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6f7c4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2b680c6980 0x7f2b680cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6f7c4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2b680cb410 0x7f2b680cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6f7c4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b6815b1a0 con 0x7f2b680c7770 2024-09-10T10:27:35.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6f7c4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f2b6815b9e0 con 0x7f2b680c6980 2024-09-10T10:27:35.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6f7c4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2b6815b420 con 0x7f2b680cb410 2024-09-10T10:27:35.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6d560640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2b680c6980 0x7f2b680cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6d560640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2b680c6980 0x7f2b680cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38260/0 (socket says 172.21.15.120:38260) 2024-09-10T10:27:35.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6cd5f640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f2b680c7770 0x7f2b6815aa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:33602/0 (socket says 172.21.15.120:33602) 2024-09-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6d560640 1 -- 172.21.15.120:0/718851942 learned_addr learned my addr 172.21.15.120:0/718851942 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6dd61640 1 --2- 172.21.15.120:0/718851942 >> v2:172.21.15.134:3300/0 conn(0x7f2b680cb410 0x7f2b680cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6d560640 1 -- 172.21.15.120:0/718851942 >> v2:172.21.15.134:3300/0 conn(0x7f2b680cb410 msgr2=0x7f2b680cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6d560640 1 --2- 172.21.15.120:0/718851942 >> v2:172.21.15.134:3300/0 conn(0x7f2b680cb410 0x7f2b680cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6d560640 1 -- 172.21.15.120:0/718851942 >> v1:172.21.15.120:6789/0 conn(0x7f2b680c7770 legacy=0x7f2b6815aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b6d560640 1 -- 172.21.15.120:0/718851942 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6815b700 con 0x7f2b680c6980 2024-09-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.914+0000 7f2b5ffff640 1 -- 172.21.15.120:0/718851942 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2184307695 0 0) 0x7f2b58001110 con 0x7f2b680c7770 2024-09-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.915+0000 7f2b6d560640 1 --2- 172.21.15.120:0/718851942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 0x7f2b680cd730 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2b6806dee0 tx=0x7f2b6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e51bd7d9798a12d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.915+0000 7f2b5ffff640 1 -- 172.21.15.120:0/718851942 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64037070 con 0x7f2b680c6980 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.915+0000 7f2b5ffff640 1 -- 172.21.15.120:0/718851942 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b64032440 con 0x7f2b680c6980 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.915+0000 7f2b5ffff640 1 -- 172.21.15.120:0/718851942 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64032810 con 0x7f2b680c6980 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.916+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/718851942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 msgr2=0x7f2b680cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.916+0000 7f2b6f7c4640 1 --2- 172.21.15.120:0/718851942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 0x7f2b680cd730 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2b6806dee0 tx=0x7f2b6402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.916+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/718851942 shutdown_connections 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.916+0000 7f2b6f7c4640 1 --2- 172.21.15.120:0/718851942 >> v2:172.21.15.134:3300/0 conn(0x7f2b680cb410 0x7f2b680cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.916+0000 7f2b6f7c4640 1 --2- 172.21.15.120:0/718851942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 0x7f2b680cd730 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.916+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/718851942 >> 172.21.15.120:0/718851942 conn(0x7f2b6806cb70 msgr2=0x7f2b6806e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.916+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/718851942 shutdown_connections 2024-09-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.916+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/718851942 wait complete. 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.917+0000 7f2b6f7c4640 1 Processor -- start 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.917+0000 7f2b6f7c4640 1 -- start start 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6f7c4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 0x7f2b681e7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6f7c4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b680c7770 0x7f2b681e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6f7c4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b680cb410 0x7f2b68157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6f7c4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2b680cd210 con 0x7f2b680c6980 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6f7c4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b680ccf30 con 0x7f2b680c7770 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6f7c4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2b680d0fe0 con 0x7f2b680cb410 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6d560640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 0x7f2b681e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6d560640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 0x7f2b681e7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38274/0 (socket says 172.21.15.120:38274) 2024-09-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6d560640 1 -- 172.21.15.120:0/953827074 learned_addr learned my addr 172.21.15.120:0/953827074 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6cd5f640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b680c7770 0x7f2b681e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6dd61640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b680cb410 0x7f2b68157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6d560640 1 -- 172.21.15.120:0/953827074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b680cb410 msgr2=0x7f2b68157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6d560640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b680cb410 0x7f2b68157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6d560640 1 -- 172.21.15.120:0/953827074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b680c7770 msgr2=0x7f2b681e4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6d560640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b680c7770 0x7f2b681e4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6d560640 1 -- 172.21.15.120:0/953827074 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b681516b0 con 0x7f2b680c6980 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b6d560640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 0x7f2b681e7710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2b6402a220 tx=0x7f2b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.918+0000 7f2b5dffb640 1 -- 172.21.15.120:0/953827074 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64041020 con 0x7f2b680c6980 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.919+0000 7f2b5dffb640 1 -- 172.21.15.120:0/953827074 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b640306c0 con 0x7f2b680c6980 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.919+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/953827074 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b681f4e20 con 0x7f2b680c6980 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.919+0000 7f2b5dffb640 1 -- 172.21.15.120:0/953827074 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64037040 con 0x7f2b680c6980 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.919+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/953827074 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b681f4060 con 0x7f2b680c6980 2024-09-10T10:27:35.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.920+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/953827074 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b68076410 con 0x7f2b680c6980 2024-09-10T10:27:35.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.920+0000 7f2b5dffb640 1 -- 172.21.15.120:0/953827074 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b64044020 con 0x7f2b680c6980 2024-09-10T10:27:35.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.921+0000 7f2b5dffb640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2b4c06be30 0x7f2b4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:35.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.921+0000 7f2b6cd5f640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2b4c06be30 0x7f2b4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:35.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.921+0000 7f2b5dffb640 1 -- 172.21.15.120:0/953827074 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f2b640abaa0 con 0x7f2b680c6980 2024-09-10T10:27:35.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.921+0000 7f2b6cd5f640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2b4c06be30 0x7f2b4c06e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b681e42a0 tx=0x7f2b58007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:35.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:35.925+0000 7f2b5dffb640 1 -- 172.21.15.120:0/953827074 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2b68076410 con 0x7f2b680c6980 2024-09-10T10:27:36.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.093+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/953827074 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f2b680c3ea0 con 0x7f2b680c6980 2024-09-10T10:27:36.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.094+0000 7f2b5dffb640 1 -- 172.21.15.120:0/953827074 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v28) ==== 56+0+3702 (secure 0 0 0) 0x7f2b680c3ea0 con 0x7f2b680c6980 2024-09-10T10:27:36.096 INFO:teuthology.orchestra.run.smithi120.stdout:epoch 28 2024-09-10T10:27:36.096 INFO:teuthology.orchestra.run.smithi120.stdout:fsid 7d0d0830-4370-41c3-9acc-b67b07024f68 2024-09-10T10:27:36.096 INFO:teuthology.orchestra.run.smithi120.stdout:created 2024-09-10T10:26:18.018818+0000 2024-09-10T10:27:36.096 INFO:teuthology.orchestra.run.smithi120.stdout:modified 2024-09-10T10:27:23.940655+0000 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:crush_version 5 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:full_ratio 0.95 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:backfillfull_ratio 0.9 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:nearfull_ratio 0.85 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:require_min_compat_client luminous 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:min_compat_client luminous 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:require_osd_release squid 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:stretch_mode_enabled false 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:pool 2 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.50 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:pool 3 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.38 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 26 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-10T10:27:36.097 INFO:teuthology.orchestra.run.smithi120.stdout:max_osd 8 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0 up in weight 1 up_from 13 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] [v2:172.21.15.120:6810/3664783490,v1:172.21.15.120:6811/3664783490] exists,up e3b9c3c3-cf7d-4b33-913e-60e928439d8e 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] [v2:172.21.15.120:6802/3972761964,v1:172.21.15.120:6803/3972761964] exists,up 62c20a9d-e6ab-446d-998c-9ee46e46aa90 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] [v2:172.21.15.120:6818/2916410859,v1:172.21.15.120:6819/2916410859] exists,up 1528f8e0-5e13-4ec7-9d98-0d94b7a505c3 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] [v2:172.21.15.120:6826/1224278950,v1:172.21.15.120:6827/1224278950] exists,up 74942f78-c1fc-4886-abf0-950722c5edc6 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] [v2:172.21.15.134:6818/1874635282,v1:172.21.15.134:6819/1874635282] exists,up e71319d4-5318-4441-84a8-7665ae037596 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] [v2:172.21.15.134:6810/457724087,v1:172.21.15.134:6811/457724087] exists,up 29e193cd-37af-43e5-bd0c-2f85ce0631e8 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] [v2:172.21.15.134:6826/50133702,v1:172.21.15.134:6827/50133702] exists,up 55523ec2-cf12-44b3-aeea-f1cedabc996a 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] [v2:172.21.15.134:6802/251960875,v1:172.21.15.134:6803/251960875] exists,up 097b1149-3e3c-4e5a-890b-d4cc4361d51b 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.16 [5,1] 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.1d [5,2] 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.28 [3,2] 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.36 [5,2] 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 4.d [1,5] 2024-09-10T10:27:36.098 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 4.10 [1,2] 2024-09-10T10:27:36.099 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 4.1b [1,2] 2024-09-10T10:27:36.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/953827074 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2b4c06be30 msgr2=0x7f2b4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2b4c06be30 0x7f2b4c06e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b681e42a0 tx=0x7f2b58007990 comp rx=0 tx=0).stop 2024-09-10T10:27:36.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/953827074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 msgr2=0x7f2b681e7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 0x7f2b681e7710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2b6402a220 tx=0x7f2b640046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/953827074 shutdown_connections 2024-09-10T10:27:36.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2b4c06be30 0x7f2b4c06e2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b680cb410 0x7f2b68157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b680c7770 0x7f2b681e4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 --2- 172.21.15.120:0/953827074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b680c6980 0x7f2b681e7710 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/953827074 >> 172.21.15.120:0/953827074 conn(0x7f2b6806cb70 msgr2=0x7f2b680cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:36.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/953827074 shutdown_connections 2024-09-10T10:27:36.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.096+0000 7f2b6f7c4640 1 -- 172.21.15.120:0/953827074 wait complete. 2024-09-10T10:27:36.109 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] [v2:172.21.15.120:6810/3664783490,v1:172.21.15.120:6811/3664783490] exists,up e3b9c3c3-cf7d-4b33-913e-60e928439d8e', 'osd.1 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] [v2:172.21.15.120:6802/3972761964,v1:172.21.15.120:6803/3972761964] exists,up 62c20a9d-e6ab-446d-998c-9ee46e46aa90', 'osd.2 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] [v2:172.21.15.120:6818/2916410859,v1:172.21.15.120:6819/2916410859] exists,up 1528f8e0-5e13-4ec7-9d98-0d94b7a505c3', 'osd.3 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] [v2:172.21.15.120:6826/1224278950,v1:172.21.15.120:6827/1224278950] exists,up 74942f78-c1fc-4886-abf0-950722c5edc6', 'osd.4 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] [v2:172.21.15.134:6818/1874635282,v1:172.21.15.134:6819/1874635282] exists,up e71319d4-5318-4441-84a8-7665ae037596', 'osd.5 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] [v2:172.21.15.134:6810/457724087,v1:172.21.15.134:6811/457724087] exists,up 29e193cd-37af-43e5-bd0c-2f85ce0631e8', 'osd.6 up in weight 1 up_from 13 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] [v2:172.21.15.134:6826/50133702,v1:172.21.15.134:6827/50133702] exists,up 55523ec2-cf12-44b3-aeea-f1cedabc996a', 'osd.7 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] [v2:172.21.15.134:6802/251960875,v1:172.21.15.134:6803/251960875] exists,up 097b1149-3e3c-4e5a-890b-d4cc4361d51b'] 2024-09-10T10:27:36.110 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-10T10:27:36.209 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:36.209 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mon.a", 2024-09-10T10:27:36.209 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster": "ceph", 2024-09-10T10:27:36.209 INFO:teuthology.orchestra.run.smithi120.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-10T10:27:36.209 INFO:teuthology.orchestra.run.smithi120.stdout: "admin_socket_mode": "", 2024-09-10T10:27:36.209 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-10T10:27:36.209 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_client_required": "cephx, none", 2024-09-10T10:27:36.209 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_cluster_required": "cephx", 2024-09-10T10:27:36.209 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_debug": "true", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_service_required": "cephx", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "auth_supported": "cephx", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_aio": "true", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_aio_poll_ms": "250", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_aio_reap_max": "16", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_async_discard": "false", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_block_size": "4096", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_debug_aio": "false", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-10T10:27:36.210 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_debug_inflight_ios": "false", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_enable_discard": "false", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_flock_retry": "3", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_inject_crash": "0", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_ioring": "false", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_ioring_hipri": "false", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-10T10:27:36.211 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bdev_type": "", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_alloc_size": "1048576", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_allocator": "hybrid", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_buffered_io": "true", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_check_for_zeros": "false", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_compact_log_sync": "false", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-10T10:27:36.212 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_max_log_runway": "4194304", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_max_prefetch": "1048576", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_min_flush_size": "524288", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_min_log_runway": "1048576", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_replay_recovery": "false", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_sync_write": "false", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_allocation_from_file": "true", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_allocator": "hybrid", 2024-09-10T10:27:36.213 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bdev_label_multi": "true", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_create": "true", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_db_create": "false", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_db_path": "", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_db_size": "0", 2024-09-10T10:27:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_path": "", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_preallocate_file": "false", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_size": "107374182400", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_wal_create": "false", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_wal_path": "", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_block_wal_size": "100663296", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bluefs": "true", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-10T10:27:36.215 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_autotune": "true", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_size": "0", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_type": "2q", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-10T10:27:36.216 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_clone_cow": "true", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_mode": "none", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_csum_type": "crc32c", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_freelist": "false", 2024-09-10T10:27:36.217 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_small_allocations": "0", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_default_buffered_read": "true", 2024-09-10T10:27:36.218 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_default_buffered_write": "false", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fail_eio": "false", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-10T10:27:36.219 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_mount": "false", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_umount": "false", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-10T10:27:36.220 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_ignore_data_csum": "false", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_log_op_age": "5.000000", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_alloc_size": "0", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_blob_size": "0", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_max_deferred_txc": "32", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_min_alloc_size": "0", 2024-09-10T10:27:36.221 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_nid_prealloc": "1024", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_retry_disk_reads": "3", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_rocksdb_cf": "true", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-10T10:27:36.222 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_spdk_mem": "512", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_tracing": "false", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-10T10:27:36.223 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_zero_block_detection": "false", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-10T10:27:36.224 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_cluster_require_signatures": "false", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_cluster_require_version": "2", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_require_signatures": "false", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_require_version": "2", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_service_require_signatures": "false", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_service_require_version": "2", 2024-09-10T10:27:36.225 INFO:teuthology.orchestra.run.smithi120.stdout: "cephx_sign_messages": "true", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "chdir": "", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_acl_type": "", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_asio_thread_count": "2", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_cache_mid": "0.750000", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_cache_size": "16384", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_caps_release_delay": "5", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_check_pool_perm": "true", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_debug_force_sync_read": "false", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_debug_getattr_caps": "false", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_debug_inject_features": "", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_debug_inject_tick_delay": "0", 2024-09-10T10:27:36.226 INFO:teuthology.orchestra.run.smithi120.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_die_on_failed_remount": "false", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_dirsize_rbytes": "true", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_force_lazyio": "false", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_fs": "", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_inject_release_failure": "false", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_max_inline_size": "4096", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_mds_namespace": "", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_metadata": "", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_mount_gid": "-1", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_mount_timeout": "300", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_mount_uid": "-1", 2024-09-10T10:27:36.227 INFO:teuthology.orchestra.run.smithi120.stdout: "client_mountpoint": "/", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_notify_timeout": "10", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc": "true", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc_max_dirty": "104857600", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc_max_objects": "1000", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc_size": "209715200", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_oc_target_dirty": "8388608", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_permissions": "true", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_quota": "true", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_quota_df": "true", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_readahead_max_bytes": "0", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_readahead_max_periods": "4", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_readahead_min": "131072", 2024-09-10T10:27:36.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_reconnect_stale": "false", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_shutdown_timeout": "30", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_snapdir": ".snap", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_tick_interval": "1", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_trace": "", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_try_dentry_invalidate": "false", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_use_faked_inos": "false", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "client_use_random_mds": "false", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_graylog": "false", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_graylog_port": "12201", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_monitors": "default=true", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_syslog": "false", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-10T10:27:36.229 INFO:teuthology.orchestra.run.smithi120.stdout: "clog_to_syslog_level": "info", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "-", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_network": "", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_network_interface": "", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "compressor_zlib_isal": "false", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "compressor_zlib_level": "5", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "compressor_zlib_winsize": "-15", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "compressor_zstd_level": "1", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_alien_op_num_threads": "6", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-10T10:27:36.230 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_osd_stat_interval": "0", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_seastar_cpu_cores": "", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "crimson_seastar_num_threads": "0", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_location": "", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_location_hook": "", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_location_hook_timeout": "10", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "daemonize": "false", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "daos_pool": "tank", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_alienstore": "0/5", 2024-09-10T10:27:36.231 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_allow_any_pool_priority": "false", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_asok": "1/5", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_asok_assert_abort": "false", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_asserts_on_shutdown": "true", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_auth": "20/20", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_bdev": "1/3", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_bluefs": "1/5", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_bluestore": "1/5", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_buffer": "0/1", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_ceph_exporter": "1/5", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_cephfs_mirror": "0/5", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_cephsqlite": "0/5", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_client": "0/5", 2024-09-10T10:27:36.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_compressor": "1/5", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_context": "0/1", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_crimson_interrupt": "0/5", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_crush": "1/1", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_crypto": "1/5", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_cyanstore": "0/5", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_deliberately_leak_memory": "false", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_disable_randomized_ping": "false", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_dpdk": "1/5", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_eventtrace": "1/5", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_filer": "0/1", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_filestore": "1/3", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_finisher": "1/1", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_fuse": "1/5", 2024-09-10T10:27:36.233 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_heartbeat_testing_span": "0", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_heartbeatmap": "1/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_javaclient": "1/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_journal": "1/3", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_journaler": "0/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_kstore": "1/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_lockdep": "0/1", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mclock": "1/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds": "1/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_balancer": "1/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_locker": "1/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_log": "1/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_log_expire": "1/5", 2024-09-10T10:27:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_migrator": "1/5", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mds_quiesce": "3/5", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_memstore": "1/5", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mgr": "2/5", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mgrc": "1/5", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_mon": "20/20", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_monc": "0/10", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_ms": "1/1", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_none": "0/5", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_objclass": "0/5", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_objectcacher": "0/5", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_objecter": "0/1", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_optracker": "0/5", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_osd": "1/5", 2024-09-10T10:27:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_paxos": "20/20", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_perfcounter": "1/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_prioritycache": "1/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rados": "0/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rbd": "0/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rbd_mirror": "0/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rbd_pwl": "0/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rbd_replay": "0/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_refs": "0/0", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_reserver": "1/1", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw": "1/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_access": "1/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_datacache": "1/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_dbstore": "1/5", 2024-09-10T10:27:36.236 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_flight": "1/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rgw_sync": "1/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_rocksdb": "4/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore": "0/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_backref": "0/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_cache": "0/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_cleaner": "0/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_device": "0/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_epm": "0/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_journal": "0/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_lba": "0/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_odata": "0/5", 2024-09-10T10:27:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_omap": "0/5", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_onode": "0/5", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_t": "0/5", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_seastore_tm": "0/5", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_striper": "0/1", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_test": "0/5", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_throttle": "1/1", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_timer": "0/1", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_tp": "0/5", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "debug_trace": "1/5", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "device_failure_prediction_mode": "none", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-10T10:27:36.238 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "err_to_graylog": "false", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "err_to_journald": "false", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "err_to_stderr": "true", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "err_to_syslog": "false", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event_tracing": "false", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_addr": "0.0.0.0", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_http_port": "9926", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_prio_limit": "5", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_sort_metrics": "true", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "exporter_stats_period": "5", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "fake_statfs_for_testing": "0", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "fatal_signal_handlers": "true", 2024-09-10T10:27:36.239 INFO:teuthology.orchestra.run.smithi120.stdout: "filer_max_purge_ops": "10", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filer_max_truncate_ops": "128", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_apply_finisher_threads": "1", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_blackhole": "false", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_btrfs_clone_range": "true", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_btrfs_snap": "true", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_caller_concurrency": "10", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_collect_device_partition_information": "true", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_commit_timeout": "600.000000", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_debug_inject_read_err": "false", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_debug_omap_check": "false", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_debug_verify_split": "false", 2024-09-10T10:27:36.240 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_dump_file": "", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fadvise": "true", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fail_eio": "true", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fd_cache_shards": "16", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fd_cache_size": "128", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fiemap": "false", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fiemap_threshold": "4096", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_inject_stall": "0", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_journal_parallel": "false", 2024-09-10T10:27:36.241 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_journal_trailing": "false", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_journal_writeahead": "false", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_kill_at": "0", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattrs": "0", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_xattr_value_size": "0", 2024-09-10T10:27:36.242 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_merge_threshold": "-10", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_odsync_write": "false", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_omap_backend": "rocksdb", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_omap_backend_path": "", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_op_thread_timeout": "60", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_op_threads": "2", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_punch_hole": "false", 2024-09-10T10:27:36.243 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_queue_max_ops": "50", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_seek_data_hole": "false", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_sloppy_crc": "false", 2024-09-10T10:27:36.244 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_splice": "false", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_split_multiple": "2", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_split_rand_factor": "20", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_update_to": "1000", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_enable": "true", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-10T10:27:36.245 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_xfs_extsize": "false", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "filestore_zfs_snap": "false", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "fio_dir": "/tmp/fio", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_allow_other": "true", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_atomic_o_trunc": "true", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_big_writes": "true", 2024-09-10T10:27:36.246 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_debug": "false", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_default_permissions": "false", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_disable_pagecache": "false", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_max_write": "0", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_multithreaded": "true", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_require_active_mds": "true", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_set_user_groups": "true", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_splice_move": "true", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_splice_read": "true", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_splice_write": "true", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fuse_use_invalidate_cb": "true", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-10T10:27:36.247 INFO:teuthology.orchestra.run.smithi120.stdout: "gss_target_name": "ceph", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_file": "", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_inject_failure": "0", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_interval": "5", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "host": "smithi120", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-10T10:27:36.248 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "inject_early_sigterm": "false", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "jaeger_agent_port": "6799", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "jaeger_tracing_enable": "false", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_aio": "true", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_align_min_size": "65536", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_block_align": "true", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_block_size": "4096", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_dio": "true", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_discard": "false", 2024-09-10T10:27:36.249 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_force_aio": "false", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_ignore_corruption": "false", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_max_write_bytes": "10485760", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_max_write_entries": "100", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_replay_from": "0", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_write_header_frequency": "0", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journal_zero_on_create": "false", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journaler_prefetch_periods": "10", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journaler_prezero_periods": "5", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "journaler_write_head_interval": "15", 2024-09-10T10:27:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: "key": "", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "keyfile": "", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_backend": "rocksdb", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_default_stripe_size": "65536", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_fsck_on_mount": "false", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_max_bytes": "67108864", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_max_ops": "512", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_nid_prealloc": "1024", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_onode_map_size": "1024", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_sync_submit_transaction": "false", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "kstore_sync_transaction": "false", 2024-09-10T10:27:36.251 INFO:teuthology.orchestra.run.smithi120.stdout: "librados_thread_count": "2", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "lockdep": "false", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "lockdep_force_backtrace": "false", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_coarse_timestamps": "true", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_flush_on_exit": "false", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_graylog_host": "127.0.0.1", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_graylog_port": "12201", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_max_new": "1000", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_max_recent": "10000", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_stderr_prefix": "", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_stop_at_utilization": "0.970000", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": "true", 2024-09-10T10:27:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_graylog": "false", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_journald": "false", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_stderr": "false", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_syslog": "false", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "max_rotating_auth_attempts": "10", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_action_on_write_error": "1", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_alternate_name_max": "8192", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_asio_thread_count": "2", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_export_pin": "true", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_fragment_dirs": "true", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_fragment_interval": "5", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-10T10:27:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_interval": "10", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_max": "-1", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_max_until": "-1", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_merge_size": "50", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_midchunk": "0.300000", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_min_start": "0.200000", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_minchunk": "0.001000", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_mode": "0", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_need_max": "1.200000", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_need_min": "0.800000", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_overload_epochs": "2", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-10T10:27:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_split_bits": "3", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_split_size": "10000", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_target_decay": "10.000000", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_beacon_grace": "15.000000", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_beacon_interval": "4.000000", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_mid": "0.700000", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_quiesce_delay": "0", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-10T10:27:36.255 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_release_free_interval": "10", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_reservation": "0.050000", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_trim_interval": "1", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache_trim_threshold": "262144", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_client_prealloc_inos": "1000", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_connect_bootstrapping": "false", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_damage_table_max_entries": "10000", 2024-09-10T10:27:36.256 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_debug_auth_pins": "false", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_debug_frag": "false", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_debug_scatterstat": "false", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_debug_subtrees": "false", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_decay_halflife": "5.000000", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_default_dir_hash": "2", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_defer_session_stale": "true", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_deny_all_reconnect": "false", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dir_keys_per_op": "16384", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dir_max_commit_size": "10", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dir_max_entries": "0", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dir_prefetch": "true", 2024-09-10T10:27:36.257 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dump_cache_on_map": "false", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_early_reply": "true", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_enable_op_tracker": "true", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_enforce_unique_name": "true", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_export_ephemeral_random": "true", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-10T10:27:36.258 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_health_summarize_threshold": "10", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_health_dummy": "false", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_migrator_session_race": "false", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_join_fs": "", 2024-09-10T10:27:36.259 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_journal_format": "1", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_create_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_export_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_import_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_journal_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_journal_expire_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_journal_replay_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_link_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_mdstable_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_openc_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_rename_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_kill_shutdown_at": "0", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_event_large_threshold": "524288", 2024-09-10T10:27:36.260 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_events_per_segment": "1024", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_max_events": "-1", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_max_segments": "128", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_pause": "false", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_segment_size": "0", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_trim_threshold": "128", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log_warn_factor": "2.000000", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_caps_per_client": "1048576", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_completed_flushes": "100000", 2024-09-10T10:27:36.261 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_completed_requests": "100000", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_export_size": "20971520", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_file_recover": "32", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_purge_files": "64", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_purge_ops": "8192", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_max_snaps_per_dir": "100", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_metrics_update_interval": "2", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_min_caps_per_client": "100", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_min_caps_working_set": "10000", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_numa_node": "-1", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-10T10:27:36.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_complaint_time": "30.000000", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_history_duration": "600", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_history_size": "20", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_history_slow_op_size": "20", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_op_log_threshold": "5", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_ping_grace": "15", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_ping_interval": "5", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_max_caps": "30000", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-10T10:27:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_recall_warning_threshold": "262144", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_replay_interval": "1.000000", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_root_ino_gid": "0", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_root_ino_uid": "0", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-10T10:27:36.264 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_shutdown_check": "0", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_skip_ino": "0", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_snap_max_uid": "4294967294", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_snap_min_uid": "0", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_snap_rstat": "false", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_standby_replay_damaged": "false", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_symlink_recovery": "true", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_thrash_exports": "0", 2024-09-10T10:27:36.265 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_thrash_fragments": "0", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_tick_interval": "5.000000", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_valgrind_exit": "false", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_verify_backtrace": "1", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_verify_scatter": "false", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_wipe_sessions": "false", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool_debug": "false", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "memstore_device_bytes": "1073741824", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "memstore_page_set": "false", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "memstore_page_size": "65536", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_client_bytes": "134217728", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_client_messages": "512", 2024-09-10T10:27:36.266 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_disabled_modules": "", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_enable_op_tracker": "true", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_max_pg_num_change": "128", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_mds_bytes": "134217728", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_mds_messages": "128", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_mon_bytes": "134217728", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_mon_messages": "128", 2024-09-10T10:27:36.267 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_history_duration": "600", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_history_size": "20", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_op_log_threshold": "5", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_osd_bytes": "536870912", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_osd_messages": "8192", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_pool": "true", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_standby_modules": "true", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_stats_period": "5", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_stats_threshold": "5", 2024-09-10T10:27:36.268 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_tick_period": "2", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_allow_pool_delete": "true", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_allow_pool_size_one": "true", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_auth_validate_all_caps": "true", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_bytes": "104857600", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_directed_command_retry": "2", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-10T10:27:36.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_hunt_parallel": "3", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_log_interval": "1.000000", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_ping_interval": "10.000000", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_client_target_rank": "-1", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_file_level": "debug", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_file": "true", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-10T10:27:36.270 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_journald": "false", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_compact_on_bootstrap": "false", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_compact_on_start": "false", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_compact_on_trim": "true", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_cpu_threads": "4", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_crush_min_required_version": "hammer", 2024-09-10T10:27:36.271 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_daemon_bytes": "419430400", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_data_avail_crit": "5", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_data_avail_warn": "5", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_data_size_warn": "16106127360", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_dump_json": "false", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_dump_transactions": "false", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_extra_checks": "false", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-10T10:27:36.272 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_no_require_reef": "false", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_no_require_squid": "false", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_down_mkfs_grace": "300", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_down_uptime_grace": "60", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_election_default_strategy": "1", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_election_timeout": "5.000000", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_elector_ping_divisor": "2", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_enable_op_tracker": "true", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_fake_pool_delete": "false", 2024-09-10T10:27:36.273 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_force_quorum_join": "false", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_globalid_prealloc": "10000", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_detail_to_clog": "false", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_log_update_period": "5", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_max_detail": "50", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_to_clog": "true", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_to_clog_interval": "600", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_host": "172.21.15.120,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_host_override": "", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_initial_members": "", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-10T10:27:36.274 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_lease": "5.000000", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_log_full_interval": "50", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_log_max": "10000", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_log_max_summary": "50", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_log_epochs": "500", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_osd": "10000", 2024-09-10T10:27:36.275 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_pg_per_osd": "10000", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_pool_pg_num": "65536", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mds_force_trim_to": "0", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mds_skip_sanity": "false", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_memory_autotune": "true", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_memory_target": "2147483648", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_beacon_grace": "30", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_digest_period": "5", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_inactive_grace": "60", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-10T10:27:36.276 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_min_osdmap_epochs": "500", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_complaint_time": "120", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_history_duration": "600", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_history_size": "20", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_history_slow_op_size": "20", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_op_log_threshold": "5", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_auto_mark_in": "false", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-10T10:27:36.277 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_cache_size": "500", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_down_out_interval": "600", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_force_trim_to": "0", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-10T10:27:36.278 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_min_down_reporters": "2", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_report_timeout": "900", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-10T10:27:36.279 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_stuck_threshold": "60", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-10T10:27:36.280 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_probe_timeout": "2.000000", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_reweight_max_change": "0.050000", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_reweight_max_osds": "4", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_scrub_interval": "86400", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_scrub_max_keys": "100", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_scrub_timeout": "300", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_session_timeout": "300", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_smart_report_timeout": "5", 2024-09-10T10:27:36.281 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_stat_smooth_intervals": "6", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_stretch_pool_min_size": "2", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_stretch_pool_size": "4", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_debug": "false", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_provider_kill_at": "0", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_requester_kill_at": "0", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_sync_timeout": "60.000000", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_target_pg_per_osd": "100", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_tick_interval": "5", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_timecheck_interval": "300.000000", 2024-09-10T10:27:36.282 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_use_min_delay_socket": "false", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_older_version_delay": "604800", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_misplaced": "false", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_older_version": "true", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-10T10:27:36.283 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "monmap": "", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-10T10:27:36.284 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "motr_tracing_enabled": "false", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_op_threads": "3", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_cm": "false", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_device_name": "", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_dscp": "96", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_local_gid": "", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_port_num": "1", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-10T10:27:36.285 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_sl": "3", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_support_srq": "true", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_rdma_type": "ib", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_async_reap_threshold": "5", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_before_connect": "false", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_ipv4": "true", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_ipv6": "false", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_msgr1": "true", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_msgr2": "true", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_port_max": "7568", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_port_min": "6800", 2024-09-10T10:27:36.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_retry_count": "3", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_bind_retry_delay": "5", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_blackhole_client": "false", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_blackhole_mds": "false", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_blackhole_mgr": "false", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_blackhole_mon": "false", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_blackhole_osd": "false", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_client_mode": "crc secure", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_cluster_mode": "crc secure", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_cluster_type": "", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_compress_secure": "false", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_connection_idle_timeout": "900", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_connection_ready_timeout": "10", 2024-09-10T10:27:36.287 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_crc_data": "true", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_crc_header": "true", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_die_on_bad_msg": "false", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_die_on_bug": "true", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_die_on_old_message": "true", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_die_on_skipped_message": "false", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_coremask": "0xF", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_enable_tso": "true", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-10T10:27:36.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_hugepages": "", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_lro": "true", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_memory_channel": "4", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_pmd": "", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_port_id": "0", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_dump_on_send": "false", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_initial_backoff": "0.200000", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_delay_max": "1.000000", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-10T10:27:36.289 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_delay_type": "", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_network_congestion": "0", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_inject_socket_failures": "0", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_learn_addr_from_peer": "true", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_max_accept_failures": "4", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_max_backoff": "15.000000", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_mon_client_mode": "secure crc", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_mon_service_mode": "secure crc", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_osd_compress_min_size": "1024", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_osd_compress_mode": "none", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-10T10:27:36.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_pq_min_cost": "65536", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_public_type": "", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_service_mode": "crc secure", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_tcp_listen_backlog": "512", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_tcp_nodelay": "true", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_tcp_rcvbuf": "0", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "ms_type": "async+posix", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "no_config_file": "false", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_completion_locks_per_session": "32", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_inflight_ops": "1024", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-10T10:27:36.291 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_tick_interval": "5.000000", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter_timeout": "10.000000", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "objectstore_blackhole": "false", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "openssl_engine_opts": "", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_delay_time": "5.000000", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_hist_halflife": "1000", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_max_low_ops": "2", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_max_ops": "4", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_agent_slop": "0.020000", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-10T10:27:36.292 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_asio_thread_count": "2", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_async_recovery_min_cost": "100", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backfill_scan_max": "512", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backfill_scan_min": "64", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backoff_on_degraded": "false", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backoff_on_peering": "false", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_backoff_on_unfound": "true", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_beacon_report_interval": "300", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bench_duration": "30", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-10T10:27:36.293 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bench_max_block_size": "67108864", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_blkin_trace_all": "false", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_check_for_log_corruption": "false", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_class_update_on_start": "true", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_client_message_cap": "256", 2024-09-10T10:27:36.294 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_client_message_size_cap": "524288000", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_client_op_priority": "63", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_client_watch_timeout": "30", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_command_max_records": "256", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_compact_on_start": "false", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_crush_update_on_start": "true", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_crush_update_weight_set": "true", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-10T10:27:36.295 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_feed_pullee": "-1", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_misdirected_ops": "false", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_no_acting_change": "false", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_no_purge_strays": "false", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_op_order": "false", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-10T10:27:36.296 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_shutdown": "false", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_trim_objects": "false", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_verify_snaps": "false", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_keys": "1024", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-10T10:27:36.297 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_stride": "524288", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_default_notify_timeout": "30", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_delete_sleep": "0.000000", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_discard_disconnected_ops": "true", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_enable_op_tracker": "true", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-10T10:27:36.298 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_fast_info": "true", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_fast_shutdown": "true", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_function_tracing": "false", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_grace": "20", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_interval": "6", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_min_peers": "10", 2024-09-10T10:27:36.299 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_min_size": "2000", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_stale": "600", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_hit_set_max_size": "100000", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_hit_set_min_size": "1000", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_journal_size": "5120", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_kill_backfill_at": "0", 2024-09-10T10:27:36.300 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_map_cache_size": "50", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_map_dedup": "true", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_map_message_max": "40", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_map_share_max_epochs": "40", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_attr_name_len": "100", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_attr_size": "0", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_backfills": "1", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_markdown_count": "5", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_markdown_period": "600", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_object_name_len": "2048", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_object_namespace_len": "256", 2024-09-10T10:27:36.301 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_object_size": "134217728", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_pg_blocked_by": "16", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_pg_log_entries": "10000", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_pgls": "1024", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_push_cost": "8388608", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_push_objects": "10", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_scrubs": "3", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_trimming_pgs": "2", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_write_op_reply_len": "64", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_max_write_size": "90", 2024-09-10T10:27:36.302 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_profile": "balanced", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-10T10:27:36.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_base": "805306368", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_cache_min": "134217728", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_target": "4294967296", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_target_autotune": "false", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-10T10:27:36.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_min_pg_log_entries": "250", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_min_recovery_priority": "0", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mon_report_interval": "5", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_num_cache_shards": "32", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_num_op_tracker_shard": "32", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_numa_auto_affinity": "true", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_numa_node": "-1", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_numa_prefer_iface": "true", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_objecter_finishers": "1", 2024-09-10T10:27:36.305 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_objectstore": "bluestore", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_objectstore_fuse": "false", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_objectstore_tracing": "false", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_complaint_time": "30.000000", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_history_duration": "600", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_history_size": "20", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_history_slow_op_size": "20", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_log_threshold": "5", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_shards": "0", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_shards_hdd": "5", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_shards_ssd": "8", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-10T10:27:36.306 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_pq_min_cost": "65536", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_queue_cut_off": "high", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_op_thread_timeout": "15", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_open_classes_on_start": "true", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_os_flags": "0", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_peering_op_priority": "255", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_delete_cost": "1048576", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_delete_priority": "5", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-10T10:27:36.307 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_log_trim_max": "10000", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_log_trim_min": "100", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-10T10:27:36.308 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_crimson": "false", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_flags": "0", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_min_size": "0", 2024-09-10T10:27:36.309 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_pg_num": "32", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_pgp_num": "0", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_read_ratio": "70", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_size": "2", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_default_type": "replicated", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_push_per_object_cost": "1000", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recover_clone_overlap": "true", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_cost": "20971520", 2024-09-10T10:27:36.310 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_active": "0", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_max_single_start": "1", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_op_priority": "3", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_priority": "5", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_sleep": "0.000000", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-10T10:27:36.311 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_repair_during_recovery": "false", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_requested_scrub_priority": "120", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_auto_repair": "false", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_begin_hour": "0", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_begin_week_day": "0", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_chunk_max": "25", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_chunk_min": "5", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_cost": "52428800", 2024-09-10T10:27:36.312 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_during_recovery": "false", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_end_hour": "0", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_end_week_day": "0", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_event_cost": "4096", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_invalid_stats": "true", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_max_preemptions": "5", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_priority": "5", 2024-09-10T10:27:36.313 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_scrub_sleep": "0.000000", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_skip_data_digest": "false", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_smart_report_timeout": "5", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_cost": "1048576", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_priority": "5", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-10T10:27:36.314 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_target_transaction_size": "30", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-10T10:27:36.315 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_tracing": "false", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_use_stale_snap": "false", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "osdc_blkin_trace_all": "false", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_kill_at": "0", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_max_join_drift": "10", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_min": "500", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_min_wait": "0.050000", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_propose_interval": "1.000000", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_service_trim_max": "500", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_service_trim_min": "250", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_stash_full_interval": "25", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_trim_max": "500", 2024-09-10T10:27:36.316 INFO:teuthology.orchestra.run.smithi120.stdout: "paxos_trim_min": "250", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "perf": "true", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "-", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrv": "", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "public_bind_addr": "-", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "public_network": "", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "public_network_interface": "", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "qat_compressor_busy_polling": "false", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "qat_compressor_enabled": "false", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "qat_compressor_session_max_number": "256", 2024-09-10T10:27:36.317 INFO:teuthology.orchestra.run.smithi120.stdout: "rados_mon_op_timeout": "0", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rados_osd_op_timeout": "0", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rados_tracing": "false", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_atime_update_interval": "60", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_balance_parent_reads": "false", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_balance_snap_reads": "false", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_blkin_trace_all": "false", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache": "true", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-10T10:27:36.318 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_policy": "writearound", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_size": "33554432", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_clone_copy_on_read": "false", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_compression_hint": "none", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_concurrent_management_ops": "10", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_clone_format": "auto", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_data_pool": "", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_features": "61", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_format": "2", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_map_options": "", 2024-09-10T10:27:36.319 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_order": "22", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_pool": "rbd", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_stripe_count": "0", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_default_stripe_unit": "0", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_enable_alloc_hint": "true", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_io_scheduler": "simple", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-10T10:27:36.320 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_order": "24", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_pool": "", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_journal_splay_width": "4", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_localize_parent_reads": "false", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_localize_snap_reads": "false", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-10T10:27:36.321 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-10T10:27:36.322 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-10T10:27:36.323 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_mtime_update_interval": "60", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_non_blocking_aio": "true", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_op_thread_timeout": "60", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_op_threads": "1", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_parent_cache_enabled": "false", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_plugins": "", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_bps_burst": "0", 2024-09-10T10:27:36.324 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_bps_limit": "0", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_exclude_ops": "0", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_iops_burst": "0", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_iops_limit": "0", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-10T10:27:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_read_from_replica_policy": "default", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_skip_partial_discard": "true", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_tracing": "false", 2024-09-10T10:27:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_validate_names": "true", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_validate_pool": "true", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "restapi_base_url": "", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "restapi_log_level": "", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_acl_grants_max_num": "100", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_admin_entry": "admin", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_backend_store": "rados", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_barbican_url": "", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_beast_enable_async": "true", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_counters_cache": "false", 2024-09-10T10:27:36.327 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_cache_enabled": "true", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_cache_expiry_interval": "900", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_cache_lru_size": "25000", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_config_store": "rados", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_content_length_compat": "false", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_copy_obj_progress": "true", 2024-09-10T10:27:36.328 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_cors_rules_max_num": "100", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_cross_domain_policy": "", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_addr": "", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_password": "", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_kmip_username": "", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_require_ssl": "true", 2024-09-10T10:27:36.329 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-10T10:27:36.330 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_addr": "", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_auth": "token", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_namespace": "", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_prefix": "", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_token_file": "", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_curl_buffersize": "524288", 2024-09-10T10:27:36.331 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_curl_low_speed_time": "300", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_d4n_port": "6379", 2024-09-10T10:27:36.332 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_log_changes_size": "1000", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_log_num_shards": "128", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_log_window": "30", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-10T10:27:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-10T10:27:36.334 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dns_name": "", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dns_s3website_name": "", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_dynamic_resharding": "true", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_gc_threads": "true", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_lc_threads": "true", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_ops_log": "false", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_quota_threads": "true", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_static_website": "false", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enable_usage_log": "false", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_enforce_swift_acls": "true", 2024-09-10T10:27:36.335 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_exit_timeout_secs": "120", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_expose_bucket": "false", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_extended_http_attrs": "", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_filter": "none", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_frontends": "beast port=7480", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_deferred": "50", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_objs": "32", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-10T10:27:36.336 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_gc_processor_period": "3600", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_init_timeout": "300", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_kafka_connection_idle": "300", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-10T10:27:36.337 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_domain": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_password": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_password_path": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_project": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_tenant": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_token": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_token_path": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_admin_user": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_api_version": "2", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_barbican_domain": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_barbican_password": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_barbican_project": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-10T10:27:36.338 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_barbican_user": "", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_url": "", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_debug_interval": "-1", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_lock_max_time": "90", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_max_objs": "32", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_max_rules": "1000", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_max_worker": "3", 2024-09-10T10:27:36.339 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lc_thread_delay": "0", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_dnattr": "uid", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_searchfilter": "", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ldap_uri": "", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_log_http_headers": "", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-10T10:27:36.340 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_log_object_name_utc": "false", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_attr_name_len": "0", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_attr_size": "0", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_chunk_size": "4194304", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_listing_results": "1000", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_notify_retries": "10", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_put_param_size": "1048576", 2024-09-10T10:27:36.341 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_put_size": "5368709120", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_max_slo_entries": "1000", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_md_log_max_shards": "64", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_mp_lock_max_time": "600", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-10T10:27:36.342 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_max_gc": "300", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_num_async_rados_threads": "32", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_num_control_oids": "8", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_numa_node": "-1", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-10T10:27:36.343 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_objexp_chunk_size": "100", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_objexp_gc_interval": "600", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_op_thread_timeout": "600", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_op_tracing": "false", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_opa_token": "", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_opa_url": "", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_opa_verify_ssl": "true", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-10T10:27:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ops_log_rados": "false", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_ops_log_socket_path": "", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_period_push_interval": "2.000000", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_cache_lanes": "3", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_cache_partitions": "3", 2024-09-10T10:27:36.345 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-10T10:27:36.346 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_print_continue": "true", 2024-09-10T10:27:36.346 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-10T10:27:36.346 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-10T10:27:36.346 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-10T10:27:36.346 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-10T10:27:36.346 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-10T10:27:36.346 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_rados_tracing": "false", 2024-09-10T10:27:36.346 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_realm": "", 2024-09-10T10:27:36.346 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_realm_id": "", 2024-09-10T10:27:36.346 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_region": "", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_request_uri": "", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_reshard_batch_size": "64", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_reshard_max_aio": "128", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_reshard_num_logs": "16", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_reshard_thread_interval": "600", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_resolve_cname": "false", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-10T10:27:36.347 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_run_sync_thread": "true", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_scheduler_type": "throttler", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_script_uri": "", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_service_provider_name": "", 2024-09-10T10:27:36.348 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_client_id": "", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_client_secret": "", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_entry": "sts", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_key": "sts", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_min_session_duration": "900", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sts_token_introspection_url": "", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_account_in_url": "false", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_auth_entry": "auth", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_auth_url": "", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_custom_header": "", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_need_stats": "true", 2024-09-10T10:27:36.349 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_tenant_name": "", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_token_expiration": "86400", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_url": "", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_url_prefix": "swift", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_lease_period": "120", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-10T10:27:36.350 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_thread_pool_size": "512", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_comment": "", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_createby": "", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_encoding": "", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_flag": "false", 2024-09-10T10:27:36.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_origin": "", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_torrent_tracker": "", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_trust_forwarded_https": "false", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_usage_max_shards": "32", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_usage_max_user_shards": "1", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_use_opa_authz": "false", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_counters_cache": "false", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-10T10:27:36.352 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_max_buckets": "1000", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_policies_max_num": "100", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_user_unique_email": "true", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_verify_ssl": "true", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zone": "", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zone_id": "", 2024-09-10T10:27:36.353 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zonegroup": "", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zonegroup_id": "", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rgwlc_auto_session_clear": "true", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_block_size": "4096", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-10T10:27:36.354 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_size": "536870912", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_index_type": "binary_search", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_partition_filters": "false", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_perf": "false", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-10T10:27:36.355 INFO:teuthology.orchestra.run.smithi120.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "run_dir": "/var/run/ceph", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_block_create": "true", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_cache_lru_size": "67108864", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_default_max_object_size": "16777216", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_device_size": "53687091200", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_full_integrity_check": "false", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_journal_batch_capacity": "16", 2024-09-10T10:27:36.356 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_main_device_type": "SSD", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "seastore_segment_size": "67108864", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "set_keepcaps": "false", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "setgroup": "", 2024-09-10T10:27:36.357 INFO:teuthology.orchestra.run.smithi120.stdout: "setuser": "", 2024-09-10T10:27:36.358 INFO:teuthology.orchestra.run.smithi120.stdout: "setuser_match_path": "", 2024-09-10T10:27:36.358 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-10T10:27:36.358 INFO:teuthology.orchestra.run.smithi120.stdout: "thp": "false", 2024-09-10T10:27:36.358 INFO:teuthology.orchestra.run.smithi120.stdout: "threadpool_default_timeout": "60", 2024-09-10T10:27:36.358 INFO:teuthology.orchestra.run.smithi120.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-10T10:27:36.358 INFO:teuthology.orchestra.run.smithi120.stdout: "throttler_perf_counter": "true" 2024-09-10T10:27:36.358 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:36.360 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-10T10:27:36.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.454+0000 7fc2ddcf1640 1 Processor -- start 2024-09-10T10:27:36.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.454+0000 7fc2ddcf1640 1 -- start start 2024-09-10T10:27:36.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2ddcf1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8154c10 0x7fc2d815b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2ddcf1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc2d80cca70 0x7fc2d8073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2ddcf1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2d815e0b0 con 0x7fc2d80cafa0 2024-09-10T10:27:36.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2ddcf1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc2d815de60 con 0x7fc2d80cca70 2024-09-10T10:27:36.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2ddcf1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d815e390 con 0x7fc2d8154c10 2024-09-10T10:27:36.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2d7fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fc2d80cafa0 0x7fc2d80cb6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:33618/0 (socket says 172.21.15.120:33618) 2024-09-10T10:27:36.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2d7fff640 1 -- 172.21.15.120:0/4083229833 learned_addr learned my addr 172.21.15.120:0/4083229833 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2d6ffd640 1 --2- 172.21.15.120:0/4083229833 >> v2:172.21.15.134:3300/0 conn(0x7fc2d80cca70 0x7fc2d8073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2d77fe640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8154c10 0x7fc2d815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 129065447 0 0) 0x7fc2d815e0b0 con 0x7fc2d80cafa0 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.455+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2c0003520 con 0x7fc2d80cafa0 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1612528667 0 0) 0x7fc2c0003520 con 0x7fc2d80cafa0 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8154c10 msgr2=0x7fc2d815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8154c10 0x7fc2d815b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 >> v2:172.21.15.134:3300/0 conn(0x7fc2d80cca70 msgr2=0x7fc2d8073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 --2- 172.21.15.120:0/4083229833 >> v2:172.21.15.134:3300/0 conn(0x7fc2d80cca70 0x7fc2d8073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d815e670 con 0x7fc2d80cafa0 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7fc2d0001670 con 0x7fc2d80cafa0 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 >> v1:172.21.15.120:6789/0 conn(0x7fc2d80cafa0 legacy=0x7fc2d80cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8154c10 0x7fc2c00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c0003f30 0x7fc2c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c0004d40 0x7fc2c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d815de60 con 0x7fc2c0003f30 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d815e390 con 0x7fc2c0004d40 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c000b9c0 con 0x7fc2d8154c10 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d6ffd640 1 --2- 172.21.15.120:0/4083229833 >> v2:172.21.15.134:3300/0 conn(0x7fc2d80cca70 0x7fc2d8073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d77fe640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8154c10 0x7fc2c00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d6ffd640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c0003f30 0x7fc2c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c0004d40 0x7fc2c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d6ffd640 1 -- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8154c10 msgr2=0x7fc2c00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d6ffd640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8154c10 0x7fc2c00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d6ffd640 1 -- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c0004d40 msgr2=0x7fc2c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d6ffd640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c0004d40 0x7fc2c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.456+0000 7fc2d6ffd640 1 -- 172.21.15.120:0/4083229833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c0003520 con 0x7fc2c0003f30 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.457+0000 7fc2d6ffd640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c0003f30 0x7fc2c0004800 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009c50 tx=0x7fc2c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35dc5719d2443610 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.457+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc036070 con 0x7fc2c0003f30 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.457+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2cc031400 con 0x7fc2c0003f30 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.457+0000 7fc2d67fc640 1 -- 172.21.15.120:0/4083229833 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc009850 con 0x7fc2c0003f30 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.458+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/4083229833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c0003f30 msgr2=0x7fc2c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.458+0000 7fc2ddcf1640 1 --2- 172.21.15.120:0/4083229833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c0003f30 0x7fc2c0004800 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009c50 tx=0x7fc2c0005830 comp rx=0 tx=0).stop 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.458+0000 7fc2d7fff640 1 -- 172.21.15.120:0/4083229833 reap_dead start 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.458+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/4083229833 shutdown_connections 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.458+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/4083229833 >> 172.21.15.120:0/4083229833 conn(0x7fc2d806ce80 msgr2=0x7fc2d806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.458+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/4083229833 shutdown_connections 2024-09-10T10:27:36.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.458+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/4083229833 wait complete. 2024-09-10T10:27:36.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.459+0000 7fc2ddcf1640 1 Processor -- start 2024-09-10T10:27:36.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.459+0000 7fc2ddcf1640 1 -- start start 2024-09-10T10:27:36.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2ddcf1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c0003f30 0x7fc2d80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2ddcf1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d8157500 0x7fc2d80c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2ddcf1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8157a50 0x7fc2d80b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2ddcf1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8154ff0 con 0x7fc2d8157500 2024-09-10T10:27:36.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2ddcf1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d815ec30 con 0x7fc2c0003f30 2024-09-10T10:27:36.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2ddcf1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d80cbe00 con 0x7fc2d8157a50 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d8157500 0x7fc2d80c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d8157500 0x7fc2d80c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38278/0 (socket says 172.21.15.120:38278) 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d6ffd640 1 -- 172.21.15.120:0/582181675 learned_addr learned my addr 172.21.15.120:0/582181675 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d77fe640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c0003f30 0x7fc2d80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8157a50 0x7fc2d80b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d6ffd640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8157a50 msgr2=0x7fc2d80b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d6ffd640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d8157a50 0x7fc2d80b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d6ffd640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c0003f30 msgr2=0x7fc2d80c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d6ffd640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c0003f30 0x7fc2d80c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d6ffd640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d8158300 con 0x7fc2d8157500 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2d6ffd640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d8157500 0x7fc2d80c9aa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009d70 tx=0x7fc2cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc040020 con 0x7fc2d8157500 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2cc006470 con 0x7fc2d8157500 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc036040 con 0x7fc2d8157500 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d81f1430 con 0x7fc2d8157500 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.460+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d81f0670 con 0x7fc2d8157500 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.461+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc2d815b370 con 0x7fc2d8157500 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.462+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2cc043020 con 0x7fc2d8157500 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.462+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc2c406be30 0x7fc2c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.462+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fc2cc0ab910 con 0x7fc2d8157500 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.462+0000 7fc2d77fe640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc2c406be30 0x7fc2c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.463+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7fc2cc04a020 con 0x7fc2d8157500 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.463+0000 7fc2ddcf1640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d80b42f0 0x7fc2d81f0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.463+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81521d0 con 0x7fc2d80b42f0 2024-09-10T10:27:36.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.463+0000 7fc2d77fe640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc2c406be30 0x7fc2c406e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80133c0 tx=0x7fc2c800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.463+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d80b42f0 0x7fc2d81f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.463+0000 7fc2d7fff640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7fc2d80cf5c0 con 0x7fc2d80b42f0 2024-09-10T10:27:36.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.463+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d80b42f0 0x7fc2d81f0840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000bb40 tx=0x7fc2d0007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.463+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0019070 con 0x7fc2d80b42f0 2024-09-10T10:27:36.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.464+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.0 v2:172.21.15.120:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc2d0013570 con 0x7fc2d80b42f0 2024-09-10T10:27:36.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.464+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d80b42f0 msgr2=0x7fc2d81f0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.464+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d80b42f0 0x7fc2d81f0840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000bb40 tx=0x7fc2d0007430 comp rx=0 tx=0).stop 2024-09-10T10:27:36.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.464+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d80b42f0 msgr2=0x7fc2d81f0840 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:27:36.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.464+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d80b42f0 0x7fc2d81f0840 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.470+0000 7fc2ddcf1640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d81f0d80 0x7fc2d8150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.470+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d80c6d30 con 0x7fc2d81f0d80 2024-09-10T10:27:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.470+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d81f0d80 0x7fc2d8150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.470+0000 7fc2d7fff640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7fc2d80c9090 con 0x7fc2d81f0d80 2024-09-10T10:27:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.470+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d81f0d80 0x7fc2d8150f40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0000c00 tx=0x7fc2d0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.470+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0019040 con 0x7fc2d81f0d80 2024-09-10T10:27:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.471+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.0 v2:172.21.15.120:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fc2d001bab0 con 0x7fc2d81f0d80 2024-09-10T10:27:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.471+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d81f0d80 msgr2=0x7fc2d8150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.471+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d81f0d80 0x7fc2d8150f40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0000c00 tx=0x7fc2d0000fe0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.471+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d81f0d80 msgr2=0x7fc2d8150f40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:27:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.471+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d81f0d80 0x7fc2d8150f40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.474 INFO:teuthology.orchestra.run.smithi120.stdout:mon.a: {} 2024-09-10T10:27:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T10:27:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.472+0000 7fc2ddcf1640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d8151480 0x7fc2d81f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.472+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d80c9fe0 con 0x7fc2d8151480 2024-09-10T10:27:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.472+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d8151480 0x7fc2d81f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.472+0000 7fc2d7fff640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7fc2d8151890 con 0x7fc2d8151480 2024-09-10T10:27:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.472+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d8151480 0x7fc2d81f3a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0005fd0 tx=0x7fc2d0007430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.472+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0029070 con 0x7fc2d8151480 2024-09-10T10:27:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.473+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc2d0024410 con 0x7fc2d8151480 2024-09-10T10:27:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.473+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d8151480 msgr2=0x7fc2d81f3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.473+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d8151480 0x7fc2d81f3a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0005fd0 tx=0x7fc2d0007430 comp rx=0 tx=0).stop 2024-09-10T10:27:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.473+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d8151480 msgr2=0x7fc2d81f3a10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:27:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.473+0000 7fc2d7fff640 1 -- 172.21.15.120:0/582181675 reap_dead start 2024-09-10T10:27:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.473+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d8151480 0x7fc2d81f3a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.479+0000 7fc2ddcf1640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d80c8b50 0x7fc2d80b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.479+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81f3f50 con 0x7fc2d80c8b50 2024-09-10T10:27:36.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.479+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d80c8b50 0x7fc2d80b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.479+0000 7fc2d7fff640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7fc2d81f40d0 con 0x7fc2d80c8b50 2024-09-10T10:27:36.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.479+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d80c8b50 0x7fc2d80b3db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0000c00 tx=0x7fc2d0024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.479+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0019070 con 0x7fc2d80c8b50 2024-09-10T10:27:36.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.481+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fc2d0037880 con 0x7fc2d80c8b50 2024-09-10T10:27:36.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.481+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d80c8b50 msgr2=0x7fc2d80b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.481+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d80c8b50 0x7fc2d80b3db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0000c00 tx=0x7fc2d0024000 comp rx=0 tx=0).stop 2024-09-10T10:27:36.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.481+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d80c8b50 msgr2=0x7fc2d80b3db0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:27:36.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.481+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d80c8b50 0x7fc2d80b3db0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.484 INFO:teuthology.orchestra.run.smithi120.stdout:mon.b: {} 2024-09-10T10:27:36.484 INFO:teuthology.orchestra.run.smithi120.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T10:27:36.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.481+0000 7fc2ddcf1640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d81f42e0 0x7fc2d815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.481+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d81f46c0 con 0x7fc2d81f42e0 2024-09-10T10:27:36.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.482+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d81f42e0 0x7fc2d815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.482+0000 7fc2d7fff640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fc2d81f4840 con 0x7fc2d81f42e0 2024-09-10T10:27:36.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.482+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d81f42e0 0x7fc2d815ba20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0013e30 tx=0x7fc2d0007430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.483+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d004f070 con 0x7fc2d81f42e0 2024-09-10T10:27:36.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.483+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc2d0030670 con 0x7fc2d81f42e0 2024-09-10T10:27:36.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.483+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d81f42e0 msgr2=0x7fc2d815ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.483+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d81f42e0 0x7fc2d815ba20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0013e30 tx=0x7fc2d0007430 comp rx=0 tx=0).stop 2024-09-10T10:27:36.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.483+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d81f42e0 msgr2=0x7fc2d815ba20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:27:36.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.483+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d81f42e0 0x7fc2d815ba20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.488+0000 7fc2ddcf1640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d815bf60 0x7fc2d815c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.488+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d815c8b0 con 0x7fc2d815bf60 2024-09-10T10:27:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.489+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d815bf60 0x7fc2d815c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.489+0000 7fc2d7fff640 1 -- 172.21.15.120:0/582181675 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7fc2d8203f50 con 0x7fc2d815bf60 2024-09-10T10:27:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.489+0000 7fc2d7fff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d815bf60 0x7fc2d815c370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0007f30 tx=0x7fc2d0040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.489+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d004f070 con 0x7fc2d815bf60 2024-09-10T10:27:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fc2d0051620 con 0x7fc2d815bf60 2024-09-10T10:27:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d815bf60 msgr2=0x7fc2d815c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d815bf60 0x7fc2d815c370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0007f30 tx=0x7fc2d0040000 comp rx=0 tx=0).stop 2024-09-10T10:27:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2bffff640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d815bf60 msgr2=0x7fc2d815c370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:27:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2bffff640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d815bf60 0x7fc2d815c370 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.494 INFO:teuthology.orchestra.run.smithi120.stdout:mon.c: {} 2024-09-10T10:27:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T10:27:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc2c406be30 msgr2=0x7fc2c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2ddcf1640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc2c406be30 0x7fc2c406e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80133c0 tx=0x7fc2c800b040 comp rx=0 tx=0).stop 2024-09-10T10:27:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d8157500 msgr2=0x7fc2d80c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2ddcf1640 1 --2- 172.21.15.120:0/582181675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2d8157500 0x7fc2d80c9aa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009d70 tx=0x7fc2cc004730 comp rx=0 tx=0).stop 2024-09-10T10:27:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2d7fff640 1 -- 172.21.15.120:0/582181675 reap_dead start 2024-09-10T10:27:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 shutdown_connections 2024-09-10T10:27:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.491+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 >> 172.21.15.120:0/582181675 conn(0x7fc2d806ce80 msgr2=0x7fc2d8152b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.492+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 shutdown_connections 2024-09-10T10:27:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.492+0000 7fc2ddcf1640 1 -- 172.21.15.120:0/582181675 wait complete. 2024-09-10T10:27:36.504 DEBUG:teuthology.orchestra.run.smithi134:> type ceph-objectstore-tool 2024-09-10T10:27:36.522 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-10T10:27:36.522 DEBUG:teuthology.orchestra.run.smithi120:> type ceph-objectstore-tool 2024-09-10T10:27:36.537 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-10T10:27:36.537 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T10:27:36.551 INFO:tasks.workunit:Pulling workunits from ref 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:27:36.551 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T10:27:36.551 INFO:tasks.workunit:timeout=3h 2024-09-10T10:27:36.551 INFO:tasks.workunit:cleanup=True 2024-09-10T10:27:36.551 DEBUG:teuthology.orchestra.run.smithi120:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T10:27:36.552 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-10T10:27:36.552 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:27:36.552 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:27:36.552 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-10T10:27:36.552 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-10T10:27:36.552 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-10T10:27:36.552 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-10T10:27:36.553 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-10T10:27:36.553 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-10T10:27:36.553 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:27:36.553 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-10T10:27:36.553 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:27:36.554 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-10T10:27:36.554 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-10T10:27:36.564 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:36.563+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:36.595 INFO:teuthology.orchestra.run.smithi120.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T10:27:36.595 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-10T10:27:36.596 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 29h/41d Inode: 1 Links: 2 2024-09-10T10:27:36.596 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:27:36.596 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-10T10:27:36.596 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-10T10:27:36.596 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 10:26:46.776591779 +0000 2024-09-10T10:27:36.596 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 10:26:54.050143643 +0000 2024-09-10T10:27:36.596 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 10:26:46.776591779 +0000 2024-09-10T10:27:36.596 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T10:27:36.596 DEBUG:teuthology.orchestra.run.smithi120:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T10:27:36.655 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:36.654+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:36.679 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:27:36.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.683+0000 7fbe3ffdf640 1 Processor -- start 2024-09-10T10:27:36.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.683+0000 7fbe3ffdf640 1 -- start start 2024-09-10T10:27:36.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3ffdf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe38154d50 0x7fbe3815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3ffdf640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fbe3815ba10 0x7fbe38162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3ffdf640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe3806cf70 con 0x7fbe380c71b0 2024-09-10T10:27:36.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3ffdf640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fbe3806cd20 con 0x7fbe3815ba10 2024-09-10T10:27:36.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3ffdf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbe381578a0 con 0x7fbe38154d50 2024-09-10T10:27:36.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3d57a640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fbe380c71b0 0x7fbe380ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:33620/0 (socket says 172.21.15.120:33620) 2024-09-10T10:27:36.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7f4220842640 1 Processor -- start 2024-09-10T10:27:36.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3e57c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fbe3815ba10 0x7fbe38162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3d57a640 1 -- 172.21.15.120:0/4199339954 learned_addr learned my addr 172.21.15.120:0/4199339954 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:36.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3dd7b640 1 --2- 172.21.15.120:0/4199339954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe38154d50 0x7fbe3815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7f4220842640 1 -- start start 2024-09-10T10:27:36.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3e57c640 1 -- 172.21.15.120:0/4199339954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe38154d50 msgr2=0x7fbe3815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:36.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3e57c640 1 --2- 172.21.15.120:0/4199339954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe38154d50 0x7fbe3815b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3e57c640 1 -- 172.21.15.120:0/4199339954 >> v1:172.21.15.120:6789/0 conn(0x7fbe380c71b0 legacy=0x7fbe380ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3e57c640 1 -- 172.21.15.120:0/4199339954 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3806d250 con 0x7fbe3815ba10 2024-09-10T10:27:36.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.685+0000 7fbe3cd79640 1 -- 172.21.15.120:0/4199339954 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1336632889 0 0) 0x7fbe3806cf70 con 0x7fbe380c71b0 2024-09-10T10:27:36.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.686+0000 7f4220842640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06cf90 0x7f421c0c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.686+0000 7f4220842640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f421c06d370 0x7f421c1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.686+0000 7f4220842640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421c0c4890 con 0x7f421c0c7b30 2024-09-10T10:27:36.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.686+0000 7f4220842640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f421c0c8fc0 con 0x7f421c06d370 2024-09-10T10:27:36.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.686+0000 7f4220842640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f421c0c4630 con 0x7f421c06cf90 2024-09-10T10:27:36.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7f421a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06cf90 0x7f421c0c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7fbe3e57c640 1 --2- 172.21.15.120:0/4199339954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe3815ba10 0x7fbe38162140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009e40 tx=0x7fbe3002cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402b6c5dc50273f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7f4219d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f421c06d370 0x7f421c1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7f421a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06cf90 0x7f421c0c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:55008/0 (socket says 172.21.15.120:55008) 2024-09-10T10:27:36.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7f421a59c640 1 -- 172.21.15.120:0/2018818652 learned_addr learned my addr 172.21.15.120:0/2018818652 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:36.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7fbe3cd79640 1 -- 172.21.15.120:0/4199339954 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3002e8d0 con 0x7fbe3815ba10 2024-09-10T10:27:36.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7fbe3cd79640 1 -- 172.21.15.120:0/4199339954 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe3002ea70 con 0x7fbe3815ba10 2024-09-10T10:27:36.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7fbe3cd79640 1 -- 172.21.15.120:0/4199339954 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3002ee40 con 0x7fbe3815ba10 2024-09-10T10:27:36.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7f4219d9b640 1 -- 172.21.15.120:0/2018818652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06cf90 msgr2=0x7f421c0c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:36.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7f4219d9b640 1 --2- 172.21.15.120:0/2018818652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06cf90 0x7f421c0c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7f4219d9b640 1 -- 172.21.15.120:0/2018818652 >> v1:172.21.15.134:6789/0 conn(0x7f421c0c7b30 legacy=0x7f421c06ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.687+0000 7f4219d9b640 1 -- 172.21.15.120:0/2018818652 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c070d90 con 0x7f421c06d370 2024-09-10T10:27:36.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/4199339954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe3815ba10 msgr2=0x7fbe38162140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/4199339954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe3815ba10 0x7fbe38162140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009e40 tx=0x7fbe3002cbb0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7f4219d9b640 1 --2- 172.21.15.120:0/2018818652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f421c06d370 0x7f421c1514f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f421000dbc0 tx=0x7f421002e490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7de75650460bee29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/4199339954 shutdown_connections 2024-09-10T10:27:36.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/4199339954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe3815ba10 0x7fbe38162140 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/4199339954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe38154d50 0x7fbe3815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/4199339954 >> 172.21.15.120:0/4199339954 conn(0x7fbe380728b0 msgr2=0x7fbe38072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:36.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7f421959a640 1 -- 172.21.15.120:0/2018818652 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421003a070 con 0x7f421c06d370 2024-09-10T10:27:36.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7f421959a640 1 -- 172.21.15.120:0/2018818652 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421000b650 con 0x7f421c06d370 2024-09-10T10:27:36.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7f421959a640 1 -- 172.21.15.120:0/2018818652 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421000ba20 con 0x7f421c06d370 2024-09-10T10:27:36.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/4199339954 shutdown_connections 2024-09-10T10:27:36.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.688+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/4199339954 wait complete. 2024-09-10T10:27:36.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.689+0000 7f4220842640 1 -- 172.21.15.120:0/2018818652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f421c06d370 msgr2=0x7f421c1514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.689+0000 7f4220842640 1 --2- 172.21.15.120:0/2018818652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f421c06d370 0x7f421c1514f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f421000dbc0 tx=0x7f421002e490 comp rx=0 tx=0).stop 2024-09-10T10:27:36.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.689+0000 7f4220842640 1 -- 172.21.15.120:0/2018818652 shutdown_connections 2024-09-10T10:27:36.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.689+0000 7f4220842640 1 --2- 172.21.15.120:0/2018818652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f421c06d370 0x7f421c1514f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.689+0000 7f4220842640 1 --2- 172.21.15.120:0/2018818652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06cf90 0x7f421c0c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.689+0000 7f4220842640 1 -- 172.21.15.120:0/2018818652 >> 172.21.15.120:0/2018818652 conn(0x7f421c067d40 msgr2=0x7f421c070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:36.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.689+0000 7f4220842640 1 -- 172.21.15.120:0/2018818652 shutdown_connections 2024-09-10T10:27:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.690+0000 7f4220842640 1 -- 172.21.15.120:0/2018818652 wait complete. 2024-09-10T10:27:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.691+0000 7fbe3ffdf640 1 Processor -- start 2024-09-10T10:27:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.691+0000 7fbe3ffdf640 1 -- start start 2024-09-10T10:27:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.691+0000 7f4220842640 1 Processor -- start 2024-09-10T10:27:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.691+0000 7f4220842640 1 -- start start 2024-09-10T10:27:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f4220842640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f421c06cf90 0x7f421c1540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f4220842640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06d370 0x7f421c1ed020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f4220842640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f421c0c7b30 0x7f421c143a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f4220842640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f421c0c8660 con 0x7f421c06cf90 2024-09-10T10:27:36.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f4220842640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f421c0c8380 con 0x7f421c0c7b30 2024-09-10T10:27:36.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f4220842640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f421c0c8940 con 0x7f421c06d370 2024-09-10T10:27:36.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f421a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f421c06cf90 0x7f421c1540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f421a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f421c06cf90 0x7f421c1540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38306/0 (socket says 172.21.15.120:38306) 2024-09-10T10:27:36.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f421a59c640 1 -- 172.21.15.120:0/900500347 learned_addr learned my addr 172.21.15.120:0/900500347 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:36.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f4219d9b640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06d370 0x7f421c1ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f421ad9d640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f421c0c7b30 0x7f421c143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7fbe3ffdf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe380c71b0 0x7fbe380cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f421a59c640 1 -- 172.21.15.120:0/900500347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06d370 msgr2=0x7f421c1ed020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f421a59c640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06d370 0x7f421c1ed020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f421a59c640 1 -- 172.21.15.120:0/900500347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f421c0c7b30 msgr2=0x7f421c143a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f421a59c640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f421c0c7b30 0x7f421c143a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7f421a59c640 1 -- 172.21.15.120:0/900500347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c143f50 con 0x7f421c06cf90 2024-09-10T10:27:36.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7fbe3ffdf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe38154d50 0x7fbe380cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7fbe3dd7b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe380c71b0 0x7fbe380cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.692+0000 7fbe3dd7b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe380c71b0 0x7fbe380cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:55036/0 (socket says 172.21.15.120:55036) 2024-09-10T10:27:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7f421a59c640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f421c06cf90 0x7f421c1540e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f420c00fad0 tx=0x7f420c00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3d57a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe38154d50 0x7fbe380cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3d57a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe38154d50 0x7fbe380cbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38314/0 (socket says 172.21.15.120:38314) 2024-09-10T10:27:36.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3dd7b640 1 -- 172.21.15.120:0/1464218880 learned_addr learned my addr 172.21.15.120:0/1464218880 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:36.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3815ba10 0x7fbe38151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3815afb0 con 0x7fbe38154d50 2024-09-10T10:27:36.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7f420affd640 1 -- 172.21.15.120:0/900500347 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c01d070 con 0x7f421c06cf90 2024-09-10T10:27:36.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7f420affd640 1 -- 172.21.15.120:0/900500347 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c018440 con 0x7f421c06cf90 2024-09-10T10:27:36.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7f420affd640 1 -- 172.21.15.120:0/900500347 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c018810 con 0x7f421c06cf90 2024-09-10T10:27:36.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7f4220842640 1 -- 172.21.15.120:0/900500347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c152690 con 0x7f421c06cf90 2024-09-10T10:27:36.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3e57c640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3815ba10 0x7fbe38151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3806d810 con 0x7fbe3815ba10 2024-09-10T10:27:36.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3806eae0 con 0x7fbe380c71b0 2024-09-10T10:27:36.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7f4220842640 1 -- 172.21.15.120:0/900500347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421c146920 con 0x7f421c06cf90 2024-09-10T10:27:36.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3d57a640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe380c71b0 msgr2=0x7fbe380cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3d57a640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe380c71b0 0x7fbe380cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3d57a640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3815ba10 msgr2=0x7fbe38151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3d57a640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3815ba10 0x7fbe38151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.693+0000 7fbe3d57a640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe38159b60 con 0x7fbe38154d50 2024-09-10T10:27:36.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.694+0000 7fbe3dd7b640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe380c71b0 0x7fbe380cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:36.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.694+0000 7fbe3e57c640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3815ba10 0x7fbe38151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:36.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.694+0000 7fbe3d57a640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe38154d50 0x7fbe380cbc80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe28007fd0 tx=0x7fbe2800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.694+0000 7f4220842640 1 -- 172.21.15.120:0/900500347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421c0cd080 con 0x7f421c06cf90 2024-09-10T10:27:36.706 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:36.707 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:36.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.694+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28017070 con 0x7fbe38154d50 2024-09-10T10:27:36.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.695+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe28007500 con 0x7fbe38154d50 2024-09-10T10:27:36.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.695+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280059e0 con 0x7fbe38154d50 2024-09-10T10:27:36.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.695+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe38159e40 con 0x7fbe38154d50 2024-09-10T10:27:36.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.695+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe380caf40 con 0x7fbe38154d50 2024-09-10T10:27:36.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.695+0000 7f420affd640 1 -- 172.21.15.120:0/900500347 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f420c020070 con 0x7f421c06cf90 2024-09-10T10:27:36.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.696+0000 7f420affd640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f41fc06be30 0x7f41fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.696+0000 7f420affd640 1 -- 172.21.15.120:0/900500347 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f420c08b800 con 0x7f421c06cf90 2024-09-10T10:27:36.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.696+0000 7f4219d9b640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f41fc06be30 0x7f41fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.697+0000 7f4219d9b640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f41fc06be30 0x7f41fc06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4210037000 tx=0x7f421000d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.696+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fbe38154540 con 0x7fbe38154d50 2024-09-10T10:27:36.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.697+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe2801a070 con 0x7fbe38154d50 2024-09-10T10:27:36.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.698+0000 7fbe2e7fc640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbe1c06beb0 0x7fbe1c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.698+0000 7fbe3dd7b640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbe1c06beb0 0x7fbe1c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.698+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fbe28056c20 con 0x7fbe38154d50 2024-09-10T10:27:36.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.698+0000 7fbe3dd7b640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbe1c06beb0 0x7fbe1c06e370 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbe34006fd0 tx=0x7fbe34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.710 INFO:tasks.workunit.client.0.smithi120.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T10:27:36.710 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-10T10:27:36.710 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:36.711 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:27:36.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.706+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v28) ==== 54+0+16 (secure 0 0 0) 0x7fbe28021070 con 0x7fbe38154d50 2024-09-10T10:27:36.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.706+0000 7f420affd640 1 -- 172.21.15.120:0/900500347 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f420c0596d0 con 0x7f421c06cf90 2024-09-10T10:27:36.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.706+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fbe38153a20 0x7fbe381524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.706+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe38151da0 con 0x7fbe38153a20 2024-09-10T10:27:36.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.706+0000 7fbe3e57c640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fbe38153a20 0x7fbe381524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.707+0000 7fbe3e57c640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fbe38153a20 0x7fbe381524d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.708+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe38151da0 con 0x7fbe38153a20 2024-09-10T10:27:36.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.735+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbe38157ac0 con 0x7fbe38153a20 2024-09-10T10:27:36.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.736+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbe38157ac0 con 0x7fbe38153a20 2024-09-10T10:27:36.738 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:36.738 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:36.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.736+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fbe380cb110 0x7fbe381f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.736+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fbe381f8c20 con 0x7fbe380cb110 2024-09-10T10:27:36.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.736+0000 7fbe3dd7b640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fbe380cb110 0x7fbe381f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.736+0000 7fbe3dd7b640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fbe380cb110 0x7fbe381f86e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.737+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe381f8c20 con 0x7fbe380cb110 2024-09-10T10:27:36.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.752+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbe38159740 con 0x7fbe380cb110 2024-09-10T10:27:36.755 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:36.753+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:36.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.754+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbe38159740 con 0x7fbe380cb110 2024-09-10T10:27:36.756 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:36.757 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:36.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.754+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fbe381f5150 0x7fbe38062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.754+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fbe381f5550 con 0x7fbe381f5150 2024-09-10T10:27:36.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.754+0000 7fbe3d57a640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fbe381f5150 0x7fbe38062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.755+0000 7fbe3d57a640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fbe381f5150 0x7fbe38062630 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.755+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe381f5550 con 0x7fbe381f5150 2024-09-10T10:27:36.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.772+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbe38062b70 con 0x7fbe381f5150 2024-09-10T10:27:36.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.774+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbe38062b70 con 0x7fbe381f5150 2024-09-10T10:27:36.776 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:36.777 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:36.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.775+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fbe38155af0 0x7fbe38155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.775+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fbe38062b70 con 0x7fbe38155af0 2024-09-10T10:27:36.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.775+0000 7fbe3e57c640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fbe38155af0 0x7fbe38155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.775+0000 7fbe3e57c640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fbe38155af0 0x7fbe38155f30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.776+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe38062b70 con 0x7fbe38155af0 2024-09-10T10:27:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.794+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbe38062b70 con 0x7fbe38155af0 2024-09-10T10:27:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.796+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbe38062b70 con 0x7fbe38155af0 2024-09-10T10:27:36.797 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.796+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fbe38156830 0x7fbe3802cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.797+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fbe38062b70 con 0x7fbe38156830 2024-09-10T10:27:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.797+0000 7fbe3dd7b640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fbe38156830 0x7fbe3802cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.797+0000 7fbe3dd7b640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fbe38156830 0x7fbe3802cb50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.798+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe38062b70 con 0x7fbe38156830 2024-09-10T10:27:36.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.814+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbe38062b70 con 0x7fbe38156830 2024-09-10T10:27:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.816+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbe38062b70 con 0x7fbe38156830 2024-09-10T10:27:36.817 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.816+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fbe3802d5a0 0x7fbe3820b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.816+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fbe3802da20 con 0x7fbe3802d5a0 2024-09-10T10:27:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.817+0000 7fbe3d57a640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fbe3802d5a0 0x7fbe3820b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.817+0000 7fbe3d57a640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fbe3802d5a0 0x7fbe3820b2f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.818+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe3802da20 con 0x7fbe3802d5a0 2024-09-10T10:27:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.837+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbe3802da20 con 0x7fbe3802d5a0 2024-09-10T10:27:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.839+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbe3802da20 con 0x7fbe3802d5a0 2024-09-10T10:27:36.840 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.839+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fbe3820bd40 0x7fbe3820e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.840+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fbe38210f00 con 0x7fbe3820bd40 2024-09-10T10:27:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.840+0000 7fbe3e57c640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fbe3820bd40 0x7fbe3820e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.840+0000 7fbe3e57c640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fbe3820bd40 0x7fbe3820e130 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.841+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe38210f00 con 0x7fbe3820bd40 2024-09-10T10:27:36.848 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:36.849 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:36.856 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:36.855+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:36.858 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:27:36.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.863+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbe3820e670 con 0x7fbe3820bd40 2024-09-10T10:27:36.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.865+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbe3820e670 con 0x7fbe3820bd40 2024-09-10T10:27:36.867 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:36.868 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:36.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.866+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fbe3820ecf0 0x7fbe382110e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:36.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.866+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fbe38213eb0 con 0x7fbe3820ecf0 2024-09-10T10:27:36.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.866+0000 7fbe3dd7b640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fbe3820ecf0 0x7fbe382110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:36.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.866+0000 7fbe3dd7b640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fbe3820ecf0 0x7fbe382110e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:36.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.867+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe38213eb0 con 0x7fbe3820ecf0 2024-09-10T10:27:36.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.881+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbe382116a0 con 0x7fbe3820ecf0 2024-09-10T10:27:36.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.883+0000 7fbe2e7fc640 1 -- 172.21.15.120:0/1464218880 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbe382116a0 con 0x7fbe3820ecf0 2024-09-10T10:27:36.884 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:36.886 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:36.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fbe38153a20 msgr2=0x7fbe381524d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fbe38153a20 0x7fbe381524d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fbe380cb110 msgr2=0x7fbe381f86e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fbe380cb110 0x7fbe381f86e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fbe381f5150 msgr2=0x7fbe38062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fbe381f5150 0x7fbe38062630 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3e57c640 1 -- 172.21.15.120:0/1464218880 reap_dead start 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fbe38155af0 msgr2=0x7fbe38155f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fbe38155af0 0x7fbe38155f30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fbe38156830 msgr2=0x7fbe3802cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fbe38156830 0x7fbe3802cb50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fbe3802d5a0 msgr2=0x7fbe3820b2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fbe3802d5a0 0x7fbe3820b2f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fbe3820bd40 msgr2=0x7fbe3820e130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fbe3820bd40 0x7fbe3820e130 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fbe3820ecf0 msgr2=0x7fbe382110e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fbe3820ecf0 0x7fbe382110e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3e57c640 1 -- 172.21.15.120:0/1464218880 reap_dead start 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbe1c06beb0 msgr2=0x7fbe1c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbe1c06beb0 0x7fbe1c06e370 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbe34006fd0 tx=0x7fbe34008040 comp rx=0 tx=0).stop 2024-09-10T10:27:36.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe38154d50 msgr2=0x7fbe380cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe38154d50 0x7fbe380cbc80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe28007fd0 tx=0x7fbe2800c870 comp rx=0 tx=0).stop 2024-09-10T10:27:36.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 shutdown_connections 2024-09-10T10:27:36.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbe1c06beb0 0x7fbe1c06e370 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 --2- 172.21.15.120:0/1464218880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe38154d50 0x7fbe380cbc80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 >> 172.21.15.120:0/1464218880 conn(0x7fbe380728b0 msgr2=0x7fbe38072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:36.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 shutdown_connections 2024-09-10T10:27:36.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.884+0000 7fbe3ffdf640 1 -- 172.21.15.120:0/1464218880 wait complete. 2024-09-10T10:27:36.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.916+0000 7f4220842640 1 -- 172.21.15.120:0/900500347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f421c1587c0 con 0x7f421c06cf90 2024-09-10T10:27:36.957 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:36.956+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:36.971 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:36.971 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:36.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.973+0000 7f420affd640 1 -- 172.21.15.120:0/900500347 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v29) ==== 87+0+0 (secure 0 0 0) 0x7f420c05d9d0 con 0x7f421c06cf90 2024-09-10T10:27:36.974 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-10T10:27:36.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.975+0000 7f4220842640 1 -- 172.21.15.120:0/900500347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f41fc06be30 msgr2=0x7f41fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.975+0000 7f4220842640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f41fc06be30 0x7f41fc06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4210037000 tx=0x7f421000d000 comp rx=0 tx=0).stop 2024-09-10T10:27:36.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.976+0000 7f4220842640 1 -- 172.21.15.120:0/900500347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f421c06cf90 msgr2=0x7f421c1540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:36.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.976+0000 7f4220842640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f421c06cf90 0x7f421c1540e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f420c00fad0 tx=0x7f420c00ff90 comp rx=0 tx=0).stop 2024-09-10T10:27:36.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.976+0000 7f4220842640 1 -- 172.21.15.120:0/900500347 shutdown_connections 2024-09-10T10:27:36.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.976+0000 7f4220842640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f41fc06be30 0x7f41fc06e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.976+0000 7f4220842640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f421c0c7b30 0x7f421c143a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.976+0000 7f4220842640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f421c06d370 0x7f421c1ed020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.976+0000 7f4220842640 1 --2- 172.21.15.120:0/900500347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f421c06cf90 0x7f421c1540e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:36.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.976+0000 7f4220842640 1 -- 172.21.15.120:0/900500347 >> 172.21.15.120:0/900500347 conn(0x7f421c067d40 msgr2=0x7f421c0c9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:36.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.976+0000 7f4220842640 1 -- 172.21.15.120:0/900500347 shutdown_connections 2024-09-10T10:27:36.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:36.976+0000 7f4220842640 1 -- 172.21.15.120:0/900500347 wait complete. 2024-09-10T10:27:36.983 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:27:37.072 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:37.071+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:37.133 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:37.133 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:37.142 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:27:37.183 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:37.182+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:37.261 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:37.260+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:37.285 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:37.285 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:37.294 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:27:37.362 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:37.361+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:37.396 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:37.396 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:37.404 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:27:37.462 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:37.461+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:37.508 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:37.509 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:37.517 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:27:37.601 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:37.600+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:37.661 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:37.661 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:37.663 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:37.662+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:37.669 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:27:37.786 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:37.782+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:37.826 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:37.827 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:37.832 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:27:37.892 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:37.889+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:37.974 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:37.966+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:38.004 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:38.004 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:38.021 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:27:38.098 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:38.096+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:38.140 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:38.141 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:38.149 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:27:38.169 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:38.168+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:38.270 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:38.269+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:38.302 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:38.302 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:38.310 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:27:38.372 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:38.371+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:38.429 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:38.429 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:38.438 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:27:38.484 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:38.483+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:38.543 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:38.544 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:38.551 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:27:38.593 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:38.592+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:38.673 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:38.672+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:38.696 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:38.696 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:38.705 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:27:38.774 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:38.773+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:38.809 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:38.809 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:38.817 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:27:38.875 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:38.874+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:38.902 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:27:38.922 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:38.923 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:38.931 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:27:38.976 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:38.975+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:38.989 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-10T10:27:39.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.002+0000 7f4d141c6640 1 Processor -- start 2024-09-10T10:27:39.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.002+0000 7f4d141c6640 1 -- start start 2024-09-10T10:27:39.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d141c6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4d0c154dc0 0x7f4d0c06cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d141c6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d0c0c9fc0 con 0x7f4d0c067d20 2024-09-10T10:27:39.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d141c6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d0c0c3aa0 con 0x7f4d0c06d160 2024-09-10T10:27:39.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d141c6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4d0c0ca230 con 0x7f4d0c154dc0 2024-09-10T10:27:39.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d11761640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f4d0c067d20 0x7f4d0c072d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:50584/0 (socket says 172.21.15.120:50584) 2024-09-10T10:27:39.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d11761640 1 -- 172.21.15.120:0/752088142 learned_addr learned my addr 172.21.15.120:0/752088142 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:39.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d11f62640 1 --2- 172.21.15.120:0/752088142 >> v2:172.21.15.134:3300/0 conn(0x7f4d0c154dc0 0x7f4d0c06cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d10f60640 1 -- 172.21.15.120:0/752088142 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193054096 0 0) 0x7f4d0c0c9fc0 con 0x7f4d0c067d20 2024-09-10T10:27:39.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d10f60640 1 -- 172.21.15.120:0/752088142 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cf8003520 con 0x7f4d0c067d20 2024-09-10T10:27:39.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d10f60640 1 -- 172.21.15.120:0/752088142 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2007446219 0 0) 0x7f4d0c0c3aa0 con 0x7f4d0c06d160 2024-09-10T10:27:39.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d10f60640 1 -- 172.21.15.120:0/752088142 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d0c0c9fc0 con 0x7f4d0c06d160 2024-09-10T10:27:39.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.003+0000 7f4d11f62640 1 -- 172.21.15.120:0/752088142 >> v1:172.21.15.134:6789/0 conn(0x7f4d0c06d160 legacy=0x7f4d0c165310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.004+0000 7f4d10f60640 1 -- 172.21.15.120:0/752088142 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3224197138 0 0) 0x7f4cf8003520 con 0x7f4d0c067d20 2024-09-10T10:27:39.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.004+0000 7f4d11f62640 1 -- 172.21.15.120:0/752088142 >> v1:172.21.15.120:6789/0 conn(0x7f4d0c067d20 legacy=0x7f4d0c072d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.004+0000 7f4d11f62640 1 -- 172.21.15.120:0/752088142 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c1551a0 con 0x7f4d0c154dc0 2024-09-10T10:27:39.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.004+0000 7f4d11f62640 1 --2- 172.21.15.120:0/752088142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d0c154dc0 0x7f4d0c06cc20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4d080097a0 tx=0x7f4d0802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1716b4b24b07ec6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.004+0000 7f4d10f60640 1 -- 172.21.15.120:0/752088142 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08037070 con 0x7f4d0c154dc0 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.004+0000 7f4d10f60640 1 -- 172.21.15.120:0/752088142 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0802abc0 con 0x7f4d0c154dc0 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.004+0000 7f4d10f60640 1 -- 172.21.15.120:0/752088142 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc001670 con 0x7f4d0c154dc0 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.005+0000 7f4d141c6640 1 -- 172.21.15.120:0/752088142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d0c154dc0 msgr2=0x7f4d0c06cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.005+0000 7f4d141c6640 1 --2- 172.21.15.120:0/752088142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d0c154dc0 0x7f4d0c06cc20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4d080097a0 tx=0x7f4d0802a130 comp rx=0 tx=0).stop 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.005+0000 7f4d141c6640 1 -- 172.21.15.120:0/752088142 shutdown_connections 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.005+0000 7f4d141c6640 1 --2- 172.21.15.120:0/752088142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d0c154dc0 0x7f4d0c06cc20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.005+0000 7f4d141c6640 1 -- 172.21.15.120:0/752088142 >> 172.21.15.120:0/752088142 conn(0x7f4d0c0c9400 msgr2=0x7f4d0c0c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.005+0000 7f4d141c6640 1 -- 172.21.15.120:0/752088142 shutdown_connections 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.005+0000 7f4d141c6640 1 -- 172.21.15.120:0/752088142 wait complete. 2024-09-10T10:27:39.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.006+0000 7f4d141c6640 1 Processor -- start 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.006+0000 7f4d141c6640 1 -- start start 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d141c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d0c067d20 0x7f4d0c1f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d141c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d0c06d160 0x7f4d0c14af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d141c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d0c154dc0 0x7f4d0c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d141c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c072840 con 0x7f4d0c06d160 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d141c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c072560 con 0x7f4d0c154dc0 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d141c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0c072010 con 0x7f4d0c067d20 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d11761640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d0c06d160 0x7f4d0c14af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d11761640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d0c06d160 0x7f4d0c14af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44960/0 (socket says 172.21.15.120:44960) 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d11761640 1 -- 172.21.15.120:0/1263059557 learned_addr learned my addr 172.21.15.120:0/1263059557 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d11f62640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d0c067d20 0x7f4d0c1f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d12763640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d0c154dc0 0x7f4d0c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d11761640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d0c067d20 msgr2=0x7f4d0c1f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d11761640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d0c067d20 0x7f4d0c1f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d11761640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d0c154dc0 msgr2=0x7f4d0c158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.007+0000 7f4d11761640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d0c154dc0 0x7f4d0c158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.008+0000 7f4d11761640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c14b480 con 0x7f4d0c06d160 2024-09-10T10:27:39.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.008+0000 7f4d11761640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d0c06d160 0x7f4d0c14af40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00c5a0 tx=0x7f4cfc00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.008+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc017070 con 0x7f4d0c06d160 2024-09-10T10:27:39.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.008+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cfc005ce0 con 0x7f4d0c06d160 2024-09-10T10:27:39.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.008+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0c150c60 con 0x7f4d0c06d160 2024-09-10T10:27:39.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.008+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0072b0 con 0x7f4d0c06d160 2024-09-10T10:27:39.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.008+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d0c150f00 con 0x7f4d0c06d160 2024-09-10T10:27:39.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.009+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f4d0c0c6d90 con 0x7f4d0c06d160 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.010+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4cfc01a070 con 0x7f4d0c06d160 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.010+0000 7f4d027fc640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4cf006be30 0x7f4cf006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.010+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f4cfc08a6c0 con 0x7f4d0c06d160 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.010+0000 7f4d11f62640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4cf006be30 0x7f4cf006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.010+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v29) ==== 54+0+16 (secure 0 0 0) 0x7f4d0c0c6d90 con 0x7f4d0c06d160 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.011+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f4d0c15a320 0x7f4d0c1f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.011+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d0c0c6280 con 0x7f4d0c15a320 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.011+0000 7f4d12763640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f4d0c15a320 0x7f4d0c1f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.011+0000 7f4d11f62640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4cf006be30 0x7f4cf006e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d08034000 tx=0x7f4d0802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.011+0000 7f4d12763640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f4d0c15a320 0x7f4d0c1f0c30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.012+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d0c0c6280 con 0x7f4d0c15a320 2024-09-10T10:27:39.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.026+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4d0c0c9150 con 0x7f4d0c15a320 2024-09-10T10:27:39.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.027+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4d0c0c9150 con 0x7f4d0c15a320 2024-09-10T10:27:39.029 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:39.029 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:39.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.028+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f4d0c15a700 0x7f4d0c1e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.028+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f4d0c0c3670 con 0x7f4d0c15a700 2024-09-10T10:27:39.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.028+0000 7f4d11f62640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f4d0c15a700 0x7f4d0c1e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.028+0000 7f4d11f62640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f4d0c15a700 0x7f4d0c1e4a70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.029+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d0c0c3670 con 0x7f4d0c15a700 2024-09-10T10:27:39.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.043+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4d0c067b00 con 0x7f4d0c15a700 2024-09-10T10:27:39.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.045+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4d0c067b00 con 0x7f4d0c15a700 2024-09-10T10:27:39.046 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:39.047 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:39.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.045+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f4d0c1f1170 0x7f4d0c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.045+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f4d0c0c9980 con 0x7f4d0c1f1170 2024-09-10T10:27:39.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.045+0000 7f4d11761640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f4d0c1f1170 0x7f4d0c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.045+0000 7f4d11761640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f4d0c1f1170 0x7f4d0c0cbcb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.046+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d0c0c9980 con 0x7f4d0c1f1170 2024-09-10T10:27:39.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.059+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4d0c0cc440 con 0x7f4d0c1f1170 2024-09-10T10:27:39.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.061+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4d0c0cc440 con 0x7f4d0c1f1170 2024-09-10T10:27:39.062 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:39.063 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:39.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.061+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f4d0c062f00 0x7f4d0c1e7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.061+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4d0c0cc650 con 0x7f4d0c062f00 2024-09-10T10:27:39.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.062+0000 7f4d12763640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f4d0c062f00 0x7f4d0c1e7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.062+0000 7f4d12763640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f4d0c062f00 0x7f4d0c1e7240 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.062+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d04019410 con 0x7f4d0c062f00 2024-09-10T10:27:39.076 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:39.075+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.076+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4d0c063590 con 0x7f4d0c062f00 2024-09-10T10:27:39.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.078+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4d0c063590 con 0x7f4d0c062f00 2024-09-10T10:27:39.079 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:39.079 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:39.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.078+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f4d0c1e7c90 0x7f4d0c02c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.078+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4d0c1e80d0 con 0x7f4d0c1e7c90 2024-09-10T10:27:39.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.078+0000 7f4d11f62640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f4d0c1e7c90 0x7f4d0c02c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.079+0000 7f4d11f62640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f4d0c1e7c90 0x7f4d0c02c640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.080+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d0c1e80d0 con 0x7f4d0c1e7c90 2024-09-10T10:27:39.081 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:39.082 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:39.090 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:27:39.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.093+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4d0c1e80d0 con 0x7f4d0c1e7c90 2024-09-10T10:27:39.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.095+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4d0c1e80d0 con 0x7f4d0c1e7c90 2024-09-10T10:27:39.096 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:39.097 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:39.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.095+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f4d0c02d090 0x7f4d0c02d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.095+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f4d0c02da30 con 0x7f4d0c02d090 2024-09-10T10:27:39.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.095+0000 7f4d11761640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f4d0c02d090 0x7f4d0c02d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.096+0000 7f4d11761640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f4d0c02d090 0x7f4d0c02d4f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.096+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d0c02da30 con 0x7f4d0c02d090 2024-09-10T10:27:39.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.097+0000 7ff110f31640 1 Processor -- start 2024-09-10T10:27:39.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.097+0000 7ff110f31640 1 -- start start 2024-09-10T10:27:39.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.098+0000 7ff110f31640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff10c0c8df0 0x7ff10c0c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.098+0000 7ff110f31640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff10c15c230 con 0x7ff10c0c3df0 2024-09-10T10:27:39.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.098+0000 7ff110f31640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff10c15c510 con 0x7ff10c152d60 2024-09-10T10:27:39.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.098+0000 7ff110f31640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff10c15cdb0 con 0x7ff10c0c8df0 2024-09-10T10:27:39.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.099+0000 7ff109d9b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff10c0c8df0 0x7ff10c0c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.099+0000 7ff10ad9d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff10c0c3df0 0x7ff10c163190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:50594/0 (socket says 172.21.15.120:50594) 2024-09-10T10:27:39.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.099+0000 7ff10ad9d640 1 -- 172.21.15.120:0/1900430138 learned_addr learned my addr 172.21.15.120:0/1900430138 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:39.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.099+0000 7ff109d9b640 1 -- 172.21.15.120:0/1900430138 >> v1:172.21.15.134:6789/0 conn(0x7ff10c152d60 legacy=0x7ff10c159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.099+0000 7ff109d9b640 1 -- 172.21.15.120:0/1900430138 >> v1:172.21.15.120:6789/0 conn(0x7ff10c0c3df0 legacy=0x7ff10c163190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.099+0000 7ff10959a640 1 -- 172.21.15.120:0/1900430138 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2997551191 0 0) 0x7ff10c15c510 con 0x7ff10c152d60 2024-09-10T10:27:39.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.099+0000 7ff109d9b640 1 -- 172.21.15.120:0/1900430138 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10c15c7f0 con 0x7ff10c0c8df0 2024-09-10T10:27:39.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.100+0000 7ff109d9b640 1 --2- 172.21.15.120:0/1900430138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c8df0 0x7ff10c0c3410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff1000097a0 tx=0x7ff10002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d66c018b414817 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.100+0000 7ff10959a640 1 -- 172.21.15.120:0/1900430138 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100037070 con 0x7ff10c0c8df0 2024-09-10T10:27:39.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.100+0000 7ff10959a640 1 -- 172.21.15.120:0/1900430138 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff10002ae50 con 0x7ff10c0c8df0 2024-09-10T10:27:39.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.100+0000 7ff10959a640 1 -- 172.21.15.120:0/1900430138 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100032660 con 0x7ff10c0c8df0 2024-09-10T10:27:39.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.101+0000 7ff110f31640 1 -- 172.21.15.120:0/1900430138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c8df0 msgr2=0x7ff10c0c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.101+0000 7ff110f31640 1 --2- 172.21.15.120:0/1900430138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c8df0 0x7ff10c0c3410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff1000097a0 tx=0x7ff10002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.101+0000 7ff110f31640 1 -- 172.21.15.120:0/1900430138 shutdown_connections 2024-09-10T10:27:39.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.101+0000 7ff110f31640 1 --2- 172.21.15.120:0/1900430138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c8df0 0x7ff10c0c3410 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.101+0000 7ff110f31640 1 -- 172.21.15.120:0/1900430138 >> 172.21.15.120:0/1900430138 conn(0x7ff10c0716e0 msgr2=0x7ff10c071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:39.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.101+0000 7ff110f31640 1 -- 172.21.15.120:0/1900430138 shutdown_connections 2024-09-10T10:27:39.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.101+0000 7ff110f31640 1 -- 172.21.15.120:0/1900430138 wait complete. 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.102+0000 7ff110f31640 1 Processor -- start 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.102+0000 7ff110f31640 1 -- start start 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff110f31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c3df0 0x7ff10c151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff110f31640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff10c0c8df0 0x7ff10c14fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff110f31640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff10c152d60 0x7ff10c14e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff110f31640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c158b90 con 0x7ff10c0c3df0 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff110f31640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c0cf3c0 con 0x7ff10c0c8df0 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff110f31640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff10c159730 con 0x7ff10c152d60 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff10a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c3df0 0x7ff10c151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff10a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c3df0 0x7ff10c151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44974/0 (socket says 172.21.15.120:44974) 2024-09-10T10:27:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff10a59c640 1 -- 172.21.15.120:0/1000352630 learned_addr learned my addr 172.21.15.120:0/1000352630 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:39.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff109d9b640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff10c0c8df0 0x7ff10c14fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff10ad9d640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff10c152d60 0x7ff10c14e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff10a59c640 1 -- 172.21.15.120:0/1000352630 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff10c152d60 msgr2=0x7ff10c14e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff10a59c640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff10c152d60 0x7ff10c14e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff10a59c640 1 -- 172.21.15.120:0/1000352630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff10c0c8df0 msgr2=0x7ff10c14fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff10a59c640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff10c0c8df0 0x7ff10c14fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.103+0000 7ff10a59c640 1 -- 172.21.15.120:0/1000352630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10c151c30 con 0x7ff10c0c3df0 2024-09-10T10:27:39.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.104+0000 7ff10a59c640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c3df0 0x7ff10c151380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00c490 tx=0x7ff0fc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.104+0000 7ff0faffd640 1 -- 172.21.15.120:0/1000352630 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc017070 con 0x7ff10c0c3df0 2024-09-10T10:27:39.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.104+0000 7ff0faffd640 1 -- 172.21.15.120:0/1000352630 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0fc005480 con 0x7ff10c0c3df0 2024-09-10T10:27:39.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.104+0000 7ff110f31640 1 -- 172.21.15.120:0/1000352630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10c14c8d0 con 0x7ff10c0c3df0 2024-09-10T10:27:39.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.104+0000 7ff0faffd640 1 -- 172.21.15.120:0/1000352630 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc003810 con 0x7ff10c0c3df0 2024-09-10T10:27:39.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.104+0000 7ff110f31640 1 -- 172.21.15.120:0/1000352630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10c1521c0 con 0x7ff10c0c3df0 2024-09-10T10:27:39.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.105+0000 7ff110f31640 1 -- 172.21.15.120:0/1000352630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff10c0694c0 con 0x7ff10c0c3df0 2024-09-10T10:27:39.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.105+0000 7ff0faffd640 1 -- 172.21.15.120:0/1000352630 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0fc01a070 con 0x7ff10c0c3df0 2024-09-10T10:27:39.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.106+0000 7ff0faffd640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff0ec06be30 0x7ff0ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.106+0000 7ff109d9b640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff0ec06be30 0x7ff0ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.106+0000 7ff0faffd640 1 -- 172.21.15.120:0/1000352630 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7ff0fc08a450 con 0x7ff10c0c3df0 2024-09-10T10:27:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.106+0000 7ff109d9b640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff0ec06be30 0x7ff0ec06e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff100034000 tx=0x7ff10002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.110+0000 7ff0faffd640 1 -- 172.21.15.120:0/1000352630 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff10c0694c0 con 0x7ff10c0c3df0 2024-09-10T10:27:39.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.114+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4d0c02da30 con 0x7f4d0c02d090 2024-09-10T10:27:39.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.116+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4d0c02da30 con 0x7f4d0c02d090 2024-09-10T10:27:39.117 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:39.118 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:39.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.116+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f4d0c207860 0x7f4d0c209c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.116+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f4d0c20caa0 con 0x7f4d0c207860 2024-09-10T10:27:39.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.117+0000 7f4d12763640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f4d0c207860 0x7f4d0c209c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.117+0000 7f4d12763640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f4d0c207860 0x7f4d0c209c50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.118+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d0c20caa0 con 0x7f4d0c207860 2024-09-10T10:27:39.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.145+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4d0c20a210 con 0x7f4d0c207860 2024-09-10T10:27:39.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.147+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4d0c20a210 con 0x7f4d0c207860 2024-09-10T10:27:39.148 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:39.149 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:39.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.147+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f4d0c20a930 0x7f4d0c20ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:39.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.147+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f4d0c20fe10 con 0x7f4d0c20a930 2024-09-10T10:27:39.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.148+0000 7f4d11f62640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f4d0c20a930 0x7f4d0c20ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:39.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.148+0000 7f4d11f62640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f4d0c20a930 0x7f4d0c20ad70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:39.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.149+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d0c20fe10 con 0x7f4d0c20a930 2024-09-10T10:27:39.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.162+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4d0c20b350 con 0x7f4d0c20a930 2024-09-10T10:27:39.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d027fc640 1 -- 172.21.15.120:0/1263059557 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4d0c20b350 con 0x7f4d0c20a930 2024-09-10T10:27:39.165 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f4d0c15a320 msgr2=0x7f4d0c1f0c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f4d0c15a320 0x7f4d0c1f0c30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f4d0c15a700 msgr2=0x7f4d0c1e4a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f4d0c15a700 0x7f4d0c1e4a70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f4d0c1f1170 msgr2=0x7f4d0c0cbcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f4d0c1f1170 0x7f4d0c0cbcb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f4d0c062f00 msgr2=0x7f4d0c1e7240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f4d0c062f00 0x7f4d0c1e7240 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f4d0c1e7c90 msgr2=0x7f4d0c02c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f4d0c1e7c90 0x7f4d0c02c640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f4d0c02d090 msgr2=0x7f4d0c02d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.164+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f4d0c02d090 0x7f4d0c02d4f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f4d0c207860 msgr2=0x7f4d0c209c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f4d0c207860 0x7f4d0c209c50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f4d0c20a930 msgr2=0x7f4d0c20ad70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f4d0c20a930 0x7f4d0c20ad70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d12763640 1 -- 172.21.15.120:0/1263059557 reap_dead start 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d12763640 1 -- 172.21.15.120:0/1263059557 reap_dead start 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d12763640 1 -- 172.21.15.120:0/1263059557 reap_dead start 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d12763640 1 -- 172.21.15.120:0/1263059557 reap_dead start 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4cf006be30 msgr2=0x7f4cf006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4cf006be30 0x7f4cf006e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d08034000 tx=0x7f4d0802e000 comp rx=0 tx=0).stop 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d0c06d160 msgr2=0x7f4d0c14af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d0c06d160 0x7f4d0c14af40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00c5a0 tx=0x7f4cfc00ca60 comp rx=0 tx=0).stop 2024-09-10T10:27:39.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 shutdown_connections 2024-09-10T10:27:39.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4cf006be30 0x7f4cf006e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 --2- 172.21.15.120:0/1263059557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d0c06d160 0x7f4d0c14af40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 >> 172.21.15.120:0/1263059557 conn(0x7f4d0c0c9400 msgr2=0x7f4d0c0c6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:39.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 shutdown_connections 2024-09-10T10:27:39.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.165+0000 7f4d141c6640 1 -- 172.21.15.120:0/1263059557 wait complete. 2024-09-10T10:27:39.181 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:39.180+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:39.238 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:39.239 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:39.239 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:27:39.239 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:39.247 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:27:39.288 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:39.287+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:39.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.311+0000 7ff110f31640 1 -- 172.21.15.120:0/1000352630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7ff10c15be60 con 0x7ff10c0c3df0 2024-09-10T10:27:39.379 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:39.378+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:39.390 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:39.391 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:39.391 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:39.391 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:39.391 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:39.399 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:27:39.484 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:39.483+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 52.763710447999998, 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.546 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:27:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:27:39.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:39.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:27:39.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 52.763379948000001, 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.552 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:27:39.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:39.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:27:39.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:27:39.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:27:39.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:27:39.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 52.763333404999997, 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:39.558 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:27:39.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.671891193, 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:27:39.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.671809849999999, 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.567 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:39.568 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:27:39.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:27:39.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:27:39.571 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.671783295000001, 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.572 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:27:39.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:27:39.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:27:39.576 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.577 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.671640646, 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.668993712999999, 2024-09-10T10:27:39.583 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.584 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.585 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:27:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:27:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.668142972999998, 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:27:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 25.595991290000001, 2024-09-10T10:27:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 25.595942153999999, 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 25.595830675999999, 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:27:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 19.622430157, 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:27:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:27:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:27:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:27:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:27:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 19.622399986000001, 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:27:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:27:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 19.622343493999999, 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:27:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:27:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:27:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 19.622242325999999, 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:27:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:27:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:27:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 13.601975229000001, 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:27:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.634 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 13.59791001, 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.635 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:27:39.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:27:39.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 13.597892140000001, 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.639 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:27:39.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:27:39.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:39.643 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 13.597845902, 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:39.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:39.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.647 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:39.648 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:39.648 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:39.649 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:27:39.649 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:39.648+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:39.749 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:39.749+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:39.793 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:39.793 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:39.793 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:27:39.793 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:39.802 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:27:39.850 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:39.849+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:39.905 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:39.905 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:39.905 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:39.905 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:39.905 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:39.913 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:27:39.951 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:39.950+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:39.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.976+0000 7ff0faffd640 1 -- 172.21.15.120:0/1000352630 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v30) ==== 97+0+0 (secure 0 0 0) 0x7ff0fc058440 con 0x7ff10c0c3df0 2024-09-10T10:27:39.978 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-10T10:27:39.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 -- 172.21.15.120:0/1000352630 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff0ec06be30 msgr2=0x7ff0ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff0ec06be30 0x7ff0ec06e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff100034000 tx=0x7ff10002e000 comp rx=0 tx=0).stop 2024-09-10T10:27:39.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 -- 172.21.15.120:0/1000352630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c3df0 msgr2=0x7ff10c151380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:39.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c3df0 0x7ff10c151380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00c490 tx=0x7ff0fc00c950 comp rx=0 tx=0).stop 2024-09-10T10:27:39.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 -- 172.21.15.120:0/1000352630 shutdown_connections 2024-09-10T10:27:39.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff0ec06be30 0x7ff0ec06e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff10c152d60 0x7ff10c14e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff10c0c8df0 0x7ff10c14fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 --2- 172.21.15.120:0/1000352630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff10c0c3df0 0x7ff10c151380 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:39.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 -- 172.21.15.120:0/1000352630 >> 172.21.15.120:0/1000352630 conn(0x7ff10c0716e0 msgr2=0x7ff10c071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:39.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 -- 172.21.15.120:0/1000352630 shutdown_connections 2024-09-10T10:27:39.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:39.980+0000 7ff110f31640 1 -- 172.21.15.120:0/1000352630 wait complete. 2024-09-10T10:27:40.052 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:40.051+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:40.061 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:40.061 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.963187636000001, 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.17492211699999999, 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 1 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:27:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096318+0000", 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.657e-06 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096326+0000", 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0560000000000005e-06 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096339+0000", 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2965999999999999e-05 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096349+0000", 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0784e-05 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096432+0000", 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2657999999999994e-05 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op must wait for map", 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096441+0000", 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4590000000000004e-06 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096935+0000", 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049331899999999998 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for peered", 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096944+0000", 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9539999999999993e-06 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267505+0000", 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.170561622 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267700+0000", 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019420400000000001 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267923+0000", 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000223451 2024-09-10T10:27:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.270575+0000", 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026522030000000001 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271131+0000", 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055599000000000002 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271191+0000", 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9947000000000001e-05 2024-09-10T10:27:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271238+0000", 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6847000000000001e-05 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.574353392000006, 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078035889999999997, 2024-09-10T10:27:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 115 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1300000000000001e-07 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485209+0000", 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8628999999999997e-05 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485214+0000", 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0849999999999996e-06 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485218+0000", 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3840000000000001e-06 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490185+0000", 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0049675270000000002 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490231+0000", 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5488000000000002e-05 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490307+0000", 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6038000000000006e-05 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491263+0000", 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095625099999999998 2024-09-10T10:27:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492908+0000", 2024-09-10T10:27:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001645355 2024-09-10T10:27:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.075 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492934+0000", 2024-09-10T10:27:40.075 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5431000000000001e-05 2024-09-10T10:27:40.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492954+0000", 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0088000000000001e-05 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.574278903000007, 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077741099999999999, 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 116 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1100000000000002e-07 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485227+0000", 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1890000000000001e-06 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485231+0000", 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4010000000000001e-06 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485233+0000", 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.728e-06 2024-09-10T10:27:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490628+0000", 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0053952109999999996 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490674+0000", 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5946999999999999e-05 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490750+0000", 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5227000000000003e-05 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491443+0000", 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069346800000000004 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492959+0000", 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015154470000000001 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492983+0000", 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.406e-05 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492999+0000", 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6121e-05 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.574266774999998, 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078059490000000004, 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 117 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.23e-07 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485240+0000", 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780000000000001e-06 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485243+0000", 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8430000000000003e-06 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485247+0000", 2024-09-10T10:27:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2279999999999998e-06 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490995+0000", 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057480730000000002 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491043+0000", 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7936000000000003e-05 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491113+0000", 2024-09-10T10:27:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0872999999999997e-05 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491636+0000", 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00052242300000000005 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493002+0000", 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001365808 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493025+0000", 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2932999999999999e-05 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493043+0000", 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8031000000000001e-05 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241911633000001, 2024-09-10T10:27:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095639920000000003, 2024-09-10T10:27:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 131 2024-09-10T10:27:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1199999999999998e-07 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817663+0000", 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1099000000000005e-05 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817670+0000", 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.8539999999999999e-06 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817674+0000", 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.574e-06 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818465+0000", 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000791445 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818509+0000", 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3569000000000002e-05 2024-09-10T10:27:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818580+0000", 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1503999999999998e-05 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821292+0000", 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027120529999999999 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827108+0000", 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0058160759999999999 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827134+0000", 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5859000000000001e-05 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827156+0000", 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1546999999999999e-05 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241822424999995, 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095198769999999995, 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 132 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2899999999999999e-07 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817683+0000", 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0379999999999998e-06 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817690+0000", 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3110000000000002e-06 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817699+0000", 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820828+0000", 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031296829999999999 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820873+0000", 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4869e-05 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820943+0000", 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9919999999999995e-05 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823683+0000", 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027395140000000002 2024-09-10T10:27:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827161+0000", 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003477857 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827183+0000", 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2612999999999998e-05 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827201+0000", 2024-09-10T10:27:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7842999999999998e-05 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241755118, 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0094944009999999995, 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 133 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817749+0000", 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2399999999999998e-07 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817752+0000", 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1970000000000001e-06 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817760+0000", 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8080000000000005e-06 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817764+0000", 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9419999999999997e-06 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821319+0000", 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035547740000000001 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821364+0000", 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5398000000000001e-05 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821434+0000", 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0382999999999993e-05 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823717+0000", 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022825089999999998 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827205+0000", 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034879009999999998 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827225+0000", 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0611999999999998e-05 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827243+0000", 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7353000000000001e-05 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241702277000002, 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095863660000000007, 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 134 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817802+0000", 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8999999999999996e-07 2024-09-10T10:27:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817871+0000", 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9572000000000004e-05 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817878+0000", 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2079999999999997e-06 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817882+0000", 2024-09-10T10:27:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5889999999999996e-06 2024-09-10T10:27:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821625+0000", 2024-09-10T10:27:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037431069999999999 2024-09-10T10:27:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821669+0000", 2024-09-10T10:27:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3455000000000003e-05 2024-09-10T10:27:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821740+0000", 2024-09-10T10:27:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1390999999999995e-05 2024-09-10T10:27:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823826+0000", 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020863399999999999 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827343+0000", 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003516267 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827369+0000", 2024-09-10T10:27:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.6044999999999999e-05 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827388+0000", 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8802000000000001e-05 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241609581999995, 2024-09-10T10:27:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095738179999999996, 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 135 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5999999999999999e-07 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817897+0000", 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0149999999999999e-06 2024-09-10T10:27:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817904+0000", 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5089999999999999e-06 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817909+0000", 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3120000000000001e-06 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823862+0000", 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059524199999999999 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823911+0000", 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9323000000000002e-05 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823983+0000", 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1736000000000006e-05 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827101+0000", 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031181009999999999 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827392+0000", 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00029090700000000003 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827426+0000", 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4127e-05 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827468+0000", 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1907999999999999e-05 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241582566000005, 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.009594488, 2024-09-10T10:27:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 136 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3700000000000001e-07 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817925+0000", 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4659999999999999e-06 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817931+0000", 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5710000000000003e-06 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817936+0000", 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7550000000000002e-06 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824178+0000", 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062415700000000001 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824225+0000", 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6966999999999999e-05 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824296+0000", 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1284000000000004e-05 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827290+0000", 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002993697 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827473+0000", 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018376699999999999 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827497+0000", 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3445e-05 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827515+0000", 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8529000000000002e-05 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241550876999995, 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096066220000000004, 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 137 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3000000000000001e-07 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818004+0000", 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0902000000000002e-05 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818013+0000", 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5420000000000005e-06 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818019+0000", 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7939999999999997e-06 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824489+0000", 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0064692580000000003 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824532+0000", 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3631000000000003e-05 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824601+0000", 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9028000000000004e-05 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827334+0000", 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027325409999999998 2024-09-10T10:27:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827520+0000", 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018642499999999999 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827541+0000", 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1151000000000001e-05 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827559+0000", 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7819999999999999e-05 2024-09-10T10:27:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241471555999993, 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010100303, 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 138 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818033+0000", 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818037+0000", 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0419999999999996e-06 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818045+0000", 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9759999999999995e-06 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818052+0000", 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5909999999999998e-06 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826594+0000", 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0085421660000000003 2024-09-10T10:27:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826671+0000", 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7121999999999998e-05 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826795+0000", 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012315199999999999 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828081+0000", 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0012869139999999999 2024-09-10T10:27:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828088+0000", 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9870000000000002e-06 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828112+0000", 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3405e-05 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828132+0000", 2024-09-10T10:27:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0394000000000002e-05 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241441184999999, 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01125819, 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 139 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818063+0000", 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7800000000000002e-07 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818067+0000", 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8500000000000004e-06 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818075+0000", 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818081+0000", 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1829999999999999e-06 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827594+0000", 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095129340000000007 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827669+0000", 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5239000000000002e-05 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827789+0000", 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000119757 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829195+0000", 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014064279999999999 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829232+0000", 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7058999999999999e-05 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829281+0000", 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8884000000000002e-05 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829321+0000", 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9527999999999998e-05 2024-09-10T10:27:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241407116000005, 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012263886, 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 140 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818097+0000", 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8999999999999997e-07 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818145+0000", 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8040000000000001e-05 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818160+0000", 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4671999999999999e-05 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818169+0000", 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1900000000000001e-06 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828158+0000", 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099891440000000001 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828228+0000", 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0407000000000005e-05 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828343+0000", 2024-09-10T10:27:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000114821 2024-09-10T10:27:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829914+0000", 2024-09-10T10:27:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001570866 2024-09-10T10:27:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830312+0000", 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00039770499999999999 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830339+0000", 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.69e-05 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830360+0000", 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1651e-05 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.241319875000002, 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012742035000000001, 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 141 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7700000000000004e-07 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818188+0000", 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8659999999999999e-06 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818196+0000", 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9480000000000008e-06 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818205+0000", 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8570000000000001e-06 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828883+0000", 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010678600999999999 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828927+0000", 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3887999999999997e-05 2024-09-10T10:27:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829001+0000", 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3442000000000006e-05 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830548+0000", 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001547194 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830884+0000", 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00033638100000000002 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830907+0000", 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3051999999999999e-05 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830926+0000", 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8229000000000001e-05 2024-09-10T10:27:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.228172138000005, 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097270880000000001, 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 145 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831332+0000", 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8900000000000001e-07 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831379+0000", 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7134e-05 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831387+0000", 2024-09-10T10:27:40.158 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7409999999999992e-06 2024-09-10T10:27:40.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.158 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.158 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831391+0000", 2024-09-10T10:27:40.158 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6190000000000002e-06 2024-09-10T10:27:40.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.158 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833155+0000", 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017641969999999999 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833237+0000", 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1957000000000002e-05 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833356+0000", 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011843100000000001 2024-09-10T10:27:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.834068+0000", 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00071179000000000001 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841004+0000", 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069359749999999996 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.160 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841034+0000", 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.042e-05 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841058+0000", 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4535000000000001e-05 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.161 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:20 2.9 e26/18 2:92e99a1d:::612.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 31.183975874000001, 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074964979999999999, 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875534+0000", 2024-09-10T10:27:40.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9109999999999998e-06 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875543+0000", 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8319999999999995e-06 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875549+0000", 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2380000000000002e-06 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875699+0000", 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014945900000000001 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875717+0000", 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8584e-05 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882982+0000", 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0072648039999999997 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883024+0000", 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2135000000000002e-05 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:27 2.9 e26/18 2:91bd965c:::501.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 31.183915240000001, 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074993129999999996, 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.166 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875589+0000", 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3999999999999997e-07 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875595+0000", 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9680000000000003e-06 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875602+0000", 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875608+0000", 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0430000000000004e-06 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875887+0000", 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000279578 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875904+0000", 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6439e-05 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883027+0000", 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0071234480000000001 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883088+0000", 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0424000000000003e-05 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 14.117836163, 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01193195, 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2399999999999999e-07 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941671+0000", 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8329999999999998e-06 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2919999999999998e-06 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941686+0000", 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944874+0000", 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031881639999999998 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945305+0000", 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00043084599999999997 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945438+0000", 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013344600000000001 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950653+0000", 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005214733 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953496+0000", 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028433989999999999 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953554+0000", 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7933000000000003e-05 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953599+0000", 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5130999999999999e-05 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.175 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 14.117810217000001, 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0093592050000000006, 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:40.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941694+0000", 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7899999999999999e-07 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941711+0000", 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7530999999999999e-05 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941720+0000", 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4279999999999996e-06 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941725+0000", 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2390000000000002e-06 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942690+0000", 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00096541400000000005 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942807+0000", 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000116777 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942967+0000", 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015997599999999999 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944708+0000", 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001740492 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950956+0000", 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062481580000000002 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951016+0000", 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0299e-05 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951052+0000", 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6412000000000003e-05 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.181 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.182 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:40.182 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:40.182 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:40.183 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:27:40.184 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:40.182+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:40.284 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:40.283+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:40.294 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:40.294 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:40.294 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:40.294 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:40.303 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:27:40.389 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:40.388+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:40.447 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:40.447 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:40.447 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:40.447 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:40.447 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:40.456 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:27:40.485 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:40.484+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:40.586 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:40.586+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:40.600 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:27:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:1 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.330436124000002, 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028809700000000001, 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9999999999999999e-07 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268119+0000", 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1810000000000003e-06 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268129+0000", 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9790000000000005e-06 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268137+0000", 2024-09-10T10:27:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4800000000000001e-06 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268198+0000", 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0918000000000002e-05 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268226+0000", 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7267000000000002e-05 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270943+0000", 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027177529999999998 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270996+0000", 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2191999999999998e-05 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:119 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'81, mlcod=16'78)", 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.104585170999997, 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029329880000000001, 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493966+0000", 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1699999999999998e-07 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494040+0000", 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3883999999999999e-05 2024-09-10T10:27:40.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494048+0000", 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4880000000000007e-06 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494057+0000", 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1310000000000006e-06 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494095+0000", 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8470000000000003e-05 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494108+0000", 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2588999999999999e-05 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496869+0000", 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027610400000000002 2024-09-10T10:27:40.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496898+0000", 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9869000000000002e-05 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.104201535000001, 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005336365, 2024-09-10T10:27:40.610 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494350+0000", 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494416+0000", 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6413000000000003e-05 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494424+0000", 2024-09-10T10:27:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.576e-06 2024-09-10T10:27:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494430+0000", 2024-09-10T10:27:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6579999999999996e-06 2024-09-10T10:27:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494466+0000", 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5648999999999997e-05 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494478+0000", 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.234e-05 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499654+0000", 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051755209999999998 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499685+0000", 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.167e-05 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.103806724999998, 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049664970000000003, 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.615 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494819+0000", 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4690000000000005e-05 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494825+0000", 2024-09-10T10:27:40.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7080000000000002e-06 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494828+0000", 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1109999999999999e-06 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494869+0000", 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1146000000000002e-05 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494881+0000", 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2295e-05 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499687+0000", 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048061459999999999 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499710+0000", 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-05 2024-09-10T10:27:40.619 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.818450521999999, 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054008370000000003, 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780104+0000", 2024-09-10T10:27:40.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3039999999999998e-06 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0738e-05 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780141+0000", 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6317999999999999e-05 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2199999999999992e-06 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780405+0000", 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025557599999999998 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780498+0000", 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2860000000000002e-05 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780591+0000", 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3094e-05 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024814469999999999 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785441+0000", 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023678280000000002 2024-09-10T10:27:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785480+0000", 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9511999999999997e-05 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785501+0000", 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0939999999999999e-05 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.818375688000003, 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054780480000000001, 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780176+0000", 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6700000000000003e-07 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:27:40.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1289999999999998e-06 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780188+0000", 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.96e-06 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780194+0000", 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9989999999999997e-06 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780882+0000", 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068832400000000003 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780968+0000", 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5959999999999997e-05 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781098+0000", 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012994100000000001 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783078+0000", 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019803540000000001 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785553+0000", 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002474515 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785608+0000", 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5319999999999999e-05 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785653+0000", 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4879000000000002e-05 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.818344144999998, 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042025200000000004, 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.631 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780217+0000", 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780224+0000", 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2529999999999998e-06 2024-09-10T10:27:40.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780230+0000", 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6110000000000001e-06 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780282+0000", 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2148000000000003e-05 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780394+0000", 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111864 2024-09-10T10:27:40.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780553+0000", 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015913799999999999 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783054+0000", 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025008809999999999 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784349+0000", 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012947480000000001 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784381+0000", 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2268999999999999e-05 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784409+0000", 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8007000000000001e-05 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.818301793000003, 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054782659999999999, 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:40.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780253+0000", 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4199999999999999e-06 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780260+0000", 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5100000000000001e-06 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780267+0000", 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.443e-06 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781353+0000", 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010859769999999999 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781457+0000", 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104545 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781561+0000", 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010409200000000001 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783532+0000", 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019702280000000001 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785685+0000", 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002153313 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785710+0000", 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5024999999999998e-05 2024-09-10T10:27:40.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785727+0000", 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7175000000000001e-05 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:40.642 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.818275702000001, 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005104407, 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780278+0000", 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.959e-06 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780285+0000", 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8040000000000004e-06 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780292+0000", 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4099999999999996e-06 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781345+0000", 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010533579999999999 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781456+0000", 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110624 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781570+0000", 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011417999999999999 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783625+0000", 2024-09-10T10:27:40.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020553780000000001 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785303+0000", 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001677827 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785345+0000", 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2364999999999998e-05 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785379+0000", 2024-09-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4054000000000003e-05 2024-09-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.818251873999998, 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0060500800000000002, 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780306+0000", 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780313+0000", 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-06 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1739999999999999e-06 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782543+0000", 2024-09-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022253419999999999 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010785699999999999 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.786349+0000", 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036978829999999999 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.726549834, 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058824769999999997, 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872002+0000", 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5149999999999999e-06 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872008+0000", 2024-09-10T10:27:40.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.66e-06 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872022+0000", 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3932e-05 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872031+0000", 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5259999999999999e-06 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872111+0000", 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9412000000000005e-05 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872231+0000", 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119843 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872340+0000", 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109442 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.875579+0000", 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032383899999999998 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877781+0000", 2024-09-10T10:27:40.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022022299999999999 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877835+0000", 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4419000000000002e-05 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877883+0000", 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8108000000000001e-05 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.725461002999999, 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049968149999999999, 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7300000000000005e-07 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.245e-06 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3898e-05 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873116+0000", 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8050000000000007e-06 2024-09-10T10:27:40.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873184+0000", 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7809000000000001e-05 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873257+0000", 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2602999999999996e-05 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873374+0000", 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011744900000000001 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.876282+0000", 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029079459999999998 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878003+0000", 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001720886 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878054+0000", 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0596999999999997e-05 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878086+0000", 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2804e-05 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.664 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.651068631000001, 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029964739999999998, 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947484+0000", 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.857e-06 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947490+0000", 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9789999999999998e-06 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947504+0000", 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3942e-05 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947512+0000", 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5620000000000004e-06 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947599+0000", 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6525000000000003e-05 2024-09-10T10:27:40.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947714+0000", 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114709 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947823+0000", 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109477 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950336+0000", 2024-09-10T10:27:40.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025131509999999999 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950386+0000", 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9974e-05 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950440+0000", 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4011000000000002e-05 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950478+0000", 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8287000000000001e-05 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.649628233000001, 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043004360000000004, 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.670 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948923+0000", 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8999999999999996e-07 2024-09-10T10:27:40.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948928+0000", 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948940+0000", 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1759e-05 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948949+0000", 2024-09-10T10:27:40.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5149999999999995e-06 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949028+0000", 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8182999999999993e-05 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949137+0000", 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109065 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:40.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949245+0000", 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108047 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950193+0000", 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094856200000000002 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953173+0000", 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029801960000000001 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953200+0000", 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6707999999999999e-05 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953223+0000", 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2758000000000001e-05 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.649572720999998, 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046333320000000004, 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948979+0000", 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-07 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6490000000000001e-06 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948990+0000", 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2560000000000002e-06 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948996+0000", 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9739999999999999e-06 2024-09-10T10:27:40.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949496+0000", 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049934299999999997 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949588+0000", 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1762999999999995e-05 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949729+0000", 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014145299999999999 2024-09-10T10:27:40.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950984+0000", 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012548419999999999 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953505+0000", 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025208129999999998 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953568+0000", 2024-09-10T10:27:40.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3695000000000006e-05 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953611+0000", 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2913000000000001e-05 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:40.681 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.677521106, 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048620219999999997, 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4899999999999995e-07 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921034+0000", 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7050000000000001e-06 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921043+0000", 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1840000000000005e-06 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921049+0000", 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4459999999999998e-06 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921575+0000", 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052602300000000003 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921646+0000", 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0475000000000005e-05 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921769+0000", 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123347 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923077+0000", 2024-09-10T10:27:40.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013078759999999999 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925794+0000", 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027172310000000001 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925853+0000", 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8739999999999997e-05 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925892+0000", 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8445999999999997e-05 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.677484071999999, 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032737339999999999, 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.687 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9919999999999999e-06 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921080+0000", 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0063e-05 2024-09-10T10:27:40.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921086+0000", 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921142+0000", 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6174999999999997e-05 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013480499999999999 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921405+0000", 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012819200000000001 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923001+0000", 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015962249999999999 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924243+0000", 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012425820000000001 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924300+0000", 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6901999999999998e-05 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924340+0000", 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9918e-05 2024-09-10T10:27:40.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(2.4 epoch 26/18 rep_tid 19 entries 26'1 (0'0) error 2:218afd09:::203.00000003:head by mds.3.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.675345626999999, 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030589089999999998, 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923206+0000", 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-09-10T10:27:40.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7249999999999997e-06 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923229+0000", 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5905000000000001e-05 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923241+0000", 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1990999999999999e-05 2024-09-10T10:27:40.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923317+0000", 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5622000000000002e-05 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926264+0000", 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029470170000000001 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.654987627000001, 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050642550000000001, 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943564+0000", 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.13e-07 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943576+0000", 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2187e-05 2024-09-10T10:27:40.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943585+0000", 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9279999999999999e-06 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943592+0000", 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0110000000000003e-06 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943681+0000", 2024-09-10T10:27:40.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9487999999999994e-05 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943767+0000", 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5866999999999996e-05 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943882+0000", 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011443299999999999 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.947042+0000", 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031600629999999998 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948540+0000", 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001498451 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948581+0000", 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0839e-05 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948627+0000", 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6375000000000002e-05 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.654919304, 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051386089999999997, 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:40.702 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943632+0000", 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1500000000000004e-07 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943643+0000", 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0954e-05 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943652+0000", 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4929999999999998e-06 2024-09-10T10:27:40.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943657+0000", 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6509999999999998e-06 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944230+0000", 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057322999999999999 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944340+0000", 2024-09-10T10:27:40.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010986 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948770+0000", 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044297060000000003 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:40.706 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:40.708 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:27:40.708 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:40.707+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:40.808 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:40.807+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:40.852 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:40.853 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:40.853 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:40.853 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:40.861 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:27:40.946 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:40.945+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:41.005 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:41.005 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:41.005 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:41.005 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:41.005 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:41.011 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:41.010+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:41.014 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:27:41.112 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:41.111+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:41.158 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:41.159 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:27:41.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:27:41.159 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:27:41.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.159 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:41.159 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:27:41.159 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.376638950999997, 2024-09-10T10:27:41.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049304329999999997, 2024-09-10T10:27:41.159 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.159 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780094+0000", 2024-09-10T10:27:41.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:27:41.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780107+0000", 2024-09-10T10:27:41.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3703000000000001e-05 2024-09-10T10:27:41.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780116+0000", 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5210000000000002e-06 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6340000000000007e-06 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001735838 2024-09-10T10:27:41.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782000+0000", 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013984100000000001 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782131+0000", 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000131025 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783737+0000", 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016054179999999999 2024-09-10T10:27:41.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784937+0000", 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001200638 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784987+0000", 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9591e-05 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785024+0000", 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6751000000000001e-05 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.376599495999997, 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052929559999999997, 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.165 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780136+0000", 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9950000000000001e-06 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780144+0000", 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.174e-06 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4949999999999999e-06 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781163+0000", 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001013438 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781279+0000", 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011579200000000001 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781388+0000", 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108834 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783243+0000", 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00185491 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785355+0000", 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021117990000000001 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785386+0000", 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1612000000000002e-05 2024-09-10T10:27:41.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785426+0000", 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9474999999999997e-05 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:8 2.10 e24/18 2:09401a81:::601.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.374408266000003, 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005352224, 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.170 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:41.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4400000000000001e-07 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782467+0000", 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142452 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782598+0000", 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013078300000000001 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782604+0000", 2024-09-10T10:27:41.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.533e-06 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7073999999999997e-05 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782668+0000", 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6861999999999999e-05 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787632+0000", 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049636510000000003 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787676+0000", 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4425000000000002e-05 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:9 2.10 e24/18 2:09401a81:::601.00000000:head v 24'4, mlcod=0'0)", 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.373771251000001, 2024-09-10T10:27:41.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047438960000000001, 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:27:41.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782962+0000", 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4499999999999997e-07 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782967+0000", 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2000000000000002e-06 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782980+0000", 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3685e-05 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782987+0000", 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4690000000000002e-06 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783112+0000", 2024-09-10T10:27:41.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012481300000000001 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783130+0000", 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7753000000000001e-05 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787678+0000", 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045487419999999997 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787705+0000", 2024-09-10T10:27:41.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6588999999999999e-05 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:10 2.10 e24/18 2:0bd6d154:::602.00000000:head v 24'5, mlcod=0'0)", 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.373664316999999, 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00466629, 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1300000000000001e-07 2024-09-10T10:27:41.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6970000000000001e-06 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783309+0000", 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000236209 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783319+0000", 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7180000000000008e-06 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783365+0000", 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6179e-05 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783383+0000", 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8382000000000001e-05 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787708+0000", 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043246059999999999 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787734+0000", 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6086000000000001e-05 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.183 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.284262122999998, 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054527359999999997, 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872472+0000", 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5409999999999999e-06 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872479+0000", 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9550000000000003e-06 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872494+0000", 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5452999999999999e-05 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872503+0000", 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1509999999999993e-06 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872571+0000", 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7747e-05 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872704+0000", 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013269900000000001 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872804+0000", 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9934999999999998e-05 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877617+0000", 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048138069999999998 2024-09-10T10:27:41.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877819+0000", 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020164799999999999 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877880+0000", 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1296000000000004e-05 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877923+0000", 2024-09-10T10:27:41.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2503999999999998e-05 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.283240556000003, 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0098302110000000002, 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873493+0000", 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-10T10:27:41.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873498+0000", 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1619999999999999e-06 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873511+0000", 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3319000000000001e-05 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873520+0000", 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3409999999999992e-06 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873586+0000", 2024-09-10T10:27:41.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6115e-05 2024-09-10T10:27:41.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873660+0000", 2024-09-10T10:27:41.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3103999999999997e-05 2024-09-10T10:27:41.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873775+0000", 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115617 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878285+0000", 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045096049999999999 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883229+0000", 2024-09-10T10:27:41.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049443999999999998 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883283+0000", 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4262999999999999e-05 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883322+0000", 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8417000000000002e-05 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.282751415999996, 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047062800000000002, 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873982+0000", 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2900000000000003e-07 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873986+0000", 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3939999999999998e-06 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873995+0000", 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0750000000000004e-06 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874002+0000", 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874061+0000", 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8770999999999997e-05 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874178+0000", 2024-09-10T10:27:41.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117108 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874290+0000", 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112503 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878269+0000", 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039784870000000002 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878593+0000", 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032385299999999998 2024-09-10T10:27:41.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878644+0000", 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1790999999999999e-05 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878687+0000", 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2738999999999999e-05 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.199 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.282151137, 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089185449999999999, 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874582+0000", 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1699999999999997e-07 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874592+0000", 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0604999999999999e-05 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874602+0000", 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874610+0000", 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1919999999999999e-06 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874696+0000", 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6033999999999997e-05 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874809+0000", 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113373 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874923+0000", 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114077 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878289+0000", 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033662380000000001 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883420+0000", 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051305760000000004 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883465+0000", 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5252e-05 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883500+0000", 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4415000000000002e-05 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.208287815999999, 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073932399999999997, 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948446+0000", 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3400000000000001e-06 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948451+0000", 2024-09-10T10:27:41.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6219999999999997e-06 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948464+0000", 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2955e-05 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948472+0000", 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0730000000000005e-06 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948534+0000", 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1727e-05 2024-09-10T10:27:41.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948649+0000", 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114991 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948757+0000", 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107935 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950179+0000", 2024-09-10T10:27:41.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014219440000000001 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055817970000000003 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955794+0000", 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3012000000000002e-05 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955838+0000", 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3844000000000002e-05 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.207748996999999, 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069680200000000001, 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948984+0000", 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6100000000000001e-07 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948993+0000", 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9809999999999992e-06 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949002+0000", 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8789999999999992e-06 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949008+0000", 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2759999999999997e-06 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949057+0000", 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9159999999999997e-05 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949128+0000", 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1292999999999997e-05 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:41.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949261+0000", 2024-09-10T10:27:41.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132564 2024-09-10T10:27:41.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950220+0000", 2024-09-10T10:27:41.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00095953300000000004 2024-09-10T10:27:41.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955890+0000", 2024-09-10T10:27:41.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056692790000000002 2024-09-10T10:27:41.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955928+0000", 2024-09-10T10:27:41.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8278000000000002e-05 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955951+0000", 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3215999999999999e-05 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:41.216 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.235772486999998, 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050489669999999997, 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5799999999999999e-07 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920964+0000", 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6720000000000002e-06 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920971+0000", 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5140000000000004e-06 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920977+0000", 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0100000000000001e-06 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921033+0000", 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5899000000000001e-05 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921172+0000", 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139011 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921288+0000", 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011605200000000001 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922851+0000", 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001562499 2024-09-10T10:27:41.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925915+0000", 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003063727 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925969+0000", 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4083999999999999e-05 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926009+0000", 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9941000000000003e-05 2024-09-10T10:27:41.221 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.235165630000001, 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045836510000000002, 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4499999999999996e-07 2024-09-10T10:27:41.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921572+0000", 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3239999999999996e-06 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921581+0000", 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9190000000000005e-06 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921589+0000", 2024-09-10T10:27:41.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0649999999999999e-06 2024-09-10T10:27:41.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921635+0000", 2024-09-10T10:27:41.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6863999999999997e-05 2024-09-10T10:27:41.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921705+0000", 2024-09-10T10:27:41.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9046000000000003e-05 2024-09-10T10:27:41.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:41.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921818+0000", 2024-09-10T10:27:41.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011313499999999999 2024-09-10T10:27:41.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923033+0000", 2024-09-10T10:27:41.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012153590000000001 2024-09-10T10:27:41.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926055+0000", 2024-09-10T10:27:41.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030223310000000001 2024-09-10T10:27:41.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926104+0000", 2024-09-10T10:27:41.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8421999999999998e-05 2024-09-10T10:27:41.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926150+0000", 2024-09-10T10:27:41.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6440999999999998e-05 2024-09-10T10:27:41.231 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.232 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.232 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.232 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:41.233 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.234726685999998, 2024-09-10T10:27:41.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051621749999999997, 2024-09-10T10:27:41.233 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.233 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.233 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.234 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:41.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:41.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:41.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3600000000000003e-07 2024-09-10T10:27:41.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922011+0000", 2024-09-10T10:27:41.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4449999999999998e-06 2024-09-10T10:27:41.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922019+0000", 2024-09-10T10:27:41.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0930000000000003e-06 2024-09-10T10:27:41.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922025+0000", 2024-09-10T10:27:41.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5060000000000001e-06 2024-09-10T10:27:41.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922076+0000", 2024-09-10T10:27:41.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0671000000000003e-05 2024-09-10T10:27:41.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922175+0000", 2024-09-10T10:27:41.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9611000000000006e-05 2024-09-10T10:27:41.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:41.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922278+0000", 2024-09-10T10:27:41.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102565 2024-09-10T10:27:41.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925217+0000", 2024-09-10T10:27:41.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029384670000000002 2024-09-10T10:27:41.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927074+0000", 2024-09-10T10:27:41.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018579499999999999 2024-09-10T10:27:41.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927129+0000", 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4428000000000001e-05 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927168+0000", 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8803000000000001e-05 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.234691015999999, 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059957930000000001, 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922042+0000", 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922045+0000", 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4209999999999999e-06 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922052+0000", 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1430000000000001e-06 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922059+0000", 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2139999999999999e-06 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922495+0000", 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043517499999999999 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922563+0000", 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8084999999999996e-05 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922679+0000", 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011609599999999999 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927208+0000", 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045292170000000003 2024-09-10T10:27:41.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927952+0000", 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00074421700000000003 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928001+0000", 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9004e-05 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928037+0000", 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.578e-05 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.234664256999999, 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045268130000000002, 2024-09-10T10:27:41.250 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922078+0000", 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922085+0000", 2024-09-10T10:27:41.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3710000000000002e-06 2024-09-10T10:27:41.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922091+0000", 2024-09-10T10:27:41.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0340000000000002e-06 2024-09-10T10:27:41.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922144+0000", 2024-09-10T10:27:41.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3106999999999998e-05 2024-09-10T10:27:41.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922248+0000", 2024-09-10T10:27:41.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010373400000000001 2024-09-10T10:27:41.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:41.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922399+0000", 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015127200000000001 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925201+0000", 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028019189999999999 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926514+0000", 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001312279 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926561+0000", 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000002e-05 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926595+0000", 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3408000000000003e-05 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.234029409000001, 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005322116, 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:41.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:41.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999996e-07 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922707+0000", 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.766e-06 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922717+0000", 2024-09-10T10:27:41.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7229999999999999e-06 2024-09-10T10:27:41.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922724+0000", 2024-09-10T10:27:41.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8410000000000002e-06 2024-09-10T10:27:41.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922792+0000", 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8557000000000001e-05 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922912+0000", 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119354 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928025+0000", 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051131919999999999 2024-09-10T10:27:41.261 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.262 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.262 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.262 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.233553256, 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049271260000000004, 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:27:41.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923180+0000", 2024-09-10T10:27:41.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:27:41.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923183+0000", 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4000000000000001e-06 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923196+0000", 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3127e-05 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923203+0000", 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.939e-06 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923270+0000", 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7087000000000003e-05 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923359+0000", 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9054999999999994e-05 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928106+0000", 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047469239999999996 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.233092615, 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045860270000000003, 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.267 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2500000000000005e-07 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923644+0000", 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8240000000000001e-06 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923659+0000", 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5358000000000001e-05 2024-09-10T10:27:41.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923666+0000", 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9659999999999999e-06 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923725+0000", 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8674000000000001e-05 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923850+0000", 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000124797 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928226+0000", 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043757830000000003 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:41.271 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.21557018, 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005230859, 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:27:41.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941164+0000", 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.863e-06 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941172+0000", 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2360000000000004e-06 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:41.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941190+0000", 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7812000000000002e-05 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941200+0000", 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7599999999999997e-06 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941350+0000", 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015070400000000001 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:41.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941501+0000", 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015044200000000001 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000167076 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943487+0000", 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018190260000000001 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946297+0000", 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028102840000000001 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946349+0000", 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1649e-05 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946393+0000", 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4007000000000001e-05 2024-09-10T10:27:41.276 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.277 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:41.277 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:41.277 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:41.277 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:27:41.279 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:27:41.279 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:41.277+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:41.345 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:27:41.364 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:27:41.406 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:41.405+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:41.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.418+0000 7fc2b5a77640 1 Processor -- start 2024-09-10T10:27:41.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.418+0000 7fc2b5a77640 1 -- start start 2024-09-10T10:27:41.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2b5a77640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc2b0155df0 0x7fc2b0158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2b5a77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00ca7c0 0x7fc2b01631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2b5a77640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2b006bc10 con 0x7fc2b00c3f50 2024-09-10T10:27:41.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2b5a77640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc2b0073690 con 0x7fc2b0155df0 2024-09-10T10:27:41.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2b5a77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b015c6f0 con 0x7fc2b00ca7c0 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2a7fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fc2b00c3f50 0x7fc2b006e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:50604/0 (socket says 172.21.15.120:50604) 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2a7fff640 1 -- 172.21.15.120:0/261958516 learned_addr learned my addr 172.21.15.120:0/261958516 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1734855624 0 0) 0x7fc2b006bc10 con 0x7fc2b00c3f50 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2affff640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00ca7c0 0x7fc2b01631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc29c003520 con 0x7fc2b00c3f50 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.419+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/261958516 >> v2:172.21.15.134:3300/0 conn(0x7fc2b0155df0 0x7fc2b0158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1559352153 0 0) 0x7fc29c003520 con 0x7fc2b00c3f50 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00ca7c0 msgr2=0x7fc2b01631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00ca7c0 0x7fc2b01631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 >> v2:172.21.15.134:3300/0 conn(0x7fc2b0155df0 msgr2=0x7fc2b0158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 --2- 172.21.15.120:0/261958516 >> v2:172.21.15.134:3300/0 conn(0x7fc2b0155df0 0x7fc2b0158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b006bed0 con 0x7fc2b00c3f50 2024-09-10T10:27:41.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7fc2980017f0 con 0x7fc2b00c3f50 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 >> v1:172.21.15.120:6789/0 conn(0x7fc2b00c3f50 legacy=0x7fc2b006e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29c0039b0 0x7fc29c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00ca7c0 0x7fc29c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 0x7fc29c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0073690 con 0x7fc29c004d40 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b015c6f0 con 0x7fc29c0039b0 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc29c00b9f0 con 0x7fc2b00ca7c0 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2affff640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 0x7fc29c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29c0039b0 0x7fc29c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2a7fff640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00ca7c0 0x7fc29c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2affff640 1 -- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00ca7c0 msgr2=0x7fc29c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2affff640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00ca7c0 0x7fc29c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2affff640 1 -- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29c0039b0 msgr2=0x7fc29c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:41.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2affff640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29c0039b0 0x7fc29c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.420+0000 7fc2affff640 1 -- 172.21.15.120:0/261958516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c003520 con 0x7fc29c004d40 2024-09-10T10:27:41.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.421+0000 7fc2affff640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 0x7fc29c0051c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00c66a0 tx=0x7fc29c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d84a9972b473c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.421+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8037070 con 0x7fc29c004d40 2024-09-10T10:27:41.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.421+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a802e050 con 0x7fc29c004d40 2024-09-10T10:27:41.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.421+0000 7fc2aeffd640 1 -- 172.21.15.120:0/261958516 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8009c10 con 0x7fc29c004d40 2024-09-10T10:27:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.422+0000 7fc2b5a77640 1 -- 172.21.15.120:0/261958516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 msgr2=0x7fc29c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.422+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/261958516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 0x7fc29c0051c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00c66a0 tx=0x7fc29c005830 comp rx=0 tx=0).stop 2024-09-10T10:27:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.422+0000 7fc2affff640 1 -- 172.21.15.120:0/261958516 reap_dead start 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.422+0000 7fc2b5a77640 1 -- 172.21.15.120:0/261958516 shutdown_connections 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.422+0000 7fc2b5a77640 1 -- 172.21.15.120:0/261958516 >> 172.21.15.120:0/261958516 conn(0x7fc2b00728b0 msgr2=0x7fc2b0072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.422+0000 7fc2b5a77640 1 -- 172.21.15.120:0/261958516 shutdown_connections 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.422+0000 7fc2b5a77640 1 -- 172.21.15.120:0/261958516 wait complete. 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.423+0000 7fc2b5a77640 1 Processor -- start 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.423+0000 7fc2b5a77640 1 -- start start 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2b5a77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 0x7fc2b00c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2b5a77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00c9ae0 0x7fc2b00c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2b5a77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2b01514c0 0x7fc2b0151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2b5a77640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b015c380 con 0x7fc29c004d40 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2b5a77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b015c0a0 con 0x7fc2b01514c0 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2b5a77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b0158e60 con 0x7fc2b00c9ae0 2024-09-10T10:27:41.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 0x7fc2b00c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 0x7fc2b00c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45004/0 (socket says 172.21.15.120:45004) 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2af7fe640 1 -- 172.21.15.120:0/655923673 learned_addr learned my addr 172.21.15.120:0/655923673 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2affff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2b01514c0 0x7fc2b0151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2a7fff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00c9ae0 0x7fc2b00c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2af7fe640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00c9ae0 msgr2=0x7fc2b00c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2b00c9ae0 0x7fc2b00c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2af7fe640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2b01514c0 msgr2=0x7fc2b0151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2b01514c0 0x7fc2b0151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2af7fe640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b01f0c70 con 0x7fc29c004d40 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2affff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2b01514c0 0x7fc2b0151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 0x7fc2b00c8b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0011990 tx=0x7fc2a0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000fad0 con 0x7fc29c004d40 2024-09-10T10:27:41.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a000fc70 con 0x7fc29c004d40 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b00c4610 con 0x7fc29c004d40 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0014440 con 0x7fc29c004d40 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.424+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b00c4bd0 con 0x7fc29c004d40 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.425+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc2b01561d0 con 0x7fc29c004d40 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.426+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2a0012040 con 0x7fc29c004d40 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.426+0000 7fc2acff9640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc29006be60 0x7fc29006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.426+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6735+0+0 (secure 0 0 0) 0x7fc2a008b950 con 0x7fc29c004d40 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.426+0000 7fc2a7fff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc29006be60 0x7fc29006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.427+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v30) ==== 54+0+16 (secure 0 0 0) 0x7fc2a0019030 con 0x7fc29c004d40 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.427+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fc2b01f3980 0x7fc2b01f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.427+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2b01f42a0 con 0x7fc2b01f3980 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.427+0000 7fc2a7fff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc29006be60 0x7fc29006e320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0156d10 tx=0x7fc298009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.427+0000 7fc2affff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fc2b01f3980 0x7fc2b01f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.427+0000 7fc2affff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fc2b01f3980 0x7fc2b01f3d60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.432 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:41.432 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:41.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.428+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2b01f42a0 con 0x7fc2b01f3980 2024-09-10T10:27:41.439 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:27:41.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.442+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc2b01f0930 con 0x7fc2b01f3980 2024-09-10T10:27:41.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.444+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc2b01f0930 con 0x7fc2b01f3980 2024-09-10T10:27:41.446 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:41.446 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:41.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.444+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fc2b0062b40 0x7fc2b0062f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.444+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc2b00c4f40 con 0x7fc2b0062b40 2024-09-10T10:27:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.444+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fc2b0062b40 0x7fc2b0062f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.445+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fc2b0062b40 0x7fc2b0062f60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.445+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2b00c4f40 con 0x7fc2b0062b40 2024-09-10T10:27:41.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.462+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc2b00c5910 con 0x7fc2b0062b40 2024-09-10T10:27:41.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.463+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc2b00c5910 con 0x7fc2b0062b40 2024-09-10T10:27:41.464 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.463+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fc2b002cb50 0x7fc2b002cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.463+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc2b0152420 con 0x7fc2b002cb50 2024-09-10T10:27:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.464+0000 7fc2a7fff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fc2b002cb50 0x7fc2b002cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.464+0000 7fc2a7fff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fc2b002cb50 0x7fc2b002cf50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.465+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2b0152420 con 0x7fc2b002cb50 2024-09-10T10:27:41.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.478+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc2b01edee0 con 0x7fc2b002cb50 2024-09-10T10:27:41.480 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:41.479+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:41.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.480+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc2b01edee0 con 0x7fc2b002cb50 2024-09-10T10:27:41.481 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:41.481 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:41.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.480+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fc2b002d850 0x7fc2b0152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.480+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc2b01ee090 con 0x7fc2b002d850 2024-09-10T10:27:41.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.480+0000 7fc2affff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fc2b002d850 0x7fc2b0152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.480+0000 7fc2affff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fc2b002d850 0x7fc2b0152e10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.481+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2b01ee090 con 0x7fc2b002d850 2024-09-10T10:27:41.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.495+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc2b0153620 con 0x7fc2b002d850 2024-09-10T10:27:41.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.496+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc2b0153620 con 0x7fc2b002d850 2024-09-10T10:27:41.497 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:41.498 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:41.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.496+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fc2b0153c20 0x7fc2b02047f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.496+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc2b0154080 con 0x7fc2b0153c20 2024-09-10T10:27:41.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.497+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fc2b0153c20 0x7fc2b02047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.497+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fc2b0153c20 0x7fc2b02047f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.498+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2a005db20 con 0x7fc2b0153c20 2024-09-10T10:27:41.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.511+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc2b01ee090 con 0x7fc2b0153c20 2024-09-10T10:27:41.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.514+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc2b01ee090 con 0x7fc2b0153c20 2024-09-10T10:27:41.515 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:41.515 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:41.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.514+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fc2b0205240 0x7fc2b0207630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.514+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fc2b020a400 con 0x7fc2b0205240 2024-09-10T10:27:41.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.514+0000 7fc2a7fff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fc2b0205240 0x7fc2b0207630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.515+0000 7fc2a7fff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fc2b0205240 0x7fc2b0207630 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.515+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2b020a400 con 0x7fc2b0205240 2024-09-10T10:27:41.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.529+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc2b0207b70 con 0x7fc2b0205240 2024-09-10T10:27:41.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.531+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc2b0207b70 con 0x7fc2b0205240 2024-09-10T10:27:41.532 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:41.532 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:41.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.531+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fc2b02081f0 0x7fc2b020a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.531+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fc2b020d4c0 con 0x7fc2b02081f0 2024-09-10T10:27:41.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.531+0000 7fc2affff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fc2b02081f0 0x7fc2b020a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.531+0000 7fc2affff640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fc2b02081f0 0x7fc2b020a5e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.532+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2b020d4c0 con 0x7fc2b02081f0 2024-09-10T10:27:41.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.546+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc2b020ab20 con 0x7fc2b02081f0 2024-09-10T10:27:41.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.548+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc2b020ab20 con 0x7fc2b02081f0 2024-09-10T10:27:41.549 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:41.549 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:41.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.548+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fc2b020b240 0x7fc2b020b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.548+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fc2b0210910 con 0x7fc2b020b240 2024-09-10T10:27:41.550 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:41.550 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:41.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.548+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fc2b020b240 0x7fc2b020b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.549+0000 7fc2af7fe640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fc2b020b240 0x7fc2b020b6a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.550+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2b0210910 con 0x7fc2b020b240 2024-09-10T10:27:41.558 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:27:41.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.563+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc2b020f790 con 0x7fc2b020b240 2024-09-10T10:27:41.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2acff9640 1 -- 172.21.15.120:0/655923673 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc2b020f790 con 0x7fc2b020b240 2024-09-10T10:27:41.567 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:41.568 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fc2b01f3980 msgr2=0x7fc2b01f3d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fc2b01f3980 0x7fc2b01f3d60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fc2b0062b40 msgr2=0x7fc2b0062f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fc2b0062b40 0x7fc2b0062f60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fc2b002cb50 msgr2=0x7fc2b002cf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fc2b002cb50 0x7fc2b002cf50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fc2b002d850 msgr2=0x7fc2b0152e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fc2b002d850 0x7fc2b0152e10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fc2b0153c20 msgr2=0x7fc2b02047f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fc2b0153c20 0x7fc2b02047f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2affff640 1 -- 172.21.15.120:0/655923673 reap_dead start 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fc2b0205240 msgr2=0x7fc2b0207630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fc2b0205240 0x7fc2b0207630 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2affff640 1 -- 172.21.15.120:0/655923673 reap_dead start 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fc2b02081f0 msgr2=0x7fc2b020a5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fc2b02081f0 0x7fc2b020a5e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fc2b020b240 msgr2=0x7fc2b020b6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fc2b020b240 0x7fc2b020b6a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2affff640 1 -- 172.21.15.120:0/655923673 reap_dead start 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc29006be60 msgr2=0x7fc29006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc29006be60 0x7fc29006e320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0156d10 tx=0x7fc298009040 comp rx=0 tx=0).stop 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 msgr2=0x7fc2b00c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 0x7fc2b00c8b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0011990 tx=0x7fc2a0011e50 comp rx=0 tx=0).stop 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 shutdown_connections 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc29006be60 0x7fc29006e320 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 --2- 172.21.15.120:0/655923673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc29c004d40 0x7fc2b00c8b70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 >> 172.21.15.120:0/655923673 conn(0x7fc2b00728b0 msgr2=0x7fc2b015a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:41.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 shutdown_connections 2024-09-10T10:27:41.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.566+0000 7fc2b5a77640 1 -- 172.21.15.120:0/655923673 wait complete. 2024-09-10T10:27:41.580 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:41.579+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:41.667 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:41.667 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:41.676 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:27:41.681 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:41.681+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:41.717 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:27:41.718 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:27:41.718 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-10T10:27:41.718 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-10T10:27:41.782 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:41.782 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:41.782 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:41.781+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:41.791 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:27:41.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.820+0000 7f798a0ba640 1 Processor -- start 2024-09-10T10:27:41.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.821+0000 7f798a0ba640 1 -- start start 2024-09-10T10:27:41.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.821+0000 7f798a0ba640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7984151ba0 0x7f79840c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.821+0000 7f798a0ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79840c7390 0x7f79840cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.821+0000 7f798a0ba640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79840c6690 con 0x7f79841517c0 2024-09-10T10:27:41.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.821+0000 7f798a0ba640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f798406fe60 con 0x7f7984151ba0 2024-09-10T10:27:41.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.821+0000 7f798a0ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f79840c6970 con 0x7f79840c7390 2024-09-10T10:27:41.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.822+0000 7f7982ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7984151ba0 0x7f79840c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.822+0000 7f7982ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7984151ba0 0x7f79840c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45012/0 (socket says 172.21.15.120:45012) 2024-09-10T10:27:41.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.822+0000 7f7982ffd640 1 -- 172.21.15.120:0/2725755202 learned_addr learned my addr 172.21.15.120:0/2725755202 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:41.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.822+0000 7f7983fff640 1 --2- 172.21.15.120:0/2725755202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79840c7390 0x7f79840cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.822+0000 7f7982ffd640 1 -- 172.21.15.120:0/2725755202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79840c7390 msgr2=0x7f79840cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:41.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.822+0000 7f7982ffd640 1 --2- 172.21.15.120:0/2725755202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79840c7390 0x7f79840cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.822+0000 7f7982ffd640 1 -- 172.21.15.120:0/2725755202 >> v1:172.21.15.134:6789/0 conn(0x7f79841517c0 legacy=0x7f7984070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.822+0000 7f7982ffd640 1 -- 172.21.15.120:0/2725755202 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79840c6c50 con 0x7f7984151ba0 2024-09-10T10:27:41.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.823+0000 7f7982ffd640 1 --2- 172.21.15.120:0/2725755202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7984151ba0 0x7f79840c47d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7978009870 tx=0x7f797802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57707f63d813b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.823+0000 7f79827fc640 1 -- 172.21.15.120:0/2725755202 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978037070 con 0x7f7984151ba0 2024-09-10T10:27:41.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.823+0000 7f79827fc640 1 -- 172.21.15.120:0/2725755202 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f797802ae50 con 0x7f7984151ba0 2024-09-10T10:27:41.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.823+0000 7f79827fc640 1 -- 172.21.15.120:0/2725755202 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978032860 con 0x7f7984151ba0 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.824+0000 7f798a0ba640 1 -- 172.21.15.120:0/2725755202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7984151ba0 msgr2=0x7f79840c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.824+0000 7f798a0ba640 1 --2- 172.21.15.120:0/2725755202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7984151ba0 0x7f79840c47d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7978009870 tx=0x7f797802a300 comp rx=0 tx=0).stop 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.824+0000 7f798a0ba640 1 -- 172.21.15.120:0/2725755202 shutdown_connections 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.824+0000 7f798a0ba640 1 --2- 172.21.15.120:0/2725755202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79840c7390 0x7f79840cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.824+0000 7f798a0ba640 1 --2- 172.21.15.120:0/2725755202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7984151ba0 0x7f79840c47d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.824+0000 7f798a0ba640 1 -- 172.21.15.120:0/2725755202 >> 172.21.15.120:0/2725755202 conn(0x7f7984072a60 msgr2=0x7f7984072e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.824+0000 7f798a0ba640 1 -- 172.21.15.120:0/2725755202 shutdown_connections 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.824+0000 7f798a0ba640 1 -- 172.21.15.120:0/2725755202 wait complete. 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.825+0000 7f798a0ba640 1 Processor -- start 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.825+0000 7f798a0ba640 1 -- start start 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f798a0ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79840c7390 0x7f7984150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f798a0ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79841517c0 0x7f79840cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f798a0ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7984151ba0 0x7f79841f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f798a0ba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79840c8890 con 0x7f79841517c0 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f798a0ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79840c85b0 con 0x7f7984151ba0 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f798a0ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f79840c6330 con 0x7f79840c7390 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f7982ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79841517c0 0x7f79840cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f7982ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79841517c0 0x7f79840cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45024/0 (socket says 172.21.15.120:45024) 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f7982ffd640 1 -- 172.21.15.120:0/2310573564 learned_addr learned my addr 172.21.15.120:0/2310573564 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f7983fff640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7984151ba0 0x7f79841f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f79837fe640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79840c7390 0x7f7984150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f7982ffd640 1 -- 172.21.15.120:0/2310573564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79840c7390 msgr2=0x7f7984150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f7982ffd640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79840c7390 0x7f7984150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f7982ffd640 1 -- 172.21.15.120:0/2310573564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7984151ba0 msgr2=0x7f79841f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f7982ffd640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7984151ba0 0x7f79841f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:41.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f7982ffd640 1 -- 172.21.15.120:0/2310573564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798414ab00 con 0x7f79841517c0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f7982ffd640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79841517c0 0x7f79840cc740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f797802a340 tx=0x7f79780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f796bfff640 1 -- 172.21.15.120:0/2310573564 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978041020 con 0x7f79841517c0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.826+0000 7f796bfff640 1 -- 172.21.15.120:0/2310573564 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f797803c870 con 0x7f79841517c0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.827+0000 7f798a0ba640 1 -- 172.21.15.120:0/2310573564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798414ade0 con 0x7f79841517c0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.827+0000 7f796bfff640 1 -- 172.21.15.120:0/2310573564 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978037040 con 0x7f79841517c0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.827+0000 7f798a0ba640 1 -- 172.21.15.120:0/2310573564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79840cb630 con 0x7f79841517c0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.828+0000 7f798a0ba640 1 -- 172.21.15.120:0/2310573564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79840c8db0 con 0x7f79841517c0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.828+0000 7f796bfff640 1 -- 172.21.15.120:0/2310573564 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7978044020 con 0x7f79841517c0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.829+0000 7f796bfff640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f797006be60 0x7f797006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.829+0000 7f796bfff640 1 -- 172.21.15.120:0/2310573564 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6735+0+0 (secure 0 0 0) 0x7f79780ab9b0 con 0x7f79841517c0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.829+0000 7f79837fe640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f797006be60 0x7f797006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:41.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.829+0000 7f79837fe640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f797006be60 0x7f797006e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7974001640 tx=0x7f7974009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:41.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:41.833+0000 7f796bfff640 1 -- 172.21.15.120:0/2310573564 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f797804b070 con 0x7f79841517c0 2024-09-10T10:27:41.883 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:41.883+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:41.927 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:41.927 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:41.937 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:27:41.984 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:41.983+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:41.994 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-10T10:27:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.012+0000 7f798a0ba640 1 -- 172.21.15.120:0/2310573564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f79841e47b0 con 0x7f79841517c0 2024-09-10T10:27:42.015 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:42.015 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:42.015 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 30, 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-10T10:26:18.018818+0000", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-10T10:27:39.976206+0000", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-10T10:26:29.078139+0000", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-10T10:26:20.091201+0000", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5806080, 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "nodeep-scrub", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "noscrub", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-10T10:27:42.016 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 4, 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "luminous", 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-10T10:27:42.017 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-10T10:27:42.018 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-10T10:27:42.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.018 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-10T10:27:42.018 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-10T10:27:42.018 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:30.118649+0000", 2024-09-10T10:27:42.018 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:27:42.018 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-10T10:27:42.019 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:42.020 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:42.021 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:42.022 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-10T10:27:42.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:27:42.024 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_metadata", 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:38.936606+0000", 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:42.025 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:42.026 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "20", 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:42.027 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:42.029 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_bias": 4, 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 16, 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_priority": 5 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: "metadata": "cephfs" 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data", 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:40.094780+0000", 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 64, 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 64, 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 64, 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 64, 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 64, 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.05000000074505806 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.3799999952316284, 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.3799999952316284, 2024-09-10T10:27:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 4, 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data_ec", 2024-09-10T10:27:42.038 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:43.814005+0000", 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 5, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,ec_overwrites", 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 3, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 4, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 3, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 1, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:27:42.039 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "26", 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:42.040 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:42.041 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "cephfs_data_ec", 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:42.042 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 8192, 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.043 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e3b9c3c3-cf7d-4b33-913e-60e928439d8e", 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-10T10:27:42.044 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.046 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:42.047 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/3664783490", 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/3664783490", 2024-09-10T10:27:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/3664783490", 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/3664783490", 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "62c20a9d-e6ab-446d-998c-9ee46e46aa90", 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:42.049 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:42.050 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:42.050 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:42.050 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:42.050 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:42.050 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:42.050 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:42.050 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.013+0000 7f796bfff640 1 -- 172.21.15.120:0/2310573564 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v30) ==== 81+0+38887 (secure 0 0 0) 0x7f7978079f60 con 0x7f79841517c0 2024-09-10T10:27:42.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.051 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.051 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-10T10:27:42.051 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:42.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.051 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.051 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-10T10:27:42.051 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:42.052 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:42.053 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:42.054 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/3972761964", 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/3972761964", 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/3972761964", 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/3972761964", 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:27:42.055 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1528f8e0-5e13-4ec7-9d98-0d94b7a505c3", 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.056 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.057 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.058 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.059 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2916410859", 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2916410859", 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2916410859", 2024-09-10T10:27:42.060 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2916410859", 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "74942f78-c1fc-4886-abf0-950722c5edc6", 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:42.061 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:42.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.064 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.064 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-10T10:27:42.064 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:42.064 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.064 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.064 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:42.064 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.064 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.065 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1224278950", 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1224278950", 2024-09-10T10:27:42.066 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1224278950", 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1224278950", 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e71319d4-5318-4441-84a8-7665ae037596", 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:42.067 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6816", 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6817", 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:42.068 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6818", 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6819", 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:42.069 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6822", 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6823", 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.070 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6820", 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6821", 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6817/1874635282", 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6819/1874635282", 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6823/1874635282", 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6821/1874635282", 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "29e193cd-37af-43e5-bd0c-2f85ce0631e8", 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:42.072 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.073 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6808", 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6809", 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.074 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6810", 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6811", 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6814", 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6815", 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6812", 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6813", 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6809/457724087", 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6811/457724087", 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6815/457724087", 2024-09-10T10:27:42.077 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6813/457724087", 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "55523ec2-cf12-44b3-aeea-f1cedabc996a", 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:42.078 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6824", 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6825", 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6826", 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6827", 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.080 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6830", 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6831", 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.081 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6828", 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6829", 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.082 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6825/50133702", 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6827/50133702", 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6831/50133702", 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6829/50133702", 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "097b1149-3e3c-4e5a-890b-d4cc4361d51b", 2024-09-10T10:27:42.083 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 19, 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.084 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6800", 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6801", 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.085 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6802", 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6803", 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.086 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6806", 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6807", 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6804", 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6805", 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6801/251960875", 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6803/251960875", 2024-09-10T10:27:42.088 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6807/251960875", 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6805/251960875", 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:42.089 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.143056+0000", 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.423968+0000", 2024-09-10T10:27:42.090 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.640076+0000", 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:27:42.091 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.536160+0000", 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:42.092 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.146033+0000", 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:42.093 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.426011+0000", 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.598992+0000", 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.094 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.182929+0000", 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [ 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.095 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.16", 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 1 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.1d", 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:27:42.096 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.28", 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.36", 2024-09-10T10:27:42.097 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.d", 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 5 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.10", 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.1b", 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:27:42.099 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_data_ec": { 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-device-class": "", 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-num-failure-domains": "0", 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-osds-per-failure-domain": "0", 2024-09-10T10:27:42.100 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-root": "default", 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "jerasure-per-chunk-alignment": "false", 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "2", 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van", 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "w": "8" 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [], 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [], 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:42.102 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 -- 172.21.15.120:0/2310573564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f797006be60 msgr2=0x7f797006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f797006be60 0x7f797006e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7974001640 tx=0x7f7974009040 comp rx=0 tx=0).stop 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 -- 172.21.15.120:0/2310573564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79841517c0 msgr2=0x7f79840cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79841517c0 0x7f79840cc740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f797802a340 tx=0x7f79780046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 -- 172.21.15.120:0/2310573564 shutdown_connections 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f797006be60 0x7f797006e320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7984151ba0 0x7f79841f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79841517c0 0x7f79840cc740 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 --2- 172.21.15.120:0/2310573564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79840c7390 0x7f7984150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 -- 172.21.15.120:0/2310573564 >> 172.21.15.120:0/2310573564 conn(0x7f7984072a60 msgr2=0x7f79840c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 -- 172.21.15.120:0/2310573564 shutdown_connections 2024-09-10T10:27:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.016+0000 7f798a0ba640 1 -- 172.21.15.120:0/2310573564 wait complete. 2024-09-10T10:27:42.105 INFO:tasks.thrashosds.thrasher:j is {'epoch': 30, 'fsid': '7d0d0830-4370-41c3-9acc-b67b07024f68', 'created': '2024-09-10T10:26:18.018818+0000', 'modified': '2024-09-10T10:27:39.976206+0000', 'last_up_change': '2024-09-10T10:26:29.078139+0000', 'last_in_change': '2024-09-10T10:26:20.091201+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:26:30.118649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:26:38.936606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:26:40.094780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T10:26:43.814005+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'e3b9c3c3-cf7d-4b33-913e-60e928439d8e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 3664783490}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 3664783490}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 3664783490}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 3664783490}]}, 'public_addr': '172.21.15.120:6809/3664783490', 'cluster_addr': '172.21.15.120:6811/3664783490', 'heartbeat_back_addr': '172.21.15.120:6815/3664783490', 'heartbeat_front_addr': '172.21.15.120:6813/3664783490', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '62c20a9d-e6ab-446d-998c-9ee46e46aa90', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 3972761964}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 3972761964}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 3972761964}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 3972761964}]}, 'public_addr': '172.21.15.120:6801/3972761964', 'cluster_addr': '172.21.15.120:6803/3972761964', 'heartbeat_back_addr': '172.21.15.120:6807/3972761964', 'heartbeat_front_addr': '172.21.15.120:6805/3972761964', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1528f8e0-5e13-4ec7-9d98-0d94b7a505c3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2916410859}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2916410859}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2916410859}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2916410859}]}, 'public_addr': '172.21.15.120:6817/2916410859', 'cluster_addr': '172.21.15.120:6819/2916410859', 'heartbeat_back_addr': '172.21.15.120:6823/2916410859', 'heartbeat_front_addr': '172.21.15.120:6821/2916410859', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '74942f78-c1fc-4886-abf0-950722c5edc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1224278950}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1224278950}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1224278950}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1224278950}]}, 'public_addr': '172.21.15.120:6825/1224278950', 'cluster_addr': '172.21.15.120:6827/1224278950', 'heartbeat_back_addr': '172.21.15.120:6831/1224278950', 'heartbeat_front_addr': '172.21.15.120:6829/1224278950', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e71319d4-5318-4441-84a8-7665ae037596', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6816', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6817', 'nonce': 1874635282}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6818', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6819', 'nonce': 1874635282}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6822', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6823', 'nonce': 1874635282}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6820', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6821', 'nonce': 1874635282}]}, 'public_addr': '172.21.15.134:6817/1874635282', 'cluster_addr': '172.21.15.134:6819/1874635282', 'heartbeat_back_addr': '172.21.15.134:6823/1874635282', 'heartbeat_front_addr': '172.21.15.134:6821/1874635282', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '29e193cd-37af-43e5-bd0c-2f85ce0631e8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6808', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6809', 'nonce': 457724087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6810', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6811', 'nonce': 457724087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6814', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6815', 'nonce': 457724087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6812', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6813', 'nonce': 457724087}]}, 'public_addr': '172.21.15.134:6809/457724087', 'cluster_addr': '172.21.15.134:6811/457724087', 'heartbeat_back_addr': '172.21.15.134:6815/457724087', 'heartbeat_front_addr': '172.21.15.134:6813/457724087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '55523ec2-cf12-44b3-aeea-f1cedabc996a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6824', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6825', 'nonce': 50133702}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6826', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6827', 'nonce': 50133702}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6830', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6831', 'nonce': 50133702}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6828', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6829', 'nonce': 50133702}]}, 'public_addr': '172.21.15.134:6825/50133702', 'cluster_addr': '172.21.15.134:6827/50133702', 'heartbeat_back_addr': '172.21.15.134:6831/50133702', 'heartbeat_front_addr': '172.21.15.134:6829/50133702', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '097b1149-3e3c-4e5a-890b-d4cc4361d51b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6800', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6801', 'nonce': 251960875}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6802', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6803', 'nonce': 251960875}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6806', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6807', 'nonce': 251960875}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6804', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6805', 'nonce': 251960875}]}, 'public_addr': '172.21.15.134:6801/251960875', 'cluster_addr': '172.21.15.134:6803/251960875', 'heartbeat_back_addr': '172.21.15.134:6807/251960875', 'heartbeat_front_addr': '172.21.15.134:6805/251960875', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.143056+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.423968+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.640076+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.536160+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.146033+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.426011+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.598992+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.182929+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.16', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.1d', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.28', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.36', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.10', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'cephfs_data_ec': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '2', 'plugin': 'jerasure', 'technique': 'reed_sol_van', 'w': '8'}, 'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-10T10:27:42.105 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 4.10 2024-09-10T10:27:42.106 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 4.10 2024-09-10T10:27:42.107 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:42.107 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:42.107 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:27:42.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.114+0000 7f917b2c9640 1 Processor -- start 2024-09-10T10:27:42.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.114+0000 7f917b2c9640 1 -- start start 2024-09-10T10:27:42.116 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:42.115+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:42.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.115+0000 7f917b2c9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f91740c8df0 0x7f91741663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.115+0000 7f917b2c9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91740cf380 con 0x7f917406cd20 2024-09-10T10:27:42.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.115+0000 7f917b2c9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91740c9280 con 0x7f9174154f10 2024-09-10T10:27:42.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.115+0000 7f917b2c9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f917406da70 con 0x7f91740c8df0 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.115+0000 7f9178864640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f917406cd20 0x7f91740ce380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:50606/0 (socket says 172.21.15.120:50606) 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.115+0000 7f9178864640 1 -- 172.21.15.120:0/1695903142 learned_addr learned my addr 172.21.15.120:0/1695903142 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.116+0000 7f9179866640 1 --2- 172.21.15.120:0/1695903142 >> v2:172.21.15.134:3300/0 conn(0x7f91740c8df0 0x7f91741663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.116+0000 7f9173fff640 1 -- 172.21.15.120:0/1695903142 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3126862616 0 0) 0x7f9168001110 con 0x7f917406cd20 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.116+0000 7f9173fff640 1 -- 172.21.15.120:0/1695903142 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f915c003520 con 0x7f917406cd20 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.116+0000 7f9179866640 1 -- 172.21.15.120:0/1695903142 >> v1:172.21.15.134:6789/0 conn(0x7f9174154f10 legacy=0x7f917415e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.116+0000 7f9179866640 1 -- 172.21.15.120:0/1695903142 >> v1:172.21.15.120:6789/0 conn(0x7f917406cd20 legacy=0x7f91740ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.116+0000 7f9179866640 1 -- 172.21.15.120:0/1695903142 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917406dd50 con 0x7f91740c8df0 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.117+0000 7f9179866640 1 --2- 172.21.15.120:0/1695903142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91740c8df0 0x7f91741663a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f916c0097a0 tx=0x7f916c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a90a362516b0f49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.117+0000 7f9173fff640 1 -- 172.21.15.120:0/1695903142 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c037070 con 0x7f91740c8df0 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.117+0000 7f9173fff640 1 -- 172.21.15.120:0/1695903142 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f916c032640 con 0x7f91740c8df0 2024-09-10T10:27:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.117+0000 7f9173fff640 1 -- 172.21.15.120:0/1695903142 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c0329f0 con 0x7f91740c8df0 2024-09-10T10:27:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.117+0000 7f917b2c9640 1 -- 172.21.15.120:0/1695903142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91740c8df0 msgr2=0x7f91741663a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.117+0000 7f917b2c9640 1 --2- 172.21.15.120:0/1695903142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91740c8df0 0x7f91741663a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f916c0097a0 tx=0x7f916c02a2c0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.117+0000 7f917b2c9640 1 -- 172.21.15.120:0/1695903142 shutdown_connections 2024-09-10T10:27:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.117+0000 7f917b2c9640 1 --2- 172.21.15.120:0/1695903142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91740c8df0 0x7f91741663a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.117+0000 7f917b2c9640 1 -- 172.21.15.120:0/1695903142 >> 172.21.15.120:0/1695903142 conn(0x7f91740716e0 msgr2=0x7f9174071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.118+0000 7f917b2c9640 1 -- 172.21.15.120:0/1695903142 shutdown_connections 2024-09-10T10:27:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.118+0000 7f917b2c9640 1 -- 172.21.15.120:0/1695903142 wait complete. 2024-09-10T10:27:42.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f917b2c9640 1 Processor -- start 2024-09-10T10:27:42.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f917b2c9640 1 -- start start 2024-09-10T10:27:42.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f917b2c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f917406cd20 0x7f91741f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f917b2c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91740c8df0 0x7f91741f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f917b2c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174154f10 0x7f9174157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f917b2c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f917415b3c0 con 0x7f9174154f10 2024-09-10T10:27:42.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f917b2c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f917415b0e0 con 0x7f91740c8df0 2024-09-10T10:27:42.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f917b2c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f917406d730 con 0x7f917406cd20 2024-09-10T10:27:42.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f9179866640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174154f10 0x7f9174157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f9179866640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174154f10 0x7f9174157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45040/0 (socket says 172.21.15.120:45040) 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.119+0000 7f9179866640 1 -- 172.21.15.120:0/3625810009 learned_addr learned my addr 172.21.15.120:0/3625810009 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9178864640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91740c8df0 0x7f91741f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9179065640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f917406cd20 0x7f91741f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9179866640 1 -- 172.21.15.120:0/3625810009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f917406cd20 msgr2=0x7f91741f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9179866640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f917406cd20 0x7f91741f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9179866640 1 -- 172.21.15.120:0/3625810009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91740c8df0 msgr2=0x7f91741f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9179866640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91740c8df0 0x7f91741f1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9179866640 1 -- 172.21.15.120:0/3625810009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91741f23d0 con 0x7f9174154f10 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9179866640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174154f10 0x7f9174157a10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f916c0097a0 tx=0x7f916c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9171ffb640 1 -- 172.21.15.120:0/3625810009 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c037050 con 0x7f9174154f10 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9171ffb640 1 -- 172.21.15.120:0/3625810009 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f916c038040 con 0x7f9174154f10 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f9171ffb640 1 -- 172.21.15.120:0/3625810009 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c006e10 con 0x7f9174154f10 2024-09-10T10:27:42.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f917b2c9640 1 -- 172.21.15.120:0/3625810009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91740c5e50 con 0x7f9174154f10 2024-09-10T10:27:42.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.120+0000 7f917b2c9640 1 -- 172.21.15.120:0/3625810009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9174155ad0 con 0x7f9174154f10 2024-09-10T10:27:42.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.121+0000 7f917b2c9640 1 -- 172.21.15.120:0/3625810009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f917415aab0 con 0x7f9174154f10 2024-09-10T10:27:42.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.122+0000 7f9171ffb640 1 -- 172.21.15.120:0/3625810009 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f916c004250 con 0x7f9174154f10 2024-09-10T10:27:42.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.122+0000 7f9171ffb640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f916006be60 0x7f916006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.122+0000 7f9171ffb640 1 -- 172.21.15.120:0/3625810009 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6735+0+0 (secure 0 0 0) 0x7f916c0abcf0 con 0x7f9174154f10 2024-09-10T10:27:42.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.122+0000 7f9179065640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f916006be60 0x7f916006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.123+0000 7f9179065640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f916006be60 0x7f916006e320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9164001e70 tx=0x7f9164009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:42.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.127+0000 7f9171ffb640 1 -- 172.21.15.120:0/3625810009 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f916c0b0070 con 0x7f9174154f10 2024-09-10T10:27:42.208 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:42.207+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:42.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.216+0000 7f5da4b78640 1 Processor -- start 2024-09-10T10:27:42.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.216+0000 7f5da4b78640 1 -- start start 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5da4b78640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5da0158130 0x7f5da006d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5da4b78640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da00c4b80 con 0x7f5da0151b30 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5da4b78640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da006ce50 con 0x7f5da00c72e0 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5da4b78640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5da00c45c0 con 0x7f5da0158130 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9e59c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f5da0151b30 0x7f5da00c60c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:50610/0 (socket says 172.21.15.120:50610) 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9e59c640 1 -- 172.21.15.120:0/2128946693 learned_addr learned my addr 172.21.15.120:0/2128946693 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9ed9d640 1 --2- 172.21.15.120:0/2128946693 >> v2:172.21.15.134:3300/0 conn(0x7f5da0158130 0x7f5da006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677992586 0 0) 0x7f5d90001110 con 0x7f5da0151b30 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d8c003520 con 0x7f5da0151b30 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3620997885 0 0) 0x7f5da006ce50 con 0x7f5da00c72e0 2024-09-10T10:27:42.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d90001110 con 0x7f5da00c72e0 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 162971431 0 0) 0x7f5d8c003520 con 0x7f5da0151b30 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 >> v2:172.21.15.134:3300/0 conn(0x7f5da0158130 msgr2=0x7f5da006d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 --2- 172.21.15.120:0/2128946693 >> v2:172.21.15.134:3300/0 conn(0x7f5da0158130 0x7f5da006d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 >> v1:172.21.15.134:6789/0 conn(0x7f5da00c72e0 legacy=0x7f5da0157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da00c8bc0 con 0x7f5da0151b30 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f5d900017c0 con 0x7f5da0151b30 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 >> v1:172.21.15.120:6789/0 conn(0x7f5da0151b30 legacy=0x7f5da00c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8c003b90 0x7f5d8c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d8c004800 0x7f5d8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d8c0051c0 0x7f5d8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00c45c0 con 0x7f5d8c004800 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.217+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c00c090 con 0x7f5d8c0051c0 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.218+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8c00c290 con 0x7f5d8c003b90 2024-09-10T10:27:42.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.218+0000 7f5d97fff640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d8c004800 0x7f5d8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.218+0000 7f5d9e59c640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8c003b90 0x7f5d8c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.218+0000 7f5d9ed9d640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d8c0051c0 0x7f5d8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.218+0000 7f5d97fff640 1 -- 172.21.15.120:0/2128946693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8c003b90 msgr2=0x7f5d8c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.218+0000 7f5d97fff640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8c003b90 0x7f5d8c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.218+0000 7f5d97fff640 1 -- 172.21.15.120:0/2128946693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d8c0051c0 msgr2=0x7f5d8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.218+0000 7f5d97fff640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d8c0051c0 0x7f5d8c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.218+0000 7f5d97fff640 1 -- 172.21.15.120:0/2128946693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c003520 con 0x7f5d8c004800 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.218+0000 7f5d9ed9d640 1 -- 172.21.15.120:0/2128946693 reap_dead start 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5d97fff640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d8c004800 0x7f5d8c004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5d880018a0 tx=0x7f5d8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cb6f10b4a64ece server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d880014f0 con 0x7f5d8c004800 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d880052d0 con 0x7f5d8c004800 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5d9dd9b640 1 -- 172.21.15.120:0/2128946693 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8802ac00 con 0x7f5d8c004800 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5da4b78640 1 -- 172.21.15.120:0/2128946693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d8c004800 msgr2=0x7f5d8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5da4b78640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d8c004800 0x7f5d8c004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5d880018a0 tx=0x7f5d8c005830 comp rx=0 tx=0).stop 2024-09-10T10:27:42.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5da4b78640 1 -- 172.21.15.120:0/2128946693 shutdown_connections 2024-09-10T10:27:42.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5da4b78640 1 --2- 172.21.15.120:0/2128946693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d8c004800 0x7f5d8c004c80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5da4b78640 1 -- 172.21.15.120:0/2128946693 >> 172.21.15.120:0/2128946693 conn(0x7f5da0067950 msgr2=0x7f5da0067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:42.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5da4b78640 1 -- 172.21.15.120:0/2128946693 shutdown_connections 2024-09-10T10:27:42.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.219+0000 7f5da4b78640 1 -- 172.21.15.120:0/2128946693 wait complete. 2024-09-10T10:27:42.224 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:42.225 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:42.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5da4b78640 1 Processor -- start 2024-09-10T10:27:42.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5da4b78640 1 -- start start 2024-09-10T10:27:42.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5da4b78640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8c004800 0x7f5da0156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5da4b78640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5da0157240 0x7f5da00cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5da4b78640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da00cc9a0 0x7f5da01528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5da4b78640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5da006d3c0 con 0x7f5da0157240 2024-09-10T10:27:42.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5da4b78640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00c79c0 con 0x7f5da00cc9a0 2024-09-10T10:27:42.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5da4b78640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5da006d540 con 0x7f5d8c004800 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5d97fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5da0157240 0x7f5da00cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5d97fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5da0157240 0x7f5da00cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45060/0 (socket says 172.21.15.120:45060) 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5d97fff640 1 -- 172.21.15.120:0/691148242 learned_addr learned my addr 172.21.15.120:0/691148242 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.221+0000 7f5d9e59c640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8c004800 0x7f5da0156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d9ed9d640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da00cc9a0 0x7f5da01528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d97fff640 1 -- 172.21.15.120:0/691148242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8c004800 msgr2=0x7f5da0156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d97fff640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8c004800 0x7f5da0156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d97fff640 1 -- 172.21.15.120:0/691148242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da00cc9a0 msgr2=0x7f5da01528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d97fff640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da00cc9a0 0x7f5da01528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d97fff640 1 -- 172.21.15.120:0/691148242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da01e3330 con 0x7f5da0157240 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d9ed9d640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da00cc9a0 0x7f5da01528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d97fff640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5da0157240 0x7f5da00cc460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d8802a690 tx=0x7f5d880012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d96ffd640 1 -- 172.21.15.120:0/691148242 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88037040 con 0x7f5da0157240 2024-09-10T10:27:42.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d96ffd640 1 -- 172.21.15.120:0/691148242 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d88006e50 con 0x7f5da0157240 2024-09-10T10:27:42.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5da4b78640 1 -- 172.21.15.120:0/691148242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da01e4200 con 0x7f5da0157240 2024-09-10T10:27:42.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5d96ffd640 1 -- 172.21.15.120:0/691148242 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d880014f0 con 0x7f5da0157240 2024-09-10T10:27:42.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.222+0000 7f5da4b78640 1 -- 172.21.15.120:0/691148242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da01e3890 con 0x7f5da0157240 2024-09-10T10:27:42.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.223+0000 7f5da4b78640 1 -- 172.21.15.120:0/691148242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da01535f0 con 0x7f5da0157240 2024-09-10T10:27:42.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.223+0000 7f5d96ffd640 1 -- 172.21.15.120:0/691148242 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d88042020 con 0x7f5da0157240 2024-09-10T10:27:42.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.224+0000 7f5d96ffd640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5d8006be30 0x7f5d8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:42.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.224+0000 7f5d9e59c640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5d8006be30 0x7f5d8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:42.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.224+0000 7f5d96ffd640 1 -- 172.21.15.120:0/691148242 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6735+0+0 (secure 0 0 0) 0x7f5d880aa950 con 0x7f5da0157240 2024-09-10T10:27:42.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.225+0000 7f5d9e59c640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5d8006be30 0x7f5d8006e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d90000c00 tx=0x7f5d900078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:42.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.228+0000 7f5d96ffd640 1 -- 172.21.15.120:0/691148242 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5da01e3330 con 0x7f5da0157240 2024-09-10T10:27:42.234 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:27:42.308 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:42.307+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:42.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.310+0000 7f917b2c9640 1 -- 172.21.15.120:0/3625810009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f91740c9470 con 0x7f9174154f10 2024-09-10T10:27:42.337 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:42.337 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:42.346 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:27:42.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.408+0000 7f5da4b78640 1 -- 172.21.15.120:0/691148242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd rm-pg-upmap-items", "pgid": "4.10"} v 0) -- 0x7f5da00cda30 con 0x7f5da0157240 2024-09-10T10:27:42.430 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:42.429+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:42.488 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:42.488 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:42.497 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:27:42.539 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:42.538+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:42.610 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:42.609+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:42.640 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:42.640 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:42.649 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:27:42.711 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:42.710+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:42.791 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:42.791 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:42.799 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:27:42.812 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:42.811+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:42.902 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:42.902 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:42.910 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:27:42.912 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:42.911+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:42.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.990+0000 7f9171ffb640 1 -- 172.21.15.120:0/3625810009 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v31) ==== 91+0+0 (secure 0 0 0) 0x7f916c079ce0 con 0x7f9174154f10 2024-09-10T10:27:42.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.990+0000 7f5d96ffd640 1 -- 172.21.15.120:0/691148242 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm-pg-upmap-items", "pgid": "4.10"}]=0 clear 4.10 pg_upmap_items mapping v31) ==== 118+0+0 (secure 0 0 0) 0x7f5d88078f00 con 0x7f5da0157240 2024-09-10T10:27:42.992 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-10T10:27:42.992 INFO:teuthology.orchestra.run.smithi120.stderr:clear 4.10 pg_upmap_items mapping 2024-09-10T10:27:42.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 -- 172.21.15.120:0/691148242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5d8006be30 msgr2=0x7f5d8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5d8006be30 0x7f5d8006e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d90000c00 tx=0x7f5d900078f0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 -- 172.21.15.120:0/691148242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5da0157240 msgr2=0x7f5da00cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5da0157240 0x7f5da00cc460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d8802a690 tx=0x7f5d880012d0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 -- 172.21.15.120:0/691148242 shutdown_connections 2024-09-10T10:27:42.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5d8006be30 0x7f5d8006e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da00cc9a0 0x7f5da01528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5da0157240 0x7f5da00cc460 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 -- 172.21.15.120:0/3625810009 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f916006be60 msgr2=0x7f916006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f916006be60 0x7f916006e320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9164001e70 tx=0x7f9164009040 comp rx=0 tx=0).stop 2024-09-10T10:27:42.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 --2- 172.21.15.120:0/691148242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d8c004800 0x7f5da0156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 -- 172.21.15.120:0/3625810009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174154f10 msgr2=0x7f9174157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:42.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174154f10 0x7f9174157a10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f916c0097a0 tx=0x7f916c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 -- 172.21.15.120:0/691148242 >> 172.21.15.120:0/691148242 conn(0x7f5da0067950 msgr2=0x7f5da0070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:42.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 -- 172.21.15.120:0/3625810009 shutdown_connections 2024-09-10T10:27:42.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f916006be60 0x7f916006e320 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9174154f10 0x7f9174157a10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91740c8df0 0x7f91741f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 --2- 172.21.15.120:0/3625810009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f917406cd20 0x7f91741f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:42.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 -- 172.21.15.120:0/3625810009 >> 172.21.15.120:0/3625810009 conn(0x7f91740716e0 msgr2=0x7f91741e4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:42.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 -- 172.21.15.120:0/691148242 shutdown_connections 2024-09-10T10:27:42.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 -- 172.21.15.120:0/3625810009 shutdown_connections 2024-09-10T10:27:42.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f917b2c9640 1 -- 172.21.15.120:0/3625810009 wait complete. 2024-09-10T10:27:42.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:42.993+0000 7f5da4b78640 1 -- 172.21.15.120:0/691148242 wait complete. 2024-09-10T10:27:43.012 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:43.011+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:43.017 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:43.017 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:43.026 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:27:43.113 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:43.112+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:43.169 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:43.169 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:43.178 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:27:43.221 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:43.220+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:43.281 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:43.281 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:43.289 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:27:43.331 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:43.330+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:43.416 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:43.415+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:43.433 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:43.433 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:43.442 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:27:43.517 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:43.516+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:43.545 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:43.545 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:43.554 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:27:43.578 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:27:43.623 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:43.622+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:43.656 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:43.657 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:43.665 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:27:43.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.687+0000 7fafeeff6640 1 Processor -- start 2024-09-10T10:27:43.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.687+0000 7fafeeff6640 1 -- start start 2024-09-10T10:27:43.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.688+0000 7fafeeff6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fafe80c36b0 0x7fafe8071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.688+0000 7fafeeff6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe81585b0 con 0x7fafe8151820 2024-09-10T10:27:43.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.688+0000 7fafeeff6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe81582d0 con 0x7fafe80c7980 2024-09-10T10:27:43.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.688+0000 7fafeeff6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fafe81580a0 con 0x7fafe80c36b0 2024-09-10T10:27:43.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.688+0000 7fafed593640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fafe80c36b0 0x7fafe8071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.688+0000 7fafed593640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fafe80c36b0 0x7fafe8071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45062/0 (socket says 172.21.15.120:45062) 2024-09-10T10:27:43.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.688+0000 7fafecd92640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fafe8151820 0x7fafe80c8070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:50616/0 (socket says 172.21.15.120:50616) 2024-09-10T10:27:43.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.688+0000 7fafed593640 1 -- 172.21.15.120:0/172613160 learned_addr learned my addr 172.21.15.120:0/172613160 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:43.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.689+0000 7fafed593640 1 -- 172.21.15.120:0/172613160 >> v1:172.21.15.134:6789/0 conn(0x7fafe80c7980 legacy=0x7fafe80c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.689+0000 7fafed593640 1 -- 172.21.15.120:0/172613160 >> v1:172.21.15.120:6789/0 conn(0x7fafe8151820 legacy=0x7fafe80c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.689+0000 7fafed593640 1 -- 172.21.15.120:0/172613160 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe80c6bc0 con 0x7fafe80c36b0 2024-09-10T10:27:43.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.689+0000 7fafe77fe640 1 -- 172.21.15.120:0/172613160 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715874920 0 0) 0x7fafe81585b0 con 0x7fafe8151820 2024-09-10T10:27:43.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.690+0000 7fafed593640 1 --2- 172.21.15.120:0/172613160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 0x7fafe8071a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009e50 tx=0x7fafe002a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ed3cbd46ab501b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.690+0000 7fafe77fe640 1 -- 172.21.15.120:0/172613160 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0037070 con 0x7fafe80c36b0 2024-09-10T10:27:43.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.690+0000 7fafe77fe640 1 -- 172.21.15.120:0/172613160 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe0032630 con 0x7fafe80c36b0 2024-09-10T10:27:43.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.690+0000 7fafe77fe640 1 -- 172.21.15.120:0/172613160 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe002e050 con 0x7fafe80c36b0 2024-09-10T10:27:43.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.691+0000 7fafeeff6640 1 -- 172.21.15.120:0/172613160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 msgr2=0x7fafe8071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.691+0000 7fafeeff6640 1 --2- 172.21.15.120:0/172613160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 0x7fafe8071a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009e50 tx=0x7fafe002a840 comp rx=0 tx=0).stop 2024-09-10T10:27:43.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.691+0000 7fafeeff6640 1 -- 172.21.15.120:0/172613160 shutdown_connections 2024-09-10T10:27:43.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.691+0000 7fafeeff6640 1 --2- 172.21.15.120:0/172613160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 0x7fafe8071a10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.691+0000 7fafeeff6640 1 -- 172.21.15.120:0/172613160 >> 172.21.15.120:0/172613160 conn(0x7fafe806bc10 msgr2=0x7fafe806bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:43.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.691+0000 7fafeeff6640 1 -- 172.21.15.120:0/172613160 shutdown_connections 2024-09-10T10:27:43.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.691+0000 7fafeeff6640 1 -- 172.21.15.120:0/172613160 wait complete. 2024-09-10T10:27:43.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.692+0000 7fafeeff6640 1 Processor -- start 2024-09-10T10:27:43.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.692+0000 7fafeeff6640 1 -- start start 2024-09-10T10:27:43.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafeeff6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 0x7fafe806eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafeeff6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c7980 0x7fafe81f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafeeff6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe8151820 0x7fafe81f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafeeff6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafe80c74f0 con 0x7fafe80c36b0 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafeeff6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fafe80c6680 con 0x7fafe8151820 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafeeff6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fafe80c7180 con 0x7fafe80c7980 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafecd92640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 0x7fafe806eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafecd92640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 0x7fafe806eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45066/0 (socket says 172.21.15.120:45066) 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafecd92640 1 -- 172.21.15.120:0/3910244750 learned_addr learned my addr 172.21.15.120:0/3910244750 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafed593640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe8151820 0x7fafe81f0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafe7fff640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c7980 0x7fafe81f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafecd92640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c7980 msgr2=0x7fafe81f0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafecd92640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c7980 0x7fafe81f0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafecd92640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe8151820 msgr2=0x7fafe81f0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafecd92640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe8151820 0x7fafe81f0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.693+0000 7fafecd92640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe814a1d0 con 0x7fafe80c36b0 2024-09-10T10:27:43.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.694+0000 7fafecd92640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 0x7fafe806eef0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00c4a0 tx=0x7fafdc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.694+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc018070 con 0x7fafe80c36b0 2024-09-10T10:27:43.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.694+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafdc0074e0 con 0x7fafe80c36b0 2024-09-10T10:27:43.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.694+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe814a4b0 con 0x7fafe80c36b0 2024-09-10T10:27:43.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.694+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc004a20 con 0x7fafe80c36b0 2024-09-10T10:27:43.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.694+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe8146ed0 con 0x7fafe80c36b0 2024-09-10T10:27:43.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.695+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fafe8151c00 con 0x7fafe80c36b0 2024-09-10T10:27:43.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.695+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafdc01b070 con 0x7fafe80c36b0 2024-09-10T10:27:43.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.696+0000 7fafe57fa640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fafcc06be60 0x7fafcc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.696+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6706+0+0 (secure 0 0 0) 0x7fafdc08a400 con 0x7fafe80c36b0 2024-09-10T10:27:43.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.696+0000 7fafe7fff640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fafcc06be60 0x7fafcc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.696+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v31) ==== 54+0+16 (secure 0 0 0) 0x7fafdc08ca80 con 0x7fafe80c36b0 2024-09-10T10:27:43.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.696+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fafe8149910 0x7fafe8156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.696+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fafe8150db0 con 0x7fafe8149910 2024-09-10T10:27:43.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.697+0000 7fafed593640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fafe8149910 0x7fafe8156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.697+0000 7fafe7fff640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fafcc06be60 0x7fafcc06e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fafe8152c80 tx=0x7fafd4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.697+0000 7fafed593640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fafe8149910 0x7fafe8156d80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.698+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafe8150db0 con 0x7fafe8149910 2024-09-10T10:27:43.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.712+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fafe81572c0 con 0x7fafe8149910 2024-09-10T10:27:43.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.713+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fafe81572c0 con 0x7fafe8149910 2024-09-10T10:27:43.715 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:43.715 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:43.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.714+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fafe8154d60 0x7fafe8141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.714+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fafe8141840 con 0x7fafe8154d60 2024-09-10T10:27:43.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.714+0000 7fafecd92640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fafe8154d60 0x7fafe8141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.714+0000 7fafecd92640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fafe8154d60 0x7fafe8141300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.715+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafe8141840 con 0x7fafe8154d60 2024-09-10T10:27:43.718 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:43.718+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.729+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fafe806f430 con 0x7fafe8154d60 2024-09-10T10:27:43.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.732+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fafe806f430 con 0x7fafe8154d60 2024-09-10T10:27:43.733 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.732+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fafe81f4140 0x7fafe81f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.732+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fafe81e3c60 con 0x7fafe81f4140 2024-09-10T10:27:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.732+0000 7fafe7fff640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fafe81f4140 0x7fafe81f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.732+0000 7fafe7fff640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fafe81f4140 0x7fafe81f4520 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.733+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafe81e3c60 con 0x7fafe81f4140 2024-09-10T10:27:43.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.747+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fafe81e4140 con 0x7fafe81f4140 2024-09-10T10:27:43.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.748+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fafe81e4140 con 0x7fafe81f4140 2024-09-10T10:27:43.749 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:43.750 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:43.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.748+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fafe8062630 0x7fafe8062aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.748+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fafe81e4c20 con 0x7fafe8062630 2024-09-10T10:27:43.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.749+0000 7fafed593640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fafe8062630 0x7fafe8062aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.749+0000 7fafed593640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fafe8062630 0x7fafe8062aa0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.749+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafe0046e50 con 0x7fafe8062630 2024-09-10T10:27:43.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.763+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fafe8063230 con 0x7fafe8062630 2024-09-10T10:27:43.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.765+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fafe0046e50 con 0x7fafe8062630 2024-09-10T10:27:43.766 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:43.767 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:43.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.765+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fafe802cb50 0x7fafe802cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.765+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fafe80635b0 con 0x7fafe802cb50 2024-09-10T10:27:43.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.765+0000 7fafecd92640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fafe802cb50 0x7fafe802cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.766+0000 7fafecd92640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fafe802cb50 0x7fafe802cfb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.766+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafe80635b0 con 0x7fafe802cb50 2024-09-10T10:27:43.777 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:43.777 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:43.777 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:43.777 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.780+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fafe80635b0 con 0x7fafe802cb50 2024-09-10T10:27:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.782+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fafe80635b0 con 0x7fafe802cb50 2024-09-10T10:27:43.783 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.782+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fafe8204e40 0x7fafe8207230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.782+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fafe802da00 con 0x7fafe8204e40 2024-09-10T10:27:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.782+0000 7fafe7fff640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fafe8204e40 0x7fafe8207230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.782+0000 7fafe7fff640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fafe8204e40 0x7fafe8207230 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.783+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafe802da00 con 0x7fafe8204e40 2024-09-10T10:27:43.786 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:27:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.797+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fafe802da00 con 0x7fafe8204e40 2024-09-10T10:27:43.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.798+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fafe802da00 con 0x7fafe8204e40 2024-09-10T10:27:43.800 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.799+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fafe8207c80 0x7fafe820a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.799+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fafe820ce40 con 0x7fafe8207c80 2024-09-10T10:27:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.799+0000 7fafed593640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fafe8207c80 0x7fafe820a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.799+0000 7fafed593640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fafe8207c80 0x7fafe820a070 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.800+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafe820ce40 con 0x7fafe8207c80 2024-09-10T10:27:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.813+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fafe820a5b0 con 0x7fafe8207c80 2024-09-10T10:27:43.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.815+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fafe820a5b0 con 0x7fafe8207c80 2024-09-10T10:27:43.816 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.815+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fafe820ac30 0x7fafe820d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.815+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fafe820fe70 con 0x7fafe820ac30 2024-09-10T10:27:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.816+0000 7fafecd92640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fafe820ac30 0x7fafe820d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.816+0000 7fafecd92640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fafe820ac30 0x7fafe820d020 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:43.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.817+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafe820fe70 con 0x7fafe820ac30 2024-09-10T10:27:43.818 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:43.817+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:43.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.830+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fafe820d5e0 con 0x7fafe820ac30 2024-09-10T10:27:43.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.832+0000 7fafe57fa640 1 -- 172.21.15.120:0/3910244750 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fafe820d5e0 con 0x7fafe820ac30 2024-09-10T10:27:43.833 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:43.835 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:43.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fafe8149910 msgr2=0x7fafe8156d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fafe8149910 0x7fafe8156d80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fafe8154d60 msgr2=0x7fafe8141300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fafe8154d60 0x7fafe8141300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fafe81f4140 msgr2=0x7fafe81f4520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fafe81f4140 0x7fafe81f4520 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fafe8062630 msgr2=0x7fafe8062aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fafe8062630 0x7fafe8062aa0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fafe802cb50 msgr2=0x7fafe802cfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fafe802cb50 0x7fafe802cfb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafed593640 1 -- 172.21.15.120:0/3910244750 reap_dead start 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fafe8204e40 msgr2=0x7fafe8207230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fafe8204e40 0x7fafe8207230 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafed593640 1 -- 172.21.15.120:0/3910244750 reap_dead start 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fafe8207c80 msgr2=0x7fafe820a070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fafe8207c80 0x7fafe820a070 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fafe820ac30 msgr2=0x7fafe820d020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fafe820ac30 0x7fafe820d020 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafed593640 1 -- 172.21.15.120:0/3910244750 reap_dead start 2024-09-10T10:27:43.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fafcc06be60 msgr2=0x7fafcc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fafcc06be60 0x7fafcc06e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fafe8152c80 tx=0x7fafd4009040 comp rx=0 tx=0).stop 2024-09-10T10:27:43.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 msgr2=0x7fafe806eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:43.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 0x7fafe806eef0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00c4a0 tx=0x7fafdc00c960 comp rx=0 tx=0).stop 2024-09-10T10:27:43.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 shutdown_connections 2024-09-10T10:27:43.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fafcc06be60 0x7fafcc06e320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 --2- 172.21.15.120:0/3910244750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafe80c36b0 0x7fafe806eef0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:43.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 >> 172.21.15.120:0/3910244750 conn(0x7fafe806bc10 msgr2=0x7fafe8150270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:43.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 shutdown_connections 2024-09-10T10:27:43.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:43.833+0000 7fafeeff6640 1 -- 172.21.15.120:0/3910244750 wait complete. 2024-09-10T10:27:43.902 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:43.902 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:43.902 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:43.903 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:43.903 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:43.911 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:27:43.919 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:43.919+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:1 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.748037241000006, 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028809700000000001, 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.018 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:44.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9999999999999999e-07 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268119+0000", 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1810000000000003e-06 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268129+0000", 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9790000000000005e-06 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268137+0000", 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4800000000000001e-06 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268198+0000", 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0918000000000002e-05 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268226+0000", 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7267000000000002e-05 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270943+0000", 2024-09-10T10:27:44.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027177529999999998 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270996+0000", 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2191999999999998e-05 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:119 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'81, mlcod=16'78)", 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.522186288, 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029329880000000001, 2024-09-10T10:27:44.022 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493966+0000", 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1699999999999998e-07 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494040+0000", 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3883999999999999e-05 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494048+0000", 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4880000000000007e-06 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494057+0000", 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1310000000000006e-06 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494095+0000", 2024-09-10T10:27:44.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8470000000000003e-05 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494108+0000", 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2588999999999999e-05 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496869+0000", 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027610400000000002 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496898+0000", 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9869000000000002e-05 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.521802652000005, 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005336365, 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.027 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494350+0000", 2024-09-10T10:27:44.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494416+0000", 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6413000000000003e-05 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494424+0000", 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.576e-06 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494430+0000", 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6579999999999996e-06 2024-09-10T10:27:44.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494466+0000", 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5648999999999997e-05 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494478+0000", 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.234e-05 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499654+0000", 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051755209999999998 2024-09-10T10:27:44.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499685+0000", 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.167e-05 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.521407842000002, 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049664970000000003, 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.031 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494819+0000", 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4690000000000005e-05 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494825+0000", 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7080000000000002e-06 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494828+0000", 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1109999999999999e-06 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494869+0000", 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1146000000000002e-05 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494881+0000", 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2295e-05 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499687+0000", 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048061459999999999 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499710+0000", 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-05 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:27:44.035 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.236051639000003, 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054008370000000003, 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780104+0000", 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3039999999999998e-06 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0738e-05 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780141+0000", 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6317999999999999e-05 2024-09-10T10:27:44.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2199999999999992e-06 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780405+0000", 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025557599999999998 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780498+0000", 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2860000000000002e-05 2024-09-10T10:27:44.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780591+0000", 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3094e-05 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024814469999999999 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785441+0000", 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023678280000000002 2024-09-10T10:27:44.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785480+0000", 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9511999999999997e-05 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785501+0000", 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0939999999999999e-05 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.235976805, 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054780480000000001, 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780176+0000", 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6700000000000003e-07 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1289999999999998e-06 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780188+0000", 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.96e-06 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780194+0000", 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9989999999999997e-06 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780882+0000", 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068832400000000003 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780968+0000", 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5959999999999997e-05 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781098+0000", 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012994100000000001 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783078+0000", 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019803540000000001 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785553+0000", 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002474515 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785608+0000", 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5319999999999999e-05 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785653+0000", 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4879000000000002e-05 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.235945262000001, 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042025200000000004, 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780217+0000", 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:27:44.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780224+0000", 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2529999999999998e-06 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780230+0000", 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6110000000000001e-06 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780282+0000", 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2148000000000003e-05 2024-09-10T10:27:44.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780394+0000", 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111864 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780553+0000", 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015913799999999999 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783054+0000", 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025008809999999999 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784349+0000", 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012947480000000001 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784381+0000", 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2268999999999999e-05 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784409+0000", 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8007000000000001e-05 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.050 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.23590291, 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054782659999999999, 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780253+0000", 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4199999999999999e-06 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780260+0000", 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5100000000000001e-06 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780267+0000", 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.443e-06 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781353+0000", 2024-09-10T10:27:44.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010859769999999999 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781457+0000", 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104545 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781561+0000", 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010409200000000001 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783532+0000", 2024-09-10T10:27:44.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019702280000000001 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785685+0000", 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002153313 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785710+0000", 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5024999999999998e-05 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785727+0000", 2024-09-10T10:27:44.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7175000000000001e-05 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.235876818999998, 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005104407, 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780278+0000", 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.959e-06 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780285+0000", 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8040000000000004e-06 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780292+0000", 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4099999999999996e-06 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781345+0000", 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010533579999999999 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781456+0000", 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110624 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781570+0000", 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011417999999999999 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783625+0000", 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020553780000000001 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785303+0000", 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001677827 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785345+0000", 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2364999999999998e-05 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785379+0000", 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4054000000000003e-05 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.235852991000002, 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0060500800000000002, 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.061 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:44.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780306+0000", 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780313+0000", 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-06 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:27:44.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1739999999999999e-06 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782543+0000", 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022253419999999999 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010785699999999999 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.786349+0000", 2024-09-10T10:27:44.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036978829999999999 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.144150951, 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058824769999999997, 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872002+0000", 2024-09-10T10:27:44.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5149999999999999e-06 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872008+0000", 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.66e-06 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872022+0000", 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3932e-05 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872031+0000", 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5259999999999999e-06 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872111+0000", 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9412000000000005e-05 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872231+0000", 2024-09-10T10:27:44.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119843 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872340+0000", 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109442 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.875579+0000", 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032383899999999998 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877781+0000", 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022022299999999999 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877835+0000", 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4419000000000002e-05 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877883+0000", 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8108000000000001e-05 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.143062120000003, 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049968149999999999, 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7300000000000005e-07 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.245e-06 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3898e-05 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873116+0000", 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8050000000000007e-06 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873184+0000", 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7809000000000001e-05 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873257+0000", 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2602999999999996e-05 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873374+0000", 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011744900000000001 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.876282+0000", 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029079459999999998 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878003+0000", 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001720886 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878054+0000", 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0596999999999997e-05 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878086+0000", 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2804e-05 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.068669748000001, 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029964739999999998, 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.076 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947484+0000", 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.857e-06 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947490+0000", 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9789999999999998e-06 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947504+0000", 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3942e-05 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947512+0000", 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5620000000000004e-06 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947599+0000", 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6525000000000003e-05 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947714+0000", 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114709 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:44.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947823+0000", 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109477 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950336+0000", 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025131509999999999 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950386+0000", 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9974e-05 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950440+0000", 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4011000000000002e-05 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950478+0000", 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8287000000000001e-05 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.081 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.067229350000002, 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043004360000000004, 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948923+0000", 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8999999999999996e-07 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948928+0000", 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948940+0000", 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1759e-05 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948949+0000", 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5149999999999995e-06 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949028+0000", 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8182999999999993e-05 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949137+0000", 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109065 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949245+0000", 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108047 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950193+0000", 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094856200000000002 2024-09-10T10:27:44.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953173+0000", 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029801960000000001 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953200+0000", 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6707999999999999e-05 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953223+0000", 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2758000000000001e-05 2024-09-10T10:27:44.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.067173837999999, 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046333320000000004, 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948979+0000", 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-07 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6490000000000001e-06 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948990+0000", 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2560000000000002e-06 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948996+0000", 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9739999999999999e-06 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949496+0000", 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049934299999999997 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949588+0000", 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1762999999999995e-05 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949729+0000", 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014145299999999999 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950984+0000", 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012548419999999999 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953505+0000", 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025208129999999998 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953568+0000", 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3695000000000006e-05 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953611+0000", 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2913000000000001e-05 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.095122223000001, 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048620219999999997, 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.092 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:44.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4899999999999995e-07 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921034+0000", 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7050000000000001e-06 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921043+0000", 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1840000000000005e-06 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921049+0000", 2024-09-10T10:27:44.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4459999999999998e-06 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921575+0000", 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052602300000000003 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921646+0000", 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0475000000000005e-05 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921769+0000", 2024-09-10T10:27:44.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123347 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923077+0000", 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013078759999999999 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925794+0000", 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027172310000000001 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925853+0000", 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8739999999999997e-05 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925892+0000", 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8445999999999997e-05 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.097 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.095085188999999, 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032737339999999999, 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9919999999999999e-06 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921080+0000", 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0063e-05 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921086+0000", 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921142+0000", 2024-09-10T10:27:44.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6174999999999997e-05 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013480499999999999 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921405+0000", 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012819200000000001 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923001+0000", 2024-09-10T10:27:44.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015962249999999999 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924243+0000", 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012425820000000001 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924300+0000", 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6901999999999998e-05 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924340+0000", 2024-09-10T10:27:44.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9918e-05 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(2.4 epoch 26/18 rep_tid 19 entries 26'1 (0'0) error 2:218afd09:::203.00000003:head by mds.3.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.092946743999999, 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030589089999999998, 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923206+0000", 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7249999999999997e-06 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923229+0000", 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5905000000000001e-05 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923241+0000", 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1990999999999999e-05 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923317+0000", 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5622000000000002e-05 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926264+0000", 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029470170000000001 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:27:44.106 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.072588744000001, 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050642550000000001, 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943564+0000", 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.13e-07 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943576+0000", 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2187e-05 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943585+0000", 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9279999999999999e-06 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943592+0000", 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0110000000000003e-06 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943681+0000", 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9487999999999994e-05 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943767+0000", 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5866999999999996e-05 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943882+0000", 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011443299999999999 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.947042+0000", 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031600629999999998 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948540+0000", 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001498451 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948581+0000", 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0839e-05 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948627+0000", 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6375000000000002e-05 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:27:44.111 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.072520421, 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051386089999999997, 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943632+0000", 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1500000000000004e-07 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943643+0000", 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0954e-05 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943652+0000", 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4929999999999998e-06 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943657+0000", 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6509999999999998e-06 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944230+0000", 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057322999999999999 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944340+0000", 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010986 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948770+0000", 2024-09-10T10:27:44.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044297060000000003 2024-09-10T10:27:44.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.115 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.115 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.115 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:44.116 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:27:44.117 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:44.115+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:44.216 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:44.216+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:44.259 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:44.259 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:44.259 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:44.259 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:44.268 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:27:44.317 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:44.317+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:44.371 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:44.372 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:44.372 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:44.372 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:44.372 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:44.380 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:27:44.419 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:44.418+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:44.520 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:44.519+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:44.525 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:44.525 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:27:44.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:27:44.525 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:27:44.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.743284172000003, 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049304329999999997, 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:27:44.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780094+0000", 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780107+0000", 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3703000000000001e-05 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780116+0000", 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5210000000000002e-06 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6340000000000007e-06 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001735838 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782000+0000", 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013984100000000001 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782131+0000", 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000131025 2024-09-10T10:27:44.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783737+0000", 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016054179999999999 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784937+0000", 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001200638 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784987+0000", 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9591e-05 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785024+0000", 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6751000000000001e-05 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:44.531 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.743244717000003, 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052929559999999997, 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780136+0000", 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9950000000000001e-06 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780144+0000", 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.174e-06 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4949999999999999e-06 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781163+0000", 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001013438 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781279+0000", 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011579200000000001 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781388+0000", 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108834 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783243+0000", 2024-09-10T10:27:44.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00185491 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785355+0000", 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021117990000000001 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785386+0000", 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1612000000000002e-05 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785426+0000", 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9474999999999997e-05 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:8 2.10 e24/18 2:09401a81:::601.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.741053487000002, 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005352224, 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.537 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:44.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4400000000000001e-07 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782467+0000", 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142452 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782598+0000", 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013078300000000001 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782604+0000", 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.533e-06 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7073999999999997e-05 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782668+0000", 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6861999999999999e-05 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787632+0000", 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049636510000000003 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787676+0000", 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4425000000000002e-05 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:9 2.10 e24/18 2:09401a81:::601.00000000:head v 24'4, mlcod=0'0)", 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.740416472, 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047438960000000001, 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782962+0000", 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4499999999999997e-07 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782967+0000", 2024-09-10T10:27:44.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2000000000000002e-06 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782980+0000", 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3685e-05 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782987+0000", 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4690000000000002e-06 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783112+0000", 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012481300000000001 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783130+0000", 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7753000000000001e-05 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787678+0000", 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045487419999999997 2024-09-10T10:27:44.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787705+0000", 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6588999999999999e-05 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:10 2.10 e24/18 2:0bd6d154:::602.00000000:head v 24'5, mlcod=0'0)", 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.740309537999998, 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00466629, 2024-09-10T10:27:44.546 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1300000000000001e-07 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6970000000000001e-06 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783309+0000", 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000236209 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783319+0000", 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7180000000000008e-06 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783365+0000", 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6179e-05 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783383+0000", 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8382000000000001e-05 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787708+0000", 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043246059999999999 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787734+0000", 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6086000000000001e-05 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.550 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.650907343999997, 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054527359999999997, 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872472+0000", 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5409999999999999e-06 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872479+0000", 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9550000000000003e-06 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872494+0000", 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5452999999999999e-05 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872503+0000", 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1509999999999993e-06 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872571+0000", 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7747e-05 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872704+0000", 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013269900000000001 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872804+0000", 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9934999999999998e-05 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877617+0000", 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048138069999999998 2024-09-10T10:27:44.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877819+0000", 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020164799999999999 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877880+0000", 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1296000000000004e-05 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877923+0000", 2024-09-10T10:27:44.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2503999999999998e-05 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.649885777000001, 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0098302110000000002, 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873493+0000", 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873498+0000", 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1619999999999999e-06 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873511+0000", 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3319000000000001e-05 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873520+0000", 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3409999999999992e-06 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873586+0000", 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6115e-05 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873660+0000", 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3103999999999997e-05 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873775+0000", 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115617 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878285+0000", 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045096049999999999 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883229+0000", 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049443999999999998 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883283+0000", 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4262999999999999e-05 2024-09-10T10:27:44.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883322+0000", 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8417000000000002e-05 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.649396637000002, 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047062800000000002, 2024-09-10T10:27:44.561 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873982+0000", 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2900000000000003e-07 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873986+0000", 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3939999999999998e-06 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873995+0000", 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0750000000000004e-06 2024-09-10T10:27:44.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874002+0000", 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874061+0000", 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8770999999999997e-05 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874178+0000", 2024-09-10T10:27:44.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117108 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874290+0000", 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112503 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878269+0000", 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039784870000000002 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878593+0000", 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032385299999999998 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878644+0000", 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1790999999999999e-05 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878687+0000", 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2738999999999999e-05 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.566 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.648796357999998, 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089185449999999999, 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874582+0000", 2024-09-10T10:27:44.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1699999999999997e-07 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874592+0000", 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0604999999999999e-05 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874602+0000", 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874610+0000", 2024-09-10T10:27:44.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1919999999999999e-06 2024-09-10T10:27:44.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874696+0000", 2024-09-10T10:27:44.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6033999999999997e-05 2024-09-10T10:27:44.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874809+0000", 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113373 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874923+0000", 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114077 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878289+0000", 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033662380000000001 2024-09-10T10:27:44.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883420+0000", 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051305760000000004 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883465+0000", 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5252e-05 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883500+0000", 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4415000000000002e-05 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.574933037000001, 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073932399999999997, 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.573 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948446+0000", 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3400000000000001e-06 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948451+0000", 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6219999999999997e-06 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948464+0000", 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2955e-05 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948472+0000", 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0730000000000005e-06 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948534+0000", 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1727e-05 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948649+0000", 2024-09-10T10:27:44.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114991 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948757+0000", 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107935 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950179+0000", 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014219440000000001 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055817970000000003 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955794+0000", 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3012000000000002e-05 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955838+0000", 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3844000000000002e-05 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.574394217999998, 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069680200000000001, 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:44.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948984+0000", 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6100000000000001e-07 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948993+0000", 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9809999999999992e-06 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949002+0000", 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8789999999999992e-06 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949008+0000", 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2759999999999997e-06 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949057+0000", 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9159999999999997e-05 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949128+0000", 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1292999999999997e-05 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949261+0000", 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132564 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950220+0000", 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00095953300000000004 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955890+0000", 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056692790000000002 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955928+0000", 2024-09-10T10:27:44.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8278000000000002e-05 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955951+0000", 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3215999999999999e-05 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.602417708000001, 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050489669999999997, 2024-09-10T10:27:44.584 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5799999999999999e-07 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920964+0000", 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6720000000000002e-06 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920971+0000", 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5140000000000004e-06 2024-09-10T10:27:44.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920977+0000", 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0100000000000001e-06 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921033+0000", 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5899000000000001e-05 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921172+0000", 2024-09-10T10:27:44.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139011 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921288+0000", 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011605200000000001 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922851+0000", 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001562499 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925915+0000", 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003063727 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925969+0000", 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4083999999999999e-05 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926009+0000", 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9941000000000003e-05 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.589 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.601810851, 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045836510000000002, 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4499999999999996e-07 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921572+0000", 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3239999999999996e-06 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921581+0000", 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9190000000000005e-06 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921589+0000", 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0649999999999999e-06 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921635+0000", 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6863999999999997e-05 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921705+0000", 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9046000000000003e-05 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921818+0000", 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011313499999999999 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923033+0000", 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012153590000000001 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926055+0000", 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030223310000000001 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926104+0000", 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8421999999999998e-05 2024-09-10T10:27:44.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926150+0000", 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6440999999999998e-05 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.601371907000001, 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051621749999999997, 2024-09-10T10:27:44.595 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3600000000000003e-07 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922011+0000", 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4449999999999998e-06 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922019+0000", 2024-09-10T10:27:44.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0930000000000003e-06 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922025+0000", 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5060000000000001e-06 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922076+0000", 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0671000000000003e-05 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922175+0000", 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9611000000000006e-05 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922278+0000", 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102565 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925217+0000", 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029384670000000002 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927074+0000", 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018579499999999999 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927129+0000", 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4428000000000001e-05 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927168+0000", 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8803000000000001e-05 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.600 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.601336237000002, 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059957930000000001, 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922042+0000", 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922045+0000", 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4209999999999999e-06 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922052+0000", 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1430000000000001e-06 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922059+0000", 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2139999999999999e-06 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922495+0000", 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043517499999999999 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922563+0000", 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8084999999999996e-05 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922679+0000", 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011609599999999999 2024-09-10T10:27:44.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927208+0000", 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045292170000000003 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927952+0000", 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00074421700000000003 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928001+0000", 2024-09-10T10:27:44.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9004e-05 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928037+0000", 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.578e-05 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.601309478000001, 2024-09-10T10:27:44.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045268130000000002, 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922078+0000", 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922085+0000", 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3710000000000002e-06 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922091+0000", 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0340000000000002e-06 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922144+0000", 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3106999999999998e-05 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922248+0000", 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010373400000000001 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922399+0000", 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015127200000000001 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925201+0000", 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028019189999999999 2024-09-10T10:27:44.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926514+0000", 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001312279 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926561+0000", 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000002e-05 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926595+0000", 2024-09-10T10:27:44.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3408000000000003e-05 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.60067463, 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005322116, 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999996e-07 2024-09-10T10:27:44.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922707+0000", 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.766e-06 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922717+0000", 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7229999999999999e-06 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922724+0000", 2024-09-10T10:27:44.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8410000000000002e-06 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922792+0000", 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8557000000000001e-05 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922912+0000", 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119354 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928025+0000", 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051131919999999999 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.600198476999999, 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049271260000000004, 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.616 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923180+0000", 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923183+0000", 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4000000000000001e-06 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923196+0000", 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3127e-05 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923203+0000", 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.939e-06 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923270+0000", 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7087000000000003e-05 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923359+0000", 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9054999999999994e-05 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928106+0000", 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047469239999999996 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.599737835999999, 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045860270000000003, 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:44.620 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2500000000000005e-07 2024-09-10T10:27:44.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923644+0000", 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8240000000000001e-06 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923659+0000", 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5358000000000001e-05 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923666+0000", 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9659999999999999e-06 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923725+0000", 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8674000000000001e-05 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923850+0000", 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000124797 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928226+0000", 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043757830000000003 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.582215400999999, 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005230859, 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:44.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:44.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941164+0000", 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.863e-06 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941172+0000", 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2360000000000004e-06 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941190+0000", 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7812000000000002e-05 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941200+0000", 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7599999999999997e-06 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941350+0000", 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015070400000000001 2024-09-10T10:27:44.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941501+0000", 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015044200000000001 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000167076 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943487+0000", 2024-09-10T10:27:44.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018190260000000001 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946297+0000", 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028102840000000001 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946349+0000", 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1649e-05 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946393+0000", 2024-09-10T10:27:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4007000000000001e-05 2024-09-10T10:27:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:44.630 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:44.630 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:44.631 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:27:44.632 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:44.631+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:44.733 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:44.732+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:44.774 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:44.774 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:44.774 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:44.783 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:27:44.833 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:44.832+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:44.885 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:44.886 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:44.894 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:27:44.934 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:44.934+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:45.000 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:45.000 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:27:45.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:27:45.000 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.218089974999998, 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078690670000000004, 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:27:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2200000000000004e-07 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780048+0000", 2024-09-10T10:27:45.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2427999999999999e-05 2024-09-10T10:27:45.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780057+0000", 2024-09-10T10:27:45.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1090000000000004e-06 2024-09-10T10:27:45.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780063+0000", 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9780000000000002e-06 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011213480000000001 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781281+0000", 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6728e-05 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781382+0000", 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101455 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784820+0000", 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034376200000000002 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787779+0000", 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002959373 2024-09-10T10:27:45.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787852+0000", 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3135999999999999e-05 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787894+0000", 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1369999999999999e-05 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.218042433999997, 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079289789999999992, 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780073+0000", 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780075+0000", 2024-09-10T10:27:45.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5129999999999999e-06 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780082+0000", 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9410000000000001e-06 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780087+0000", 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1880000000000001e-06 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781620+0000", 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001532346 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781698+0000", 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8232999999999994e-05 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781816+0000", 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011825800000000001 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784824+0000", 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030076349999999998 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787937+0000", 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031134829999999998 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787969+0000", 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1745999999999997e-05 2024-09-10T10:27:45.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788001+0000", 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2132000000000003e-05 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.218012219000002, 2024-09-10T10:27:45.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079888619999999994, 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780103+0000", 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780106+0000", 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1379999999999999e-06 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780115+0000", 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4299999999999995e-06 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780122+0000", 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3380000000000001e-06 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782041+0000", 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019193700000000001 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782149+0000", 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107493 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782246+0000", 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999996e-05 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785205+0000", 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002958535 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788041+0000", 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028359650000000002 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788069+0000", 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8123000000000002e-05 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788091+0000", 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2506000000000001e-05 2024-09-10T10:27:45.017 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.217983678000003, 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080531070000000003, 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780134+0000", 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0570000000000001e-06 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780143+0000", 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-06 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780156+0000", 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3179e-05 2024-09-10T10:27:45.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023278280000000001 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782555+0000", 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0494000000000006e-05 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782675+0000", 2024-09-10T10:27:45.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120463 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785209+0000", 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002533455 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788127+0000", 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029181260000000001 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788155+0000", 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8495000000000001e-05 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788184+0000", 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8736999999999999e-05 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:27:45.023 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.217951161000002, 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008097524, 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:27:45.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780164+0000", 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.13e-07 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9780000000000001e-06 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780174+0000", 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7510000000000002e-06 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780179+0000", 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0869999999999997e-06 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782890+0000", 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027113509999999999 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782985+0000", 2024-09-10T10:27:45.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5152e-05 2024-09-10T10:27:45.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783090+0000", 2024-09-10T10:27:45.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104947 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785222+0000", 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021320269999999999 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788216+0000", 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029940629999999999 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788242+0000", 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5259999999999999e-05 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788261+0000", 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9395e-05 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.029 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.217928790000002, 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081619529999999996, 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780189+0000", 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.926e-06 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780197+0000", 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2039999999999997e-06 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780202+0000", 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783326+0000", 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003123855 2024-09-10T10:27:45.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783402+0000", 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6548e-05 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783558+0000", 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015539199999999999 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785225+0000", 2024-09-10T10:27:45.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001666989 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788290+0000", 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030647890000000001 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788316+0000", 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6662e-05 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788348+0000", 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1417000000000001e-05 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:3 2.10 2:0f65c9b8:::60a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.126335277999999, 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066829990000000002, 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871781+0000", 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2129999999999998e-06 2024-09-10T10:27:45.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871790+0000", 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3380000000000005e-06 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871805+0000", 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5353999999999999e-05 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871817+0000", 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1399999999999999e-05 2024-09-10T10:27:45.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871932+0000", 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114982 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872069+0000", 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013713199999999999 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872208+0000", 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013976900000000001 2024-09-10T10:27:45.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874946+0000", 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027378789999999999 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878355+0000", 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034089139999999999 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878413+0000", 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7540000000000001e-05 2024-09-10T10:27:45.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878462+0000", 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9478000000000003e-05 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.050892145999999, 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067332750000000004, 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.040 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947223+0000", 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7299999999999995e-07 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947227+0000", 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.895e-06 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947235+0000", 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947246+0000", 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0738e-05 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948505+0000", 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001259402 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948578+0000", 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2788999999999997e-05 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:45.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948718+0000", 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013953699999999999 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953391+0000", 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046736670000000003 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953881+0000", 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048956900000000003 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953923+0000", 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2153000000000001e-05 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953956+0000", 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2526e-05 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.050833357999998, 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066654160000000004, 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947282+0000", 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947285+0000", 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7689999999999998e-06 2024-09-10T10:27:45.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947292+0000", 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6200000000000001e-06 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947304+0000", 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1702000000000001e-05 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947363+0000", 2024-09-10T10:27:45.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9426000000000003e-05 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947485+0000", 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012227599999999999 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947588+0000", 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102641 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950690+0000", 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003102264 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953856+0000", 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031656200000000001 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953907+0000", 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1430000000000001e-05 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953947+0000", 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9164e-05 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.050796184999999, 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071148679999999999, 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.051 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947319+0000", 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947322+0000", 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.523e-06 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947331+0000", 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3240000000000003e-06 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947336+0000", 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7189999999999998e-06 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947994+0000", 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065770599999999998 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948080+0000", 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5649000000000006e-05 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111783 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953965+0000", 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057732089999999996 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954343+0000", 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037846199999999998 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954395+0000", 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.126e-05 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954433+0000", 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8693999999999999e-05 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.056 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.050665988999999, 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067554449999999997, 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:27:45.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6899999999999996e-07 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947454+0000", 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947463+0000", 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0310000000000004e-06 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947477+0000", 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4328000000000001e-05 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3869999999999998e-05 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947631+0000", 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9874e-05 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947750+0000", 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118483 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953644+0000", 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058946889999999998 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954111+0000", 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00046712099999999998 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954162+0000", 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000003e-05 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954204+0000", 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1718000000000003e-05 2024-09-10T10:27:45.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.050278950999999, 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064551169999999998, 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3e-07 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947841+0000", 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3460000000000004e-06 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947856+0000", 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947864+0000", 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4939999999999997e-06 2024-09-10T10:27:45.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947997+0000", 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013305399999999999 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948076+0000", 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9146000000000004e-05 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:27:45.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115586 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953707+0000", 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055153470000000003 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954215+0000", 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050850400000000001 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954258+0000", 2024-09-10T10:27:45.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2932000000000002e-05 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954291+0000", 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2322e-05 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.050239631, 2024-09-10T10:27:45.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088919089999999999, 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:27:45.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7199999999999999e-07 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947879+0000", 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8380000000000004e-06 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947887+0000", 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5410000000000003e-06 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947893+0000", 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6930000000000004e-06 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948405+0000", 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051236000000000001 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948502+0000", 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6916999999999994e-05 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948647+0000", 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000145265 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954479+0000", 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058316970000000003 2024-09-10T10:27:45.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956680+0000", 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002201494 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956728+0000", 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8161999999999997e-05 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956767+0000", 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.837e-05 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.04967697, 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083279530000000008, 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.073 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:27:45.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8999999999999995e-07 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948443+0000", 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2189999999999998e-06 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948450+0000", 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8879999999999997e-06 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948457+0000", 2024-09-10T10:27:45.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0750000000000003e-06 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948501+0000", 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948594+0000", 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3009000000000003e-05 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948696+0000", 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101461 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953756+0000", 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050607129999999997 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956690+0000", 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029334109999999999 2024-09-10T10:27:45.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956737+0000", 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7080999999999999e-05 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956766+0000", 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8778000000000001e-05 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.049137296000001, 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0077788589999999999, 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948986+0000", 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948998+0000", 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.19e-05 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949007+0000", 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.081e-06 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949062+0000", 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754999999999997e-05 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949172+0000", 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110128 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949282+0000", 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109289 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955941+0000", 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066594990000000001 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956654+0000", 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071279399999999999 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956706+0000", 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2027000000000002e-05 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956756+0000", 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0012999999999997e-05 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.048599201999998, 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073200349999999999, 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949516+0000", 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999995e-07 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949522+0000", 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3980000000000004e-06 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949534+0000", 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1562e-05 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949541+0000", 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1380000000000001e-06 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950700+0000", 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011593459999999999 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950786+0000", 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5556999999999994e-05 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950898+0000", 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111743 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954326+0000", 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034280510000000001 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956774+0000", 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024477430000000001 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956815+0000", 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1752999999999998e-05 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956835+0000", 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0032999999999999e-05 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.048556901000001, 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073303700000000001, 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.089 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949571+0000", 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2298e-05 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949579+0000", 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5909999999999996e-06 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949585+0000", 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7370000000000003e-06 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953802+0000", 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042170929999999999 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953915+0000", 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011256699999999999 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956888+0000", 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029734639999999999 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:24 2.3 e26/18 2:c216c0d1:::mds2_inotable:head v 26'3, mlcod=26'1)", 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.047738899999999, 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0063254399999999999, 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.093 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950380+0000", 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2590000000000002e-06 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950387+0000", 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950392+0000", 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950438+0000", 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5592e-05 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950457+0000", 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8569000000000001e-05 2024-09-10T10:27:45.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956657+0000", 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062003409999999998 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956701+0000", 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4140000000000001e-05 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.097 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:29 2.10 2:0e4da237:::203.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 25.076078097, 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064228799999999997, 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922037+0000", 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922040+0000", 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.694e-06 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922054+0000", 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906e-05 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922065+0000", 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922125+0000", 2024-09-10T10:27:45.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.982e-05 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922233+0000", 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108175 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928459+0000", 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062264570000000003 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.056814466999999, 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081214100000000008, 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941301+0000", 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.062e-06 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941306+0000", 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0919999999999997e-06 2024-09-10T10:27:45.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941316+0000", 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5640000000000002e-06 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941323+0000", 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941405+0000", 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1650000000000006e-05 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941549+0000", 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014447599999999999 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012929 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943549+0000", 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018704049999999999 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949343+0000", 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057943300000000003 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949378+0000", 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4474999999999998e-05 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949421+0000", 2024-09-10T10:27:45.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3955999999999997e-05 2024-09-10T10:27:45.107 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.107 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.107 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.107 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.107 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.107 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:45.107 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-10T10:27:45.109 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:27:45.109 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:45.108+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:45.209 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:45.209+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:45.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.249+0000 7f6171e53640 1 Processor -- start 2024-09-10T10:27:45.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.249+0000 7f6171e53640 1 -- start start 2024-09-10T10:27:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.250+0000 7f6171e53640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f616c152a80 0x7f616c0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.250+0000 7f6171e53640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f616c0cd330 0x7f616c0c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.250+0000 7f6171e53640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f616c0cc340 con 0x7f616c0c5be0 2024-09-10T10:27:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.250+0000 7f6171e53640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f616c0c7560 con 0x7f616c152a80 2024-09-10T10:27:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.250+0000 7f6171e53640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f616c0c7e00 con 0x7f616c0cd330 2024-09-10T10:27:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.250+0000 7f616b7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f616c152a80 0x7f616c0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.250+0000 7f616b7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f616c152a80 0x7f616c0cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45072/0 (socket says 172.21.15.120:45072) 2024-09-10T10:27:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.250+0000 7f616b7fe640 1 -- 172.21.15.120:0/2108349738 learned_addr learned my addr 172.21.15.120:0/2108349738 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:45.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.250+0000 7f6163fff640 1 --2- 172.21.15.120:0/2108349738 >> v2:172.21.15.134:3300/0 conn(0x7f616c0cd330 0x7f616c0c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.251+0000 7f616b7fe640 1 -- 172.21.15.120:0/2108349738 >> v2:172.21.15.134:3300/0 conn(0x7f616c0cd330 msgr2=0x7f616c0c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:45.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.251+0000 7f616b7fe640 1 --2- 172.21.15.120:0/2108349738 >> v2:172.21.15.134:3300/0 conn(0x7f616c0cd330 0x7f616c0c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.251+0000 7f616b7fe640 1 -- 172.21.15.120:0/2108349738 >> v1:172.21.15.134:6789/0 conn(0x7f616c0c5be0 legacy=0x7f616c158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:45.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.251+0000 7f616b7fe640 1 -- 172.21.15.120:0/2108349738 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c0cc620 con 0x7f616c152a80 2024-09-10T10:27:45.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.251+0000 7f616b7fe640 1 --2- 172.21.15.120:0/2108349738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 0x7f616c0cc9a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f616c0c1880 tx=0x7f615c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a75e80bdc0cf52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:45.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.251+0000 7f616affd640 1 -- 172.21.15.120:0/2108349738 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c037070 con 0x7f616c152a80 2024-09-10T10:27:45.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.251+0000 7f616affd640 1 -- 172.21.15.120:0/2108349738 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615c02ae50 con 0x7f616c152a80 2024-09-10T10:27:45.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.251+0000 7f616affd640 1 -- 172.21.15.120:0/2108349738 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c032860 con 0x7f616c152a80 2024-09-10T10:27:45.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.252+0000 7f6171e53640 1 -- 172.21.15.120:0/2108349738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 msgr2=0x7f616c0cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.252+0000 7f6171e53640 1 --2- 172.21.15.120:0/2108349738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 0x7f616c0cc9a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f616c0c1880 tx=0x7f615c02a300 comp rx=0 tx=0).stop 2024-09-10T10:27:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.252+0000 7f6171e53640 1 -- 172.21.15.120:0/2108349738 shutdown_connections 2024-09-10T10:27:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.252+0000 7f6171e53640 1 --2- 172.21.15.120:0/2108349738 >> v2:172.21.15.134:3300/0 conn(0x7f616c0cd330 0x7f616c0c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.252+0000 7f6171e53640 1 --2- 172.21.15.120:0/2108349738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 0x7f616c0cc9a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.252+0000 7f6171e53640 1 -- 172.21.15.120:0/2108349738 >> 172.21.15.120:0/2108349738 conn(0x7f616c06bc10 msgr2=0x7f616c06fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.252+0000 7f6171e53640 1 -- 172.21.15.120:0/2108349738 shutdown_connections 2024-09-10T10:27:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.252+0000 7f6171e53640 1 -- 172.21.15.120:0/2108349738 wait complete. 2024-09-10T10:27:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.253+0000 7f6171e53640 1 Processor -- start 2024-09-10T10:27:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.253+0000 7f6171e53640 1 -- start start 2024-09-10T10:27:45.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f6171e53640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f616c0c5be0 0x7f616c0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f6171e53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0cd330 0x7f616c1f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f6171e53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 0x7f616c0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f6171e53640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f616c0cc000 con 0x7f616c152a80 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f6171e53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f616c0c7a90 con 0x7f616c0cd330 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f6171e53640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f616c152e60 con 0x7f616c0c5be0 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f616bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 0x7f616c0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f616bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 0x7f616c0ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45080/0 (socket says 172.21.15.120:45080) 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f616bfff640 1 -- 172.21.15.120:0/1700950158 learned_addr learned my addr 172.21.15.120:0/1700950158 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f6163fff640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0cd330 0x7f616c1f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f616b7fe640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f616c0c5be0 0x7f616c0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f616bfff640 1 -- 172.21.15.120:0/1700950158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f616c0c5be0 msgr2=0x7f616c0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f616bfff640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f616c0c5be0 0x7f616c0c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f616bfff640 1 -- 172.21.15.120:0/1700950158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0cd330 msgr2=0x7f616c1f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f616bfff640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0cd330 0x7f616c1f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.254+0000 7f616bfff640 1 -- 172.21.15.120:0/1700950158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c06e5f0 con 0x7f616c152a80 2024-09-10T10:27:45.258 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:45.258 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:45.258 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:45.258 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:45.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.255+0000 7f616bfff640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 0x7f616c0ca890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f616400c7d0 tx=0x7f616400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:45.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.255+0000 7f6168ff9640 1 -- 172.21.15.120:0/1700950158 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164003b70 con 0x7f616c152a80 2024-09-10T10:27:45.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.255+0000 7f6168ff9640 1 -- 172.21.15.120:0/1700950158 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6164003d30 con 0x7f616c152a80 2024-09-10T10:27:45.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.255+0000 7f6168ff9640 1 -- 172.21.15.120:0/1700950158 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616400aa90 con 0x7f616c152a80 2024-09-10T10:27:45.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.255+0000 7f6171e53640 1 -- 172.21.15.120:0/1700950158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616c15cce0 con 0x7f616c152a80 2024-09-10T10:27:45.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.255+0000 7f6171e53640 1 -- 172.21.15.120:0/1700950158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616c1f45d0 con 0x7f616c152a80 2024-09-10T10:27:45.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.256+0000 7f6171e53640 1 -- 172.21.15.120:0/1700950158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616c0c56a0 con 0x7f616c152a80 2024-09-10T10:27:45.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.256+0000 7f6168ff9640 1 -- 172.21.15.120:0/1700950158 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f616400ac30 con 0x7f616c152a80 2024-09-10T10:27:45.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.257+0000 7f6168ff9640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f614c06be30 0x7f614c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.257+0000 7f6168ff9640 1 -- 172.21.15.120:0/1700950158 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6706+0+0 (secure 0 0 0) 0x7f616408b080 con 0x7f616c152a80 2024-09-10T10:27:45.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.257+0000 7f616b7fe640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f614c06be30 0x7f614c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.257+0000 7f616b7fe640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f614c06be30 0x7f614c06e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f615c034000 tx=0x7f615c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:45.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.261+0000 7f6168ff9640 1 -- 172.21.15.120:0/1700950158 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6164059650 con 0x7f616c152a80 2024-09-10T10:27:45.266 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:27:45.310 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:45.309+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:45.369 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:45.369 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:45.369 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:45.369 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:45.369 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:45.378 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:27:45.419 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:45.418+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:45.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.442+0000 7f6171e53640 1 -- 172.21.15.120:0/1700950158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f616c0ce840 con 0x7f616c152a80 2024-09-10T10:27:45.512 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:45.512+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:45.523 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:45.523 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:27:45.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:27:45.523 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:27:45.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.523 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:45.523 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:27:45.523 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.740918811999997, 2024-09-10T10:27:45.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054521709999999996, 2024-09-10T10:27:45.523 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780407+0000", 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.517e-06 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780414+0000", 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3910000000000003e-06 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780429+0000", 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5379999999999998e-05 2024-09-10T10:27:45.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780438+0000", 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9230000000000008e-06 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780885+0000", 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044711300000000001 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780979+0000", 2024-09-10T10:27:45.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4312000000000001e-05 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785856+0000", 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048765350000000004 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.740844084999999, 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028177609999999998, 2024-09-10T10:27:45.527 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780479+0000", 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9800000000000003e-07 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780488+0000", 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3349999999999996e-06 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780512+0000", 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3156999999999999e-05 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780518+0000", 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.764e-06 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780576+0000", 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7813999999999999e-05 2024-09-10T10:27:45.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780700+0000", 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123733 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783296+0000", 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025963599999999998 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:19 2.14 e24/18 2:2f55791f:::606.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.739724924999997, 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003505083, 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:27:45.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9400000000000005e-07 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781606+0000", 2024-09-10T10:27:45.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6559999999999992e-06 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781624+0000", 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7575e-05 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781630+0000", 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8630000000000002e-06 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781683+0000", 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475000000000002e-05 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781700+0000", 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7393000000000001e-05 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785058+0000", 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033575660000000002 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785103+0000", 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:1 2.14 e24/18 2:292cf221:::200.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.739556419000003, 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033666960000000002, 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.536 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781767+0000", 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781772+0000", 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5539999999999997e-06 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781781+0000", 2024-09-10T10:27:45.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6149999999999997e-06 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781787+0000", 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1589999999999998e-06 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2859000000000002e-05 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781876+0000", 2024-09-10T10:27:45.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5916000000000001e-05 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785108+0000", 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003232146 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785133+0000", 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5015999999999999e-05 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:23 2.e e24/18 2:717a0223:::608.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.738941883000003, 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027543580000000002, 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1699999999999998e-07 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782387+0000", 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3229999999999997e-06 2024-09-10T10:27:45.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782399+0000", 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2466000000000001e-05 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782406+0000", 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9460000000000001e-06 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782468+0000", 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2347999999999993e-05 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6016e-05 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785096+0000", 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026118629999999999 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785135+0000", 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8779000000000003e-05 2024-09-10T10:27:45.544 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:24 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.643760176999997, 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062416909999999997, 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877564+0000", 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877576+0000", 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2411e-05 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877592+0000", 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5812e-05 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877600+0000", 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3590000000000002e-06 2024-09-10T10:27:45.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877661+0000", 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1749999999999997e-05 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877785+0000", 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123807 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:45.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877936+0000", 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015084300000000001 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.879317+0000", 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001381415 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883686+0000", 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004368701 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883755+0000", 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9140999999999994e-05 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883804+0000", 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8772e-05 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.550 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.642934212999997, 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056063609999999998, 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878389+0000", 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699999999999999e-07 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878394+0000", 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6120000000000001e-06 2024-09-10T10:27:45.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878407+0000", 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3704e-05 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878415+0000", 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7100000000000007e-06 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878475+0000", 2024-09-10T10:27:45.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878603+0000", 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127525 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883995+0000", 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053920460000000002 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.642883847, 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057312819999999999, 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5000000000000003e-07 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878442+0000", 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.632e-06 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878450+0000", 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0509999999999997e-06 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878456+0000", 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7200000000000003e-06 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878847+0000", 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039182099999999999 2024-09-10T10:27:45.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878949+0000", 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101458 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.884170+0000", 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052210499999999996 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.574218528999999, 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033173069999999998, 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947105+0000", 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4670000000000001e-06 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947111+0000", 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6230000000000002e-06 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947123+0000", 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1883e-05 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947132+0000", 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8740000000000001e-06 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947220+0000", 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7800000000000006e-05 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947375+0000", 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015566399999999999 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947501+0000", 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125609 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948393+0000", 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00089165399999999997 2024-09-10T10:27:45.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950323+0000", 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019308769999999999 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950366+0000", 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2688999999999997e-05 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950421+0000", 2024-09-10T10:27:45.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5167000000000002e-05 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.574162977, 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0070341989999999997, 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:27:45.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947163+0000", 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3440000000000001e-06 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947174+0000", 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0248000000000001e-05 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947180+0000", 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0639999999999999e-06 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947958+0000", 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077789700000000005 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948031+0000", 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3503000000000004e-05 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:45.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948151+0000", 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119976 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953625+0000", 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054739360000000004 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954124+0000", 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049858900000000002 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954157+0000", 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3652999999999998e-05 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954194+0000", 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6455000000000002e-05 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.569 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.574071134, 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032449549999999999, 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947252+0000", 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000003e-07 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947256+0000", 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3760000000000001e-06 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947267+0000", 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0259e-05 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947273+0000", 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2190000000000001e-06 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947332+0000", 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9086000000000003e-05 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947452+0000", 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120146 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950496+0000", 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003044171 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:13 2.1 e26/18 2:81735765:::619.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.573661003000002, 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006029375, 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947662+0000", 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7199999999999997e-07 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6990000000000003e-06 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947681+0000", 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0682999999999999e-05 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947687+0000", 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7479999999999996e-06 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947742+0000", 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4487000000000002e-05 2024-09-10T10:27:45.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947761+0000", 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9497999999999999e-05 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953653+0000", 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058916530000000002 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953691+0000", 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7734999999999997e-05 2024-09-10T10:27:45.577 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:14 2.1 e26/18 2:81735765:::619.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.572340665999999, 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052525849999999997, 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:45.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5700000000000002e-07 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948988+0000", 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7490000000000004e-06 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949003+0000", 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4214e-05 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949010+0000", 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3499999999999999e-06 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949059+0000", 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8758000000000003e-05 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949078+0000", 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9806e-05 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954198+0000", 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051191509999999997 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954234+0000", 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.68e-05 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:1 2.1 e26/18 2:85bbe569:::202.00000000:head v 26'4, mlcod=0'0)", 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.572112302000001, 2024-09-10T10:27:45.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005049258, 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949211+0000", 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4999999999999998e-07 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949228+0000", 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7808999999999998e-05 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949240+0000", 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1226000000000001e-05 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949246+0000", 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0689999999999998e-06 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949292+0000", 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6211999999999997e-05 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949311+0000", 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8873e-05 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954236+0000", 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049253980000000001 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954259+0000", 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3221e-05 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.586 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:26 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.579250219999999, 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039596270000000003, 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942075+0000", 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2680000000000001e-06 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942086+0000", 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1361e-05 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942098+0000", 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2221e-05 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942106+0000", 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0960000000000001e-06 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942209+0000", 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102944 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942346+0000", 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136976 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942516+0000", 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017015699999999999 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945195+0000", 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026790920000000001 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:45.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945919+0000", 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00072374500000000003 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945983+0000", 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3418999999999997e-05 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946032+0000", 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9348000000000003e-05 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.591 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:2 2.1 e28/18 2:82971f97:::404.00000000:head v 28'5, mlcod=26'3)", 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.577591064, 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054707489999999996, 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:27:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943732+0000", 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1999999999999998e-07 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943740+0000", 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1910000000000003e-06 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943755+0000", 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5014000000000001e-05 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943763+0000", 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9519999999999994e-06 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5569999999999997e-05 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943862+0000", 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4150000000000003e-05 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949157+0000", 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052949340000000003 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949202+0000", 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5018000000000002e-05 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:6 2.e e28/18 2:75ee5fd8:::629.00000000:head v 28'5, mlcod=24'2)", 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.577529236, 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027501769999999999, 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.596 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943794+0000", 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943799+0000", 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5620000000000003e-06 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943807+0000", 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.486e-06 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943813+0000", 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6419999999999999e-05 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943888+0000", 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9309000000000001e-05 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946498+0000", 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026094669999999999 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946543+0000", 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.600 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:17 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'12, mlcod=26'10)", 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.577485475, 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027214219999999998, 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:27:45.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:27:45.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-10T10:27:45.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943849+0000", 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2023e-05 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943857+0000", 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6009999999999997e-06 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943863+0000", 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2899999999999999e-06 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943927+0000", 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3496000000000003e-05 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943949+0000", 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1801e-05 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946490+0000", 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025409579999999998 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946558+0000", 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8830000000000003e-05 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:18 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'13, mlcod=26'10)", 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.577439273, 2024-09-10T10:27:45.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027193059999999999, 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943884+0000", 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943889+0000", 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.95e-06 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943896+0000", 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3819999999999998e-06 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943901+0000", 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1719999999999998e-06 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944666+0000", 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00076445300000000001 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944686+0000", 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-05 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018748879999999999 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946603+0000", 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1947999999999999e-05 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:25 2.12 e28/18 2:4dad84a4:::mds4_sessionmap:head v 28'5, mlcod=26'3)", 2024-09-10T10:27:45.610 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.574525309999999, 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002978994, 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:27:45.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946798+0000", 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0100000000000001e-06 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946807+0000", 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9700000000000005e-06 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946820+0000", 2024-09-10T10:27:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2458e-05 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946856+0000", 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6427999999999997e-05 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946938+0000", 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2106000000000003e-05 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946958+0000", 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0162000000000001e-05 2024-09-10T10:27:45.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949723+0000", 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027645899999999999 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949776+0000", 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3270000000000003e-05 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:45.614 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:45.616 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:27:45.616 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:45.615+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:45.717 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:45.716+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:45.732 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:45.732 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:45.732 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:27:45.732 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:45.741 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:27:45.818 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:45.817+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:45.846 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:27:45.847 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:45.847 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:45.848 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:45.848 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:45.848 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:45.856 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:27:45.918 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:45.917+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:45.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.945+0000 7ff1f66f8640 1 Processor -- start 2024-09-10T10:27:45.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.945+0000 7ff1f66f8640 1 -- start start 2024-09-10T10:27:45.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1f66f8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1f0070d40 0x7ff1f015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1f66f8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff1f0071120 0x7ff1f015e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1f66f8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f00cd3e0 con 0x7ff1f00c8f80 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1f66f8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f00c8af0 con 0x7ff1f0071120 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1f66f8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f00cd6c0 con 0x7ff1f0070d40 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1f4c95640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff1f0071120 0x7ff1f015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1ef7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff1f00c8f80 0x7ff1f015e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:50620/0 (socket says 172.21.15.120:50620) 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1ef7fe640 1 -- 172.21.15.120:0/2844639715 learned_addr learned my addr 172.21.15.120:0/2844639715 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1effff640 1 --2- 172.21.15.120:0/2844639715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1f0070d40 0x7ff1f015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1f4c95640 1 -- 172.21.15.120:0/2844639715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1f0070d40 msgr2=0x7ff1f015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1f4c95640 1 --2- 172.21.15.120:0/2844639715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1f0070d40 0x7ff1f015ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1f4c95640 1 -- 172.21.15.120:0/2844639715 >> v1:172.21.15.120:6789/0 conn(0x7ff1f00c8f80 legacy=0x7ff1f015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1eeffd640 1 -- 172.21.15.120:0/2844639715 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 909416651 0 0) 0x7ff1f00cd3e0 con 0x7ff1f00c8f80 2024-09-10T10:27:45.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.946+0000 7ff1f4c95640 1 -- 172.21.15.120:0/2844639715 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00cd9a0 con 0x7ff1f0071120 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1f4c95640 1 --2- 172.21.15.120:0/2844639715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0071120 0x7ff1f015e790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009d90 tx=0x7ff1e802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d458b29ac863d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1eeffd640 1 -- 172.21.15.120:0/2844639715 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8007740 con 0x7ff1f0071120 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1eeffd640 1 -- 172.21.15.120:0/2844639715 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e80078e0 con 0x7ff1f0071120 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1eeffd640 1 -- 172.21.15.120:0/2844639715 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8007cb0 con 0x7ff1f0071120 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1f66f8640 1 -- 172.21.15.120:0/2844639715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0071120 msgr2=0x7ff1f015e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/2844639715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0071120 0x7ff1f015e790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009d90 tx=0x7ff1e802a6f0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1f66f8640 1 -- 172.21.15.120:0/2844639715 shutdown_connections 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/2844639715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0071120 0x7ff1f015e790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/2844639715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1f0070d40 0x7ff1f015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1f66f8640 1 -- 172.21.15.120:0/2844639715 >> 172.21.15.120:0/2844639715 conn(0x7ff1f0072a60 msgr2=0x7ff1f0072e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:45.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1f66f8640 1 -- 172.21.15.120:0/2844639715 shutdown_connections 2024-09-10T10:27:45.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.947+0000 7ff1f66f8640 1 -- 172.21.15.120:0/2844639715 wait complete. 2024-09-10T10:27:45.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.949+0000 7ff1f66f8640 1 Processor -- start 2024-09-10T10:27:45.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.949+0000 7ff1f66f8640 1 -- start start 2024-09-10T10:27:45.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.949+0000 7ff1f66f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0070d40 0x7ff1f01f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.949+0000 7ff1f66f8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1f0071120 0x7ff1f01f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.949+0000 7ff1f66f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1f00c8f80 0x7ff1f01f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.949+0000 7ff1f66f8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f015b050 con 0x7ff1f0070d40 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.949+0000 7ff1f66f8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f00c8780 con 0x7ff1f00c8f80 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.949+0000 7ff1f66f8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f015ecd0 con 0x7ff1f0071120 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.949+0000 7ff1effff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0070d40 0x7ff1f01f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1effff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0070d40 0x7ff1f01f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45084/0 (socket says 172.21.15.120:45084) 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1effff640 1 -- 172.21.15.120:0/1772338171 learned_addr learned my addr 172.21.15.120:0/1772338171 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1f4c95640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1f00c8f80 0x7ff1f01f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1ef7fe640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1f0071120 0x7ff1f01f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1effff640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1f0071120 msgr2=0x7ff1f01f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1effff640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1f0071120 0x7ff1f01f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1effff640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1f00c8f80 msgr2=0x7ff1f01f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1effff640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1f00c8f80 0x7ff1f01f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:45.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1effff640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00c3e30 con 0x7ff1f0070d40 2024-09-10T10:27:45.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1effff640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0070d40 0x7ff1f01f29c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000e9b0 tx=0x7ff1e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:45.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0017070 con 0x7ff1f0070d40 2024-09-10T10:27:45.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e000c9a0 con 0x7ff1f0070d40 2024-09-10T10:27:45.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000cd90 con 0x7ff1f0070d40 2024-09-10T10:27:45.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f00c4110 con 0x7ff1f0070d40 2024-09-10T10:27:45.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.950+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f01f21c0 con 0x7ff1f0070d40 2024-09-10T10:27:45.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.951+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ff1f01f1cf0 con 0x7ff1f0070d40 2024-09-10T10:27:45.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.952+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1e001a070 con 0x7ff1f0070d40 2024-09-10T10:27:45.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.952+0000 7ff1ecff9640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1dc06be30 0x7ff1dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.952+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6706+0+0 (secure 0 0 0) 0x7ff1e008b560 con 0x7ff1f0070d40 2024-09-10T10:27:45.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.952+0000 7ff1ef7fe640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1dc06be30 0x7ff1dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.952+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v32) ==== 54+0+16 (secure 0 0 0) 0x7ff1e008b9f0 con 0x7ff1f0070d40 2024-09-10T10:27:45.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.953+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ff1f01f2400 0x7ff1f015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.953+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1f0154930 con 0x7ff1f01f2400 2024-09-10T10:27:45.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.953+0000 7ff1ef7fe640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1dc06be30 0x7ff1dc06e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0154b10 tx=0x7ff1e4007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:45.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.953+0000 7ff1f4c95640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ff1f01f2400 0x7ff1f015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.953+0000 7ff1f4c95640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ff1f01f2400 0x7ff1f015e050 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:45.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.954+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1e8007640 con 0x7ff1f01f2400 2024-09-10T10:27:45.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.968+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff1f00c7c40 con 0x7ff1f01f2400 2024-09-10T10:27:45.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.970+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1f00c7c40 con 0x7ff1f01f2400 2024-09-10T10:27:45.971 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:45.971 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:45.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.970+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ff1f00c7460 0x7ff1f0062b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.970+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ff1f0072ee0 con 0x7ff1f00c7460 2024-09-10T10:27:45.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.970+0000 7ff1effff640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ff1f00c7460 0x7ff1f0062b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.970+0000 7ff1effff640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ff1f00c7460 0x7ff1f0062b40 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:45.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.971+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1f0072ee0 con 0x7ff1f00c7460 2024-09-10T10:27:45.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.985+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff1f006c270 con 0x7ff1f00c7460 2024-09-10T10:27:45.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.986+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1f006c270 con 0x7ff1f00c7460 2024-09-10T10:27:45.988 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.987+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ff1f00c5270 0x7ff1f00c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.987+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ff1f0063590 con 0x7ff1f00c5270 2024-09-10T10:27:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.987+0000 7ff1ef7fe640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ff1f00c5270 0x7ff1f00c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.987+0000 7ff1ef7fe640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ff1f00c5270 0x7ff1f00c5670 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:45.988+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1f0063590 con 0x7ff1f00c5270 2024-09-10T10:27:46.002 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:46.002 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:46.002 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:46.002 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:46.002 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.002 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:46.002 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:27:46.002 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.220606764999999, 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047203150000000001, 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.003 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779674+0000", 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0359999999999995e-06 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779702+0000", 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.7970000000000002e-05 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.004 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779724+0000", 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2544e-05 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779741+0000", 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6781000000000001e-05 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779981+0000", 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000239764 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780147+0000", 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016605100000000001 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017096200000000001 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781749+0000", 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001431288 2024-09-10T10:27:46.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784234+0000", 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024850940000000002 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784338+0000", 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104078 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.007 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784385+0000", 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6746999999999998e-05 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.1a epoch 24/18 rep_tid 2 entries 24'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.219128654999999, 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021396129999999998, 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.008 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.009 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781144+0000", 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.393e-06 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781156+0000", 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1514e-05 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8445999999999998e-05 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781202+0000", 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7439e-05 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781286+0000", 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4652000000000002e-05 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783282+0000", 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0019961689999999999 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.011 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:31 2.d e24/18 2:b50e409b:::mds_snaptable:head v 24'1, mlcod=0'0)", 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.218209164000001, 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030643699999999999, 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.012 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782063+0000", 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3699999999999999e-07 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.013 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782072+0000", 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9430000000000006e-06 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782083+0000", 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0998e-05 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782092+0000", 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.3270000000000001e-06 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782158+0000", 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6830999999999998e-05 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782176+0000", 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7370000000000001e-05 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785091+0000", 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029157879999999999 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785127+0000", 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5176000000000003e-05 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.218150272999999, 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002013515, 2024-09-10T10:27:46.016 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782122+0000", 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.229e-06 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782137+0000", 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4895999999999999e-05 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782151+0000", 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3229e-05 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782198+0000", 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7301e-05 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782271+0000", 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3516000000000006e-05 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782290+0000", 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8921000000000001e-05 2024-09-10T10:27:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784094+0000", 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00180331 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784135+0000", 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1112999999999998e-05 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:28 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.218038659999998, 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028487970000000001, 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7000000000000005e-07 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782247+0000", 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3740000000000001e-05 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782258+0000", 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0552e-05 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782265+0000", 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0700000000000001e-06 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782752+0000", 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048707399999999999 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782771+0000", 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9046999999999999e-05 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785037+0000", 2024-09-10T10:27:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002265806 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785082+0000", 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4938000000000002e-05 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:29 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:27:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.216492664999997, 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001932907, 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783780+0000", 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9900000000000009e-07 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783789+0000", 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0140000000000004e-06 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783802+0000", 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2751e-05 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783811+0000", 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3950000000000007e-06 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783894+0000", 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2887999999999997e-05 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783913+0000", 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9052999999999999e-05 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785668+0000", 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001754633 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785712+0000", 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4174000000000001e-05 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:7 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 37.127860624999997, 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026422899999999998, 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:27:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.561e-06 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872426+0000", 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3577999999999999e-05 2024-09-10T10:27:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872462+0000", 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6145e-05 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872473+0000", 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1109e-05 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872543+0000", 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9319000000000005e-05 2024-09-10T10:27:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872566+0000", 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3768999999999999e-05 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875010+0000", 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024433910000000001 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875053+0000", 2024-09-10T10:27:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3417999999999997e-05 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:8 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 37.127172236, 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0044428820000000004, 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873100+0000", 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3030000000000005e-06 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873121+0000", 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1471000000000001e-05 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873129+0000", 2024-09-10T10:27:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7339999999999997e-06 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048786499999999998 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873638+0000", 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0735000000000001e-05 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877493+0000", 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0038550189999999999 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877542+0000", 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9135000000000003e-05 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:17 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:27:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 37.126954902999998, 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042663060000000001, 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873318+0000", 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7999999999999993e-07 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873326+0000", 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8279999999999992e-06 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873341+0000", 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4688999999999999e-05 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873348+0000", 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4810000000000003e-06 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873843+0000", 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000495083 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.042 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873860+0000", 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7609999999999999e-05 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877545+0000", 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036848839999999998 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877583+0000", 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7750999999999998e-05 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.043 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:18 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 37.126232866000002, 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035807780000000002, 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.045 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874045+0000", 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9909999999999999e-06 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874055+0000", 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0147e-05 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874070+0000", 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4184e-05 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.046 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874123+0000", 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3504999999999997e-05 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874141+0000", 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7524999999999999e-05 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877586+0000", 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034458700000000002 2024-09-10T10:27:46.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877619+0000", 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2962000000000001e-05 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.18 epoch 26/18 rep_tid 5 entries 26'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.9:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 37.125171221000002, 2024-09-10T10:27:46.048 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048872359999999997, 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875101+0000", 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875110+0000", 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2609999999999999e-06 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875129+0000", 2024-09-10T10:27:46.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8685000000000002e-05 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875143+0000", 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3723000000000001e-05 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875222+0000", 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9498000000000004e-05 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879987+0000", 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047652800000000002 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:7 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'5, mlcod=26'3)", 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 32.050408924000003, 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031425310000000001, 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.052 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.053 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1100000000000005e-07 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949871+0000", 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8620000000000003e-06 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949885+0000", 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4072e-05 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949893+0000", 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5769999999999996e-06 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949953+0000", 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.978e-05 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949972+0000", 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9188e-05 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.952959+0000", 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029873859999999999 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953005+0000", 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5855000000000001e-05 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:8 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'6, mlcod=26'3)", 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 32.050356334999996, 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034236359999999999, 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949916+0000", 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6199999999999998e-07 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949922+0000", 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5390000000000004e-06 2024-09-10T10:27:46.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949929+0000", 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4339999999999997e-06 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949934+0000", 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2159999999999997e-06 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950192+0000", 2024-09-10T10:27:46.059 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00025864699999999998 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950213+0000", 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0761999999999999e-05 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953297+0000", 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030842999999999999 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953339+0000", 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1276000000000003e-05 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:34 2.11 e26/18 2:8975f766:::202.00000001:head v 26'3, mlcod=26'1)", 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 32.046383495999997, 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020619190000000002, 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.061 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953889+0000", 2024-09-10T10:27:46.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.879999999999999e-07 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953895+0000", 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2249999999999997e-06 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953906+0000", 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953912+0000", 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4840000000000003e-06 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953959+0000", 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7768000000000001e-05 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953976+0000", 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6365999999999999e-05 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955917+0000", 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001941524 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955950+0000", 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2573999999999997e-05 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.065 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:4 2.1a e26/18 2:5fdc322d:::61e.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 26.077315968000001, 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0018931340000000001, 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922956+0000", 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-07 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922962+0000", 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1890000000000004e-06 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922971+0000", 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922978+0000", 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7830000000000001e-06 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923032+0000", 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3637999999999997e-05 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923049+0000", 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7249999999999999e-05 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924803+0000", 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001753788 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924849+0000", 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6050000000000001e-05 2024-09-10T10:27:46.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:20 2.11 e26/18 2:890f50f7:::626.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 26.077208261999999, 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003328012, 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.070 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923064+0000", 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9900000000000001e-07 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923070+0000", 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7499999999999997e-06 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923079+0000", 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9080000000000001e-06 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923089+0000", 2024-09-10T10:27:46.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0078000000000001e-05 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923150+0000", 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0300999999999997e-05 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923507+0000", 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00035713599999999998 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926368+0000", 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028615469999999999 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926391+0000", 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2793e-05 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.074 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:23 2.18 e26/18 2:1c1db442:::103.00000000.inode:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 26.077148792999999, 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032793050000000002, 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.075 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923124+0000", 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0599999999999999e-07 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923130+0000", 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.81e-06 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923141+0000", 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923149+0000", 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2679999999999997e-06 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4744000000000002e-05 2024-09-10T10:27:46.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923233+0000", 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9190000000000001e-05 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926353+0000", 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031206419999999999 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926402+0000", 2024-09-10T10:27:46.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8755000000000003e-05 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:13 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'6, mlcod=26'2)", 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 26.077096163, 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031818969999999999, 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923176+0000", 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4799999999999998e-07 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923182+0000", 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.592e-06 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923191+0000", 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4029999999999996e-06 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923197+0000", 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1850000000000001e-06 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923230+0000", 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3290000000000001e-05 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923247+0000", 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6810999999999999e-05 2024-09-10T10:27:46.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926308+0000", 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030606159999999999 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926357+0000", 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9351999999999998e-05 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.083 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.d epoch 26/18 rep_tid 20 entries 26'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.15:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 26.076543664999999, 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034028819999999999, 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.084 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923729+0000", 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.5900000000000005e-07 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923739+0000", 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8439999999999993e-06 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923761+0000", 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2821999999999999e-05 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923776+0000", 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4763000000000001e-05 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923889+0000", 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000112518 2024-09-10T10:27:46.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927131+0000", 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032419760000000001 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:14 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'7, mlcod=26'2)", 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 26.075828703999999, 2024-09-10T10:27:46.087 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027184890000000001, 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:46.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5099999999999999e-07 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924449+0000", 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0310000000000004e-06 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924482+0000", 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2845999999999997e-05 2024-09-10T10:27:46.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924500+0000", 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.787e-05 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924594+0000", 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3566999999999994e-05 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924619+0000", 2024-09-10T10:27:46.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4978999999999999e-05 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927118+0000", 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024995479999999999 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927161+0000", 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2997000000000002e-05 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:46.091 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:46.092 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:46.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.001+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff1f0063590 con 0x7ff1f00c5270 2024-09-10T10:27:46.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.002+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1f0063590 con 0x7ff1f00c5270 2024-09-10T10:27:46.093 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:46.094 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:27:46.094 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:46.092+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.002+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ff1f00c5f70 0x7ff1f01efd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.003+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ff1f0063590 con 0x7ff1f00c5f70 2024-09-10T10:27:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.003+0000 7ff1f4c95640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ff1f00c5f70 0x7ff1f01efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.003+0000 7ff1f4c95640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ff1f00c5f70 0x7ff1f01efd50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.003+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1f0063590 con 0x7ff1f00c5f70 2024-09-10T10:27:46.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.013+0000 7f6168ff9640 1 -- 172.21.15.120:0/1700950158 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v33) ==== 101+0+0 (secure 0 0 0) 0x7f6164058f50 con 0x7f616c152a80 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 -- 172.21.15.120:0/1700950158 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f614c06be30 msgr2=0x7f614c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f614c06be30 0x7f614c06e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f615c034000 tx=0x7f615c02e000 comp rx=0 tx=0).stop 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 -- 172.21.15.120:0/1700950158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 msgr2=0x7f616c0ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 0x7f616c0ca890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f616400c7d0 tx=0x7f616400cc90 comp rx=0 tx=0).stop 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 -- 172.21.15.120:0/1700950158 shutdown_connections 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f614c06be30 0x7f614c06e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f616c152a80 0x7f616c0ca890 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0cd330 0x7f616c1f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 --2- 172.21.15.120:0/1700950158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f616c0c5be0 0x7f616c0c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 -- 172.21.15.120:0/1700950158 >> 172.21.15.120:0/1700950158 conn(0x7f616c06bc10 msgr2=0x7f616c06fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:46.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 -- 172.21.15.120:0/1700950158 shutdown_connections 2024-09-10T10:27:46.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.015+0000 7f6171e53640 1 -- 172.21.15.120:0/1700950158 wait complete. 2024-09-10T10:27:46.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.019+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff1f0063590 con 0x7ff1f00c5f70 2024-09-10T10:27:46.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.021+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1f0063590 con 0x7ff1f00c5f70 2024-09-10T10:27:46.098 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:46.099 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:46.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.021+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff1f01f0ad0 0x7ff1f002c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:46.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.021+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ff1f0205f20 con 0x7ff1f01f0ad0 2024-09-10T10:27:46.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.021+0000 7ff1effff640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff1f01f0ad0 0x7ff1f002c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:46.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.021+0000 7ff1effff640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff1f01f0ad0 0x7ff1f002c640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:46.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.022+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1f0205f20 con 0x7ff1f01f0ad0 2024-09-10T10:27:46.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.036+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff1f002cb80 con 0x7ff1f01f0ad0 2024-09-10T10:27:46.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.038+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1f002cb80 con 0x7ff1f01f0ad0 2024-09-10T10:27:46.101 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.038+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ff1f002d2a0 0x7ff1f002d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.038+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ff1f0209180 con 0x7ff1f002d2a0 2024-09-10T10:27:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.038+0000 7ff1ef7fe640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ff1f002d2a0 0x7ff1f002d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.038+0000 7ff1ef7fe640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ff1f002d2a0 0x7ff1f002d6c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.039+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1f0209180 con 0x7ff1f002d2a0 2024-09-10T10:27:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.053+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff1f0075900 con 0x7ff1f002d2a0 2024-09-10T10:27:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.054+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1f0075900 con 0x7ff1f002d2a0 2024-09-10T10:27:46.102 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.054+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff1f0206d30 0x7ff1f0207190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.054+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ff1f020c550 con 0x7ff1f0206d30 2024-09-10T10:27:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.055+0000 7ff1f4c95640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff1f0206d30 0x7ff1f0207190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:46.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.055+0000 7ff1f4c95640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff1f0206d30 0x7ff1f0207190 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:46.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.056+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1f020c550 con 0x7ff1f0206d30 2024-09-10T10:27:46.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.069+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff1f02077e0 con 0x7ff1f0206d30 2024-09-10T10:27:46.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.071+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1f02077e0 con 0x7ff1f0206d30 2024-09-10T10:27:46.104 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.071+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1f0207e60 0x7ff1f02082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.071+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7ff1f0208a20 con 0x7ff1f0207e60 2024-09-10T10:27:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.071+0000 7ff1effff640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1f0207e60 0x7ff1f02082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.072+0000 7ff1effff640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1f0207e60 0x7ff1f02082c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.072+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1f0208a20 con 0x7ff1f0207e60 2024-09-10T10:27:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.085+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff1f01f0310 con 0x7ff1f0207e60 2024-09-10T10:27:46.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.087+0000 7ff1ecff9640 1 -- 172.21.15.120:0/1772338171 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1f01f0310 con 0x7ff1f0207e60 2024-09-10T10:27:46.106 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ff1f01f2400 msgr2=0x7ff1f015e050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ff1f01f2400 0x7ff1f015e050 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ff1f00c7460 msgr2=0x7ff1f0062b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ff1f00c7460 0x7ff1f0062b40 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ff1f00c5270 msgr2=0x7ff1f00c5670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ff1f00c5270 0x7ff1f00c5670 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ff1f00c5f70 msgr2=0x7ff1f01efd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ff1f00c5f70 0x7ff1f01efd50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff1f01f0ad0 msgr2=0x7ff1f002c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff1f01f0ad0 0x7ff1f002c640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ff1f002d2a0 msgr2=0x7ff1f002d6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ff1f002d2a0 0x7ff1f002d6c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff1f0206d30 msgr2=0x7ff1f0207190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f4c95640 1 -- 172.21.15.120:0/1772338171 reap_dead start 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff1f0206d30 0x7ff1f0207190 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f4c95640 1 -- 172.21.15.120:0/1772338171 reap_dead start 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f4c95640 1 -- 172.21.15.120:0/1772338171 reap_dead start 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1f0207e60 msgr2=0x7ff1f02082c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1f0207e60 0x7ff1f02082c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1dc06be30 msgr2=0x7ff1dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1dc06be30 0x7ff1dc06e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0154b10 tx=0x7ff1e4007de0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0070d40 msgr2=0x7ff1f01f29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0070d40 0x7ff1f01f29c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000e9b0 tx=0x7ff1e000ee70 comp rx=0 tx=0).stop 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 shutdown_connections 2024-09-10T10:27:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff1f0206d30 0x7ff1f0207190 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1dc06be30 0x7ff1dc06e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1f0207e60 0x7ff1f02082c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 --2- 172.21.15.120:0/1772338171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1f0070d40 0x7ff1f01f29c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 >> 172.21.15.120:0/1772338171 conn(0x7ff1f0072a60 msgr2=0x7ff1f015c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 shutdown_connections 2024-09-10T10:27:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:46.088+0000 7ff1f66f8640 1 -- 172.21.15.120:0/1772338171 wait complete. 2024-09-10T10:27:46.194 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:46.193+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:46.237 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:46.237 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:46.246 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:27:46.294 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:46.293+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:46.348 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:46.349 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:46.357 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:27:46.398 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:46.397+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:46.497 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:46.496+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:46.500 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:46.500 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:46.509 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:27:46.598 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:46.597+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:46.653 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:46.653 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:46.662 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:27:46.699 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:46.698+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:46.765 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:46.765 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:46.775 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:27:46.800 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:46.799+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:46.878 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:46.878 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:46.886 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:27:46.901 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:46.900+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:46.989 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:46.989 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:46.998 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:27:47.001 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:47.000+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:47.102 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:47.101+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:47.102 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:47.102 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:47.111 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:27:47.203 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:47.202+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:47.213 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:47.213 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:47.222 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:27:47.304 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:47.303+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:47.338 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:47.338 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:47.347 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:27:47.404 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:47.403+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:47.490 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:47.490 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:47.498 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:27:47.505 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:47.504+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:47.601 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:47.601 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:47.605 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:47.604+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:47.609 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:27:47.706 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:47.705+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:47.753 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:47.753 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:47.762 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:27:47.807 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:47.807+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:47.864 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:47.864 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:47.873 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:27:47.908 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:47.908+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:48.009 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:48.008+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:48.011 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:27:48.011 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:27:48.011 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-10T10:27:48.011 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-10T10:27:48.017 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:48.017 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:48.027 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:27:48.100 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-10T10:27:48.110 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:27:48.111 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:48.110+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.111+0000 7fc801853640 1 Processor -- start 2024-09-10T10:27:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.112+0000 7fc801853640 1 -- start start 2024-09-10T10:27:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.112+0000 7fc801853640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cb2f0 0x7fc7fc15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.112+0000 7fc801853640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc7fc0cc4a0 0x7fc7fc165430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.112+0000 7fc801853640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7fc15b1d0 con 0x7fc7fc0cc0c0 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.112+0000 7fc801853640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc7fc0cbd00 con 0x7fc7fc0cc4a0 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.112+0000 7fc801853640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc7fc15ba70 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.113+0000 7fc7faffd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fc7fc0cc0c0 0x7fc7fc15e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57240/0 (socket says 172.21.15.120:57240) 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.113+0000 7fc7fb7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc7fc0cc4a0 0x7fc7fc165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.113+0000 7fc7faffd640 1 -- 172.21.15.120:0/1514169376 learned_addr learned my addr 172.21.15.120:0/1514169376 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.113+0000 7fc7fa7fc640 1 --2- 172.21.15.120:0/1514169376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cb2f0 0x7fc7fc15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.113+0000 7fc7fb7fe640 1 -- 172.21.15.120:0/1514169376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cb2f0 msgr2=0x7fc7fc15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.113+0000 7fc7fb7fe640 1 --2- 172.21.15.120:0/1514169376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cb2f0 0x7fc7fc15aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.113+0000 7fc7fb7fe640 1 -- 172.21.15.120:0/1514169376 >> v1:172.21.15.120:6789/0 conn(0x7fc7fc0cc0c0 legacy=0x7fc7fc15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.113+0000 7fc7fb7fe640 1 -- 172.21.15.120:0/1514169376 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7fc15b4b0 con 0x7fc7fc0cc4a0 2024-09-10T10:27:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.113+0000 7fc7fa7fc640 1 --2- 172.21.15.120:0/1514169376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cb2f0 0x7fc7fc15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:27:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.114+0000 7fc7fb7fe640 1 --2- 172.21.15.120:0/1514169376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cc4a0 0x7fc7fc165430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009d90 tx=0x7fc7f402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abdba601137bf4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.114+0000 7fc7f9ffb640 1 -- 172.21.15.120:0/1514169376 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4007740 con 0x7fc7fc0cc4a0 2024-09-10T10:27:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.114+0000 7fc7f9ffb640 1 -- 172.21.15.120:0/1514169376 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f40078e0 con 0x7fc7fc0cc4a0 2024-09-10T10:27:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.114+0000 7fc7f9ffb640 1 -- 172.21.15.120:0/1514169376 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4007cb0 con 0x7fc7fc0cc4a0 2024-09-10T10:27:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.115+0000 7fc801853640 1 -- 172.21.15.120:0/1514169376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cc4a0 msgr2=0x7fc7fc165430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.115+0000 7fc801853640 1 --2- 172.21.15.120:0/1514169376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cc4a0 0x7fc7fc165430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009d90 tx=0x7fc7f402a560 comp rx=0 tx=0).stop 2024-09-10T10:27:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.115+0000 7fc801853640 1 -- 172.21.15.120:0/1514169376 shutdown_connections 2024-09-10T10:27:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.115+0000 7fc801853640 1 --2- 172.21.15.120:0/1514169376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cc4a0 0x7fc7fc165430 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.115+0000 7fc801853640 1 --2- 172.21.15.120:0/1514169376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cb2f0 0x7fc7fc15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.115+0000 7fc801853640 1 -- 172.21.15.120:0/1514169376 >> 172.21.15.120:0/1514169376 conn(0x7fc7fc06d560 msgr2=0x7fc7fc06d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.115+0000 7fc801853640 1 -- 172.21.15.120:0/1514169376 shutdown_connections 2024-09-10T10:27:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.115+0000 7fc801853640 1 -- 172.21.15.120:0/1514169376 wait complete. 2024-09-10T10:27:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.117+0000 7fc801853640 1 Processor -- start 2024-09-10T10:27:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.117+0000 7fc801853640 1 -- start start 2024-09-10T10:27:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc801853640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cb2f0 0x7fc7fc0c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc801853640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cc0c0 0x7fc7fc0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc801853640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7fc0cc4a0 0x7fc7fc148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc801853640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc15e1d0 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc7faffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cb2f0 0x7fc7fc0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc7faffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cb2f0 0x7fc7fc0c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42252/0 (socket says 172.21.15.120:42252) 2024-09-10T10:27:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc7faffd640 1 -- 172.21.15.120:0/4077371090 learned_addr learned my addr 172.21.15.120:0/4077371090 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc15e540 con 0x7fc7fc0cc4a0 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc7fc0cc8b0 con 0x7fc7fc0cc0c0 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc7fa7fc640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cc0c0 0x7fc7fc0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc7fb7fe640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7fc0cc4a0 0x7fc7fc148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc7faffd640 1 -- 172.21.15.120:0/4077371090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cc0c0 msgr2=0x7fc7fc0c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc7faffd640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cc0c0 0x7fc7fc0c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc7faffd640 1 -- 172.21.15.120:0/4077371090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7fc0cc4a0 msgr2=0x7fc7fc148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc7faffd640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7fc0cc4a0 0x7fc7fc148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.118+0000 7fc7faffd640 1 -- 172.21.15.120:0/4077371090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7fc147d80 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.119+0000 7fc7faffd640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cb2f0 0x7fc7fc0c8a70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c7b0 tx=0x7fc7ec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.119+0000 7fc7e3fff640 1 -- 172.21.15.120:0/4077371090 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec018070 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.119+0000 7fc7e3fff640 1 -- 172.21.15.120:0/4077371090 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ec0072f0 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.119+0000 7fc7e3fff640 1 -- 172.21.15.120:0/4077371090 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0047e0 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.119+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7fc1f0920 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.119+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7fc1f0ee0 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.120+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7fc0c5560 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.121+0000 7fc7e3fff640 1 -- 172.21.15.120:0/4077371090 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc7ec01b070 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.121+0000 7fc7e3fff640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc7e806be60 0x7fc7e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.121+0000 7fc7e3fff640 1 -- 172.21.15.120:0/4077371090 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6706+0+0 (secure 0 0 0) 0x7fc7ec08a670 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.122+0000 7fc7fa7fc640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc7e806be60 0x7fc7e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.122+0000 7fc7fa7fc640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc7e806be60 0x7fc7e806e320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0d0050 tx=0x7fc7f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.126+0000 7fc7e3fff640 1 -- 172.21.15.120:0/4077371090 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc7ec058540 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.191 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:48.191 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:48.201 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:27:48.210 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:48.210+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:48.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.211+0000 7fa33edc4640 1 Processor -- start 2024-09-10T10:27:48.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.211+0000 7fa33edc4640 1 -- start start 2024-09-10T10:27:48.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa33edc4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa3380cca70 0x7fa33815b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa33edc4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cafa0 0x7fa3380cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa33edc4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa33815e0b0 con 0x7fa338154c10 2024-09-10T10:27:48.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa33edc4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa33815de60 con 0x7fa3380cca70 2024-09-10T10:27:48.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa33edc4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa33815e390 con 0x7fa3380cafa0 2024-09-10T10:27:48.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa33cb60640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fa338154c10 0x7fa338073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57250/0 (socket says 172.21.15.120:57250) 2024-09-10T10:27:48.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa33cb60640 1 -- 172.21.15.120:0/3464228017 learned_addr learned my addr 172.21.15.120:0/3464228017 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:48.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa337fff640 1 --2- 172.21.15.120:0/3464228017 >> v2:172.21.15.134:3300/0 conn(0x7fa3380cca70 0x7fa33815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa33d361640 1 --2- 172.21.15.120:0/3464228017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cafa0 0x7fa3380cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa3377fe640 1 -- 172.21.15.120:0/3464228017 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1418800689 0 0) 0x7fa33815e0b0 con 0x7fa338154c10 2024-09-10T10:27:48.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa3377fe640 1 -- 172.21.15.120:0/3464228017 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa328003520 con 0x7fa338154c10 2024-09-10T10:27:48.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa337fff640 1 -- 172.21.15.120:0/3464228017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cafa0 msgr2=0x7fa3380cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:48.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa337fff640 1 --2- 172.21.15.120:0/3464228017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cafa0 0x7fa3380cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa337fff640 1 -- 172.21.15.120:0/3464228017 >> v1:172.21.15.120:6789/0 conn(0x7fa338154c10 legacy=0x7fa338073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa337fff640 1 -- 172.21.15.120:0/3464228017 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33815e670 con 0x7fa3380cca70 2024-09-10T10:27:48.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.212+0000 7fa33d361640 1 --2- 172.21.15.120:0/3464228017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cafa0 0x7fa3380cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.213+0000 7fa337fff640 1 --2- 172.21.15.120:0/3464228017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3380cca70 0x7fa33815b370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa324009790 tx=0x7fa32402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec23bb630b3b97f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.213+0000 7fa3377fe640 1 -- 172.21.15.120:0/3464228017 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324037070 con 0x7fa3380cca70 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.213+0000 7fa3377fe640 1 -- 172.21.15.120:0/3464228017 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa324032440 con 0x7fa3380cca70 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.213+0000 7fa3377fe640 1 -- 172.21.15.120:0/3464228017 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324032810 con 0x7fa3380cca70 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.214+0000 7fa33edc4640 1 -- 172.21.15.120:0/3464228017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3380cca70 msgr2=0x7fa33815b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.214+0000 7fa33edc4640 1 --2- 172.21.15.120:0/3464228017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3380cca70 0x7fa33815b370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa324009790 tx=0x7fa32402a180 comp rx=0 tx=0).stop 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.214+0000 7fa33edc4640 1 -- 172.21.15.120:0/3464228017 shutdown_connections 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.214+0000 7fa33edc4640 1 --2- 172.21.15.120:0/3464228017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cafa0 0x7fa3380cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.214+0000 7fa33edc4640 1 --2- 172.21.15.120:0/3464228017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3380cca70 0x7fa33815b370 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.214+0000 7fa33edc4640 1 -- 172.21.15.120:0/3464228017 >> 172.21.15.120:0/3464228017 conn(0x7fa33806ce80 msgr2=0x7fa33806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.214+0000 7fa33edc4640 1 -- 172.21.15.120:0/3464228017 shutdown_connections 2024-09-10T10:27:48.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.214+0000 7fa33edc4640 1 -- 172.21.15.120:0/3464228017 wait complete. 2024-09-10T10:27:48.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.215+0000 7f510d36d640 1 Processor -- start 2024-09-10T10:27:48.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.215+0000 7f510d36d640 1 -- start start 2024-09-10T10:27:48.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.215+0000 7fa33edc4640 1 Processor -- start 2024-09-10T10:27:48.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.215+0000 7fa33edc4640 1 -- start start 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f510d36d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f5108154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f510d36d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f510806d270 0x7f5108154f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f510d36d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5108067ad0 con 0x7f510806ce90 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f510d36d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5108068050 con 0x7f510806d270 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f510d36d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51080c91d0 con 0x7f5108070520 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f51067fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f5108154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f51067fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f5108154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:36510/0 (socket says 172.21.15.120:36510) 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f51077fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f510806d270 0x7f5108154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f51067fc640 1 -- 172.21.15.120:0/2706724323 learned_addr learned my addr 172.21.15.120:0/2706724323 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f51067fc640 1 -- 172.21.15.120:0/2706724323 >> v2:172.21.15.134:3300/0 conn(0x7f510806d270 msgr2=0x7f5108154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f51067fc640 1 --2- 172.21.15.120:0/2706724323 >> v2:172.21.15.134:3300/0 conn(0x7f510806d270 0x7f5108154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f51067fc640 1 -- 172.21.15.120:0/2706724323 >> v1:172.21.15.134:6789/0 conn(0x7f510806ce90 legacy=0x7f510815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f5105ffb640 1 -- 172.21.15.120:0/2706724323 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710370518 0 0) 0x7f5108067ad0 con 0x7f510806ce90 2024-09-10T10:27:48.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f51067fc640 1 -- 172.21.15.120:0/2706724323 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5108067d70 con 0x7f5108070520 2024-09-10T10:27:48.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7f51077fe640 1 --2- 172.21.15.120:0/2706724323 >> v2:172.21.15.134:3300/0 conn(0x7f510806d270 0x7f5108154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:48.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7fa33edc4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3380cafa0 0x7fa3380c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.216+0000 7fa33edc4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cca70 0x7fa3381f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa33edc4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa338154c10 0x7fa3380c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa33edc4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa338154ff0 con 0x7fa338154c10 2024-09-10T10:27:48.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa33edc4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa33815ec30 con 0x7fa3380cafa0 2024-09-10T10:27:48.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa33edc4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa338073a50 con 0x7fa3380cca70 2024-09-10T10:27:48.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa33cb60640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3380cafa0 0x7fa3380c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa337fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cca70 0x7fa3381f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa337fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cca70 0x7fa3381f10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:36512/0 (socket says 172.21.15.120:36512) 2024-09-10T10:27:48.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa337fff640 1 -- 172.21.15.120:0/1927374028 learned_addr learned my addr 172.21.15.120:0/1927374028 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:48.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa33d361640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa338154c10 0x7fa3380c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa337fff640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3380cafa0 msgr2=0x7fa3380c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa337fff640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3380cafa0 0x7fa3380c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa337fff640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa338154c10 msgr2=0x7fa3380c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa337fff640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa338154c10 0x7fa3380c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7f51067fc640 1 --2- 172.21.15.120:0/2706724323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f5108154a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009870 tx=0x7f50fc02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7de7986fe830669 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa337fff640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa338158200 con 0x7fa3380cca70 2024-09-10T10:27:48.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa33cb60640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3380cafa0 0x7fa3380c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:48.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7fa337fff640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cca70 0x7fa3381f10c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3240098b0 tx=0x7fa3240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7f5105ffb640 1 -- 172.21.15.120:0/2706724323 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc037070 con 0x7f5108070520 2024-09-10T10:27:48.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.217+0000 7f5105ffb640 1 -- 172.21.15.120:0/2706724323 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50fc02ae50 con 0x7f5108070520 2024-09-10T10:27:48.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.218+0000 7f5105ffb640 1 -- 172.21.15.120:0/2706724323 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc032860 con 0x7f5108070520 2024-09-10T10:27:48.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.218+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324037040 con 0x7fa3380cca70 2024-09-10T10:27:48.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.218+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa324042070 con 0x7fa3380cca70 2024-09-10T10:27:48.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.218+0000 7fa33edc4640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3380ca970 con 0x7fa3380cca70 2024-09-10T10:27:48.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.218+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324005dd0 con 0x7fa3380cca70 2024-09-10T10:27:48.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.218+0000 7fa33edc4640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa338157f20 con 0x7fa3380cca70 2024-09-10T10:27:48.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.218+0000 7f510d36d640 1 -- 172.21.15.120:0/2706724323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 msgr2=0x7f5108154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.218+0000 7f510d36d640 1 --2- 172.21.15.120:0/2706724323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f5108154a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009870 tx=0x7f50fc02a300 comp rx=0 tx=0).stop 2024-09-10T10:27:48.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.219+0000 7f510d36d640 1 -- 172.21.15.120:0/2706724323 shutdown_connections 2024-09-10T10:27:48.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.219+0000 7f510d36d640 1 --2- 172.21.15.120:0/2706724323 >> v2:172.21.15.134:3300/0 conn(0x7f510806d270 0x7f5108154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.219+0000 7f510d36d640 1 --2- 172.21.15.120:0/2706724323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f5108154a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.219+0000 7f510d36d640 1 -- 172.21.15.120:0/2706724323 >> 172.21.15.120:0/2706724323 conn(0x7f510806fa00 msgr2=0x7f510806fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:48.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.219+0000 7f510d36d640 1 -- 172.21.15.120:0/2706724323 shutdown_connections 2024-09-10T10:27:48.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.219+0000 7fa33edc4640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fa338153de0 con 0x7fa3380cca70 2024-09-10T10:27:48.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.219+0000 7f510d36d640 1 -- 172.21.15.120:0/2706724323 wait complete. 2024-09-10T10:27:48.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.220+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa324006030 con 0x7fa3380cca70 2024-09-10T10:27:48.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.220+0000 7fa3357fa640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa31c06beb0 0x7fa31c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.220+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(33..33 src has 1..33) ==== 6706+0+0 (secure 0 0 0) 0x7fa3240ab7e0 con 0x7fa3380cca70 2024-09-10T10:27:48.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.220+0000 7fa33cb60640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa31c06beb0 0x7fa31c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.220+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v33) ==== 54+0+16 (secure 0 0 0) 0x7fa338153de0 con 0x7fa3380cca70 2024-09-10T10:27:48.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.221+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fa310001650 0x7fa310003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.221+0000 7fa33cb60640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa31c06beb0 0x7fa31c06e370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa32c005880 tx=0x7fa32c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.221+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa310006bb0 con 0x7fa310001650 2024-09-10T10:27:48.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.221+0000 7f510d36d640 1 Processor -- start 2024-09-10T10:27:48.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.221+0000 7f510d36d640 1 -- start start 2024-09-10T10:27:48.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.221+0000 7fa33d361640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fa310001650 0x7fa310003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7fa33d361640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fa310001650 0x7fa310003b10 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f510d36d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f510806ce90 0x7f51080c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f510d36d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510806d270 0x7f51080c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f510d36d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f51081f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f510d36d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51080c8ef0 con 0x7f510806d270 2024-09-10T10:27:48.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f510d36d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51080c3aa0 con 0x7f510806ce90 2024-09-10T10:27:48.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f510d36d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5108070960 con 0x7f5108070520 2024-09-10T10:27:48.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f51067fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510806d270 0x7f51080c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f5106ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f510806ce90 0x7f51080c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f51067fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510806d270 0x7f51080c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42274/0 (socket says 172.21.15.120:42274) 2024-09-10T10:27:48.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f51067fc640 1 -- 172.21.15.120:0/1850893918 learned_addr learned my addr 172.21.15.120:0/1850893918 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:48.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f5106ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f510806ce90 0x7f51080c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:48072/0 (socket says 172.21.15.120:48072) 2024-09-10T10:27:48.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.222+0000 7f51077fe640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f51081f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa310006bb0 con 0x7fa310001650 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f5106ffd640 1 -- 172.21.15.120:0/1850893918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 msgr2=0x7f51081f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f5106ffd640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f51081f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f5106ffd640 1 -- 172.21.15.120:0/1850893918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510806d270 msgr2=0x7f51080c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f5106ffd640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510806d270 0x7f51080c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f5106ffd640 1 -- 172.21.15.120:0/1850893918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510815db40 con 0x7f510806ce90 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f51067fc640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510806d270 0x7f51080c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f51077fe640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f51081f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f5106ffd640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f510806ce90 0x7f51080c58b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f50f800c400 tx=0x7f50f800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f50ef7fe640 1 -- 172.21.15.120:0/1850893918 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8017070 con 0x7f510806ce90 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f50ef7fe640 1 -- 172.21.15.120:0/1850893918 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f80035f0 con 0x7f510806ce90 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f510d36d640 1 -- 172.21.15.120:0/1850893918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510815b8b0 con 0x7f510806ce90 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f50ef7fe640 1 -- 172.21.15.120:0/1850893918 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f80039e0 con 0x7f510806ce90 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.223+0000 7f510d36d640 1 -- 172.21.15.120:0/1850893918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51081f0060 con 0x7f510806ce90 2024-09-10T10:27:48.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.225+0000 7f510d36d640 1 -- 172.21.15.120:0/1850893918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510815eac0 con 0x7f510806ce90 2024-09-10T10:27:48.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.226+0000 7f50ef7fe640 1 -- 172.21.15.120:0/1850893918 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50f801a070 con 0x7f510806ce90 2024-09-10T10:27:48.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.226+0000 7f50ef7fe640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f50f406be30 0x7f50f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.226+0000 7f50ef7fe640 1 -- 172.21.15.120:0/1850893918 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6706+0+0 (secure 0 0 0) 0x7f50f808a550 con 0x7f510806ce90 2024-09-10T10:27:48.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.226+0000 7f51067fc640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f50f406be30 0x7f50f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.227+0000 7f51067fc640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f50f406be30 0x7f50f406e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0090b0 tx=0x7f50fc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.231+0000 7f50ef7fe640 1 -- 172.21.15.120:0/1850893918 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f50f8058390 con 0x7f510806ce90 2024-09-10T10:27:48.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.246+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa310005c60 con 0x7fa310001650 2024-09-10T10:27:48.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.248+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa310005c60 con 0x7fa310001650 2024-09-10T10:27:48.249 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:48.250 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:48.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.249+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fa310004190 0x7fa310004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.249+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fa310004c00 con 0x7fa310004190 2024-09-10T10:27:48.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.249+0000 7fa33cb60640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fa310004190 0x7fa310004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.249+0000 7fa33cb60640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fa310004190 0x7fa310004640 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.250+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa310004c00 con 0x7fa310004190 2024-09-10T10:27:48.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.278+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa3100051a0 con 0x7fa310004190 2024-09-10T10:27:48.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.280+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa3100051a0 con 0x7fa310004190 2024-09-10T10:27:48.282 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:48.282 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:48.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.281+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fa310009370 0x7fa31000b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.281+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fa31000e720 con 0x7fa310009370 2024-09-10T10:27:48.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.281+0000 7fa337fff640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fa310009370 0x7fa31000b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.281+0000 7fa337fff640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fa310009370 0x7fa31000b760 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.282+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa31000e720 con 0x7fa310009370 2024-09-10T10:27:48.311 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:48.310+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:48.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.311+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa31000d730 con 0x7fa310009370 2024-09-10T10:27:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.314+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa31000d730 con 0x7fa310009370 2024-09-10T10:27:48.315 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.314+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fa31000bd20 0x7fa31000c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.314+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fa31000c6c0 con 0x7fa31000bd20 2024-09-10T10:27:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.314+0000 7fa33d361640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fa31000bd20 0x7fa31000c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.314+0000 7fa33d361640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fa31000bd20 0x7fa31000c180 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.315+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa31000c6c0 con 0x7fa31000bd20 2024-09-10T10:27:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.340+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fc7fc0cbea0 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.341+0000 7fc7e3fff640 1 -- 172.21.15.120:0/4077371090 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v33) ==== 81+0+38639 (secure 0 0 0) 0x7fc7ec05c840 con 0x7fc7fc0cb2f0 2024-09-10T10:27:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:48.343 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 33, 2024-09-10T10:27:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:27:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-10T10:26:18.018818+0000", 2024-09-10T10:27:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-10T10:27:46.006510+0000", 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-10T10:26:29.078139+0000", 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-10T10:26:20.091201+0000", 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5799936, 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-10T10:27:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 4, 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "luminous", 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-10T10:27:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:30.118649+0000", 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_metadata", 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:38.936606+0000", 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "20", 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_bias": 4, 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 16, 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_priority": 5 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "metadata": "cephfs" 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-10T10:27:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data", 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:40.094780+0000", 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:27:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 64, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 64, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 64, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 64, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 64, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.05000000074505806 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.3799999952316284, 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.3799999952316284, 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T10:27:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 4, 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data_ec", 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:43.814005+0000", 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 5, 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,ec_overwrites", 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 3, 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 4, 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 3, 2024-09-10T10:27:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 1, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "26", 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "cephfs_data_ec", 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 8192, 2024-09-10T10:27:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e3b9c3c3-cf7d-4b33-913e-60e928439d8e", 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-10T10:27:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-10T10:27:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/3664783490", 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/3664783490", 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/3664783490", 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/3664783490", 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "62c20a9d-e6ab-446d-998c-9ee46e46aa90", 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/3972761964", 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/3972761964", 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/3972761964", 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/3972761964", 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1528f8e0-5e13-4ec7-9d98-0d94b7a505c3", 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-10T10:27:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2916410859", 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2916410859", 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2916410859", 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2916410859", 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "74942f78-c1fc-4886-abf0-950722c5edc6", 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 31, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1224278950", 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1224278950", 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1224278950", 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1224278950", 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:27:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e71319d4-5318-4441-84a8-7665ae037596", 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6816", 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6817", 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6818", 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6819", 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6822", 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6823", 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6820", 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6821", 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6817/1874635282", 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6819/1874635282", 2024-09-10T10:27:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6823/1874635282", 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6821/1874635282", 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "29e193cd-37af-43e5-bd0c-2f85ce0631e8", 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6808", 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6809", 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6810", 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6811", 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6814", 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6815", 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6812", 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6813", 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6809/457724087", 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6811/457724087", 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6815/457724087", 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6813/457724087", 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "55523ec2-cf12-44b3-aeea-f1cedabc996a", 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6824", 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6825", 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6826", 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6827", 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6830", 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6831", 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6828", 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6829", 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6825/50133702", 2024-09-10T10:27:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6827/50133702", 2024-09-10T10:27:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6831/50133702", 2024-09-10T10:27:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6829/50133702", 2024-09-10T10:27:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "097b1149-3e3c-4e5a-890b-d4cc4361d51b", 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 19, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6800", 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6801", 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6802", 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6803", 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6806", 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6807", 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6804", 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6805", 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6801/251960875", 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6803/251960875", 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6807/251960875", 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6805/251960875", 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.143056+0000", 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.423968+0000", 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.640076+0000", 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.536160+0000", 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.146033+0000", 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.426011+0000", 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.598992+0000", 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:27:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.182929+0000", 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [ 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.16", 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 1 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.1d", 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.28", 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.36", 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.d", 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 5 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.1b", 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-10T10:27:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_data_ec": { 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-device-class": "", 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-num-failure-domains": "0", 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-osds-per-failure-domain": "0", 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-root": "default", 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "jerasure-per-chunk-alignment": "false", 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:27:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "2", 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van", 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "w": "8" 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [], 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [], 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.429 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:48.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.343+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc7e806be60 msgr2=0x7fc7e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.343+0000 7fc801853640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc7e806be60 0x7fc7e806e320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0d0050 tx=0x7fc7f0008040 comp rx=0 tx=0).stop 2024-09-10T10:27:48.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.343+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cb2f0 msgr2=0x7fc7fc0c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.343+0000 7fc801853640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cb2f0 0x7fc7fc0c8a70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c7b0 tx=0x7fc7ec00cc70 comp rx=0 tx=0).stop 2024-09-10T10:27:48.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.344+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 shutdown_connections 2024-09-10T10:27:48.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.344+0000 7fc801853640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc7e806be60 0x7fc7e806e320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.344+0000 7fc801853640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7fc0cc4a0 0x7fc7fc148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.344+0000 7fc801853640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7fc0cc0c0 0x7fc7fc0c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.344+0000 7fc801853640 1 --2- 172.21.15.120:0/4077371090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc0cb2f0 0x7fc7fc0c8a70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.344+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 >> 172.21.15.120:0/4077371090 conn(0x7fc7fc06d560 msgr2=0x7fc7fc149050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:48.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.344+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 shutdown_connections 2024-09-10T10:27:48.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.344+0000 7fc801853640 1 -- 172.21.15.120:0/4077371090 wait complete. 2024-09-10T10:27:48.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.345+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa31000c6c0 con 0x7fa31000bd20 2024-09-10T10:27:48.433 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:48.433 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:48.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.348+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa31000c6c0 con 0x7fa31000bd20 2024-09-10T10:27:48.433 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:48.434 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:48.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.348+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fa3100108e0 0x7fa310012cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.348+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fa310015aa0 con 0x7fa3100108e0 2024-09-10T10:27:48.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.348+0000 7fa33cb60640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fa3100108e0 0x7fa310012cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.349+0000 7fa33cb60640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fa3100108e0 0x7fa310012cd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.350+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa310015aa0 con 0x7fa3100108e0 2024-09-10T10:27:48.436 INFO:tasks.thrashosds.thrasher:j is {'epoch': 33, 'fsid': '7d0d0830-4370-41c3-9acc-b67b07024f68', 'created': '2024-09-10T10:26:18.018818+0000', 'modified': '2024-09-10T10:27:46.006510+0000', 'last_up_change': '2024-09-10T10:26:29.078139+0000', 'last_in_change': '2024-09-10T10:26:20.091201+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:26:30.118649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:26:38.936606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:26:40.094780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T10:26:43.814005+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'e3b9c3c3-cf7d-4b33-913e-60e928439d8e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 3664783490}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 3664783490}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 3664783490}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 3664783490}]}, 'public_addr': '172.21.15.120:6809/3664783490', 'cluster_addr': '172.21.15.120:6811/3664783490', 'heartbeat_back_addr': '172.21.15.120:6815/3664783490', 'heartbeat_front_addr': '172.21.15.120:6813/3664783490', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '62c20a9d-e6ab-446d-998c-9ee46e46aa90', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 3972761964}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 3972761964}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 3972761964}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 3972761964}]}, 'public_addr': '172.21.15.120:6801/3972761964', 'cluster_addr': '172.21.15.120:6803/3972761964', 'heartbeat_back_addr': '172.21.15.120:6807/3972761964', 'heartbeat_front_addr': '172.21.15.120:6805/3972761964', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1528f8e0-5e13-4ec7-9d98-0d94b7a505c3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2916410859}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2916410859}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2916410859}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2916410859}]}, 'public_addr': '172.21.15.120:6817/2916410859', 'cluster_addr': '172.21.15.120:6819/2916410859', 'heartbeat_back_addr': '172.21.15.120:6823/2916410859', 'heartbeat_front_addr': '172.21.15.120:6821/2916410859', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '74942f78-c1fc-4886-abf0-950722c5edc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1224278950}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1224278950}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1224278950}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1224278950}]}, 'public_addr': '172.21.15.120:6825/1224278950', 'cluster_addr': '172.21.15.120:6827/1224278950', 'heartbeat_back_addr': '172.21.15.120:6831/1224278950', 'heartbeat_front_addr': '172.21.15.120:6829/1224278950', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e71319d4-5318-4441-84a8-7665ae037596', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6816', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6817', 'nonce': 1874635282}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6818', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6819', 'nonce': 1874635282}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6822', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6823', 'nonce': 1874635282}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6820', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6821', 'nonce': 1874635282}]}, 'public_addr': '172.21.15.134:6817/1874635282', 'cluster_addr': '172.21.15.134:6819/1874635282', 'heartbeat_back_addr': '172.21.15.134:6823/1874635282', 'heartbeat_front_addr': '172.21.15.134:6821/1874635282', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '29e193cd-37af-43e5-bd0c-2f85ce0631e8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6808', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6809', 'nonce': 457724087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6810', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6811', 'nonce': 457724087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6814', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6815', 'nonce': 457724087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6812', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6813', 'nonce': 457724087}]}, 'public_addr': '172.21.15.134:6809/457724087', 'cluster_addr': '172.21.15.134:6811/457724087', 'heartbeat_back_addr': '172.21.15.134:6815/457724087', 'heartbeat_front_addr': '172.21.15.134:6813/457724087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '55523ec2-cf12-44b3-aeea-f1cedabc996a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6824', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6825', 'nonce': 50133702}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6826', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6827', 'nonce': 50133702}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6830', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6831', 'nonce': 50133702}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6828', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6829', 'nonce': 50133702}]}, 'public_addr': '172.21.15.134:6825/50133702', 'cluster_addr': '172.21.15.134:6827/50133702', 'heartbeat_back_addr': '172.21.15.134:6831/50133702', 'heartbeat_front_addr': '172.21.15.134:6829/50133702', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '097b1149-3e3c-4e5a-890b-d4cc4361d51b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6800', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6801', 'nonce': 251960875}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6802', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6803', 'nonce': 251960875}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6806', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6807', 'nonce': 251960875}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6804', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6805', 'nonce': 251960875}]}, 'public_addr': '172.21.15.134:6801/251960875', 'cluster_addr': '172.21.15.134:6803/251960875', 'heartbeat_back_addr': '172.21.15.134:6807/251960875', 'heartbeat_front_addr': '172.21.15.134:6805/251960875', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.143056+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.423968+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.640076+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.536160+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.146033+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.426011+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.598992+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.182929+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.16', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.1d', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.28', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.36', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'cephfs_data_ec': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '2', 'plugin': 'jerasure', 'technique': 'reed_sol_van', 'w': '8'}, 'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-10T10:27:48.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:27:48.437 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:48.430+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:48.438 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:27:48.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.368+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa310013210 con 0x7fa3100108e0 2024-09-10T10:27:48.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.370+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa310013210 con 0x7fa3100108e0 2024-09-10T10:27:48.439 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:48.440 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:48.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.371+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fa310013930 0x7fa310015d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.371+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fa310018cc0 con 0x7fa310013930 2024-09-10T10:27:48.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.371+0000 7fa337fff640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fa310013930 0x7fa310015d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.371+0000 7fa337fff640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fa310013930 0x7fa310015d20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.372+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa310018cc0 con 0x7fa310013930 2024-09-10T10:27:48.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.385+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa3100008d0 con 0x7fa310013930 2024-09-10T10:27:48.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.387+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa3100008d0 con 0x7fa310013930 2024-09-10T10:27:48.441 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:48.442 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:48.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.387+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fa310016810 0x7fa310016c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.387+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fa31001c090 con 0x7fa310016810 2024-09-10T10:27:48.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.388+0000 7fa33d361640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fa310016810 0x7fa310016c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.388+0000 7fa33d361640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fa310016810 0x7fa310016c90 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.389+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa31001c090 con 0x7fa310016810 2024-09-10T10:27:48.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.402+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa3100172e0 con 0x7fa310016810 2024-09-10T10:27:48.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.404+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa3100172e0 con 0x7fa310016810 2024-09-10T10:27:48.443 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:48.444 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:48.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.404+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fa310017a00 0x7fa310017e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.404+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fa310018510 con 0x7fa310017a00 2024-09-10T10:27:48.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.405+0000 7fa33cb60640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fa310017a00 0x7fa310017e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.405+0000 7fa33cb60640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fa310017a00 0x7fa310017e40 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.406+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa310018510 con 0x7fa310017a00 2024-09-10T10:27:48.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.419+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa310018cc0 con 0x7fa310017a00 2024-09-10T10:27:48.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.421+0000 7fa3357fa640 1 -- 172.21.15.120:0/1927374028 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa310018cc0 con 0x7fa310017a00 2024-09-10T10:27:48.444 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:48.446 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:48.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fa310001650 msgr2=0x7fa310003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fa310001650 0x7fa310003b10 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fa310004190 msgr2=0x7fa310004640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fa310004190 0x7fa310004640 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fa310009370 msgr2=0x7fa31000b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fa310009370 0x7fa31000b760 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fa31000bd20 msgr2=0x7fa31000c180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fa31000bd20 0x7fa31000c180 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fa3100108e0 msgr2=0x7fa310012cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa33d361640 1 -- 172.21.15.120:0/1927374028 reap_dead start 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fa3100108e0 0x7fa310012cd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa33d361640 1 -- 172.21.15.120:0/1927374028 reap_dead start 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fa310013930 msgr2=0x7fa310015d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fa310013930 0x7fa310015d20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fa310016810 msgr2=0x7fa310016c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fa310016810 0x7fa310016c90 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fa310017a00 msgr2=0x7fa310017e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fa310017a00 0x7fa310017e40 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa31c06beb0 msgr2=0x7fa31c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa31c06beb0 0x7fa31c06e370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa32c005880 tx=0x7fa32c009040 comp rx=0 tx=0).stop 2024-09-10T10:27:48.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa33d361640 1 -- 172.21.15.120:0/1927374028 reap_dead start 2024-09-10T10:27:48.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cca70 msgr2=0x7fa3381f10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cca70 0x7fa3381f10c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3240098b0 tx=0x7fa3240046c0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 shutdown_connections 2024-09-10T10:27:48.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 --2- 172.21.15.120:0/1927374028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3380cca70 0x7fa3381f10c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 >> 172.21.15.120:0/1927374028 conn(0x7fa33806ce80 msgr2=0x7fa3380cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:48.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 shutdown_connections 2024-09-10T10:27:48.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.422+0000 7fa322ffd640 1 -- 172.21.15.120:0/1927374028 wait complete. 2024-09-10T10:27:48.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.425+0000 7f510d36d640 1 -- 172.21.15.120:0/1850893918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f51081598b0 con 0x7f510806ce90 2024-09-10T10:27:48.532 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:48.531+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:48.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.575+0000 7f6e589bd640 1 Processor -- start 2024-09-10T10:27:48.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.575+0000 7f6e589bd640 1 -- start start 2024-09-10T10:27:48.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.576+0000 7f6e589bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 0x7f6e540c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.576+0000 7f6e589bd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6e5414e180 0x7f6e5414e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.576+0000 7f6e589bd640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e540c9d60 con 0x7f6e540c6900 2024-09-10T10:27:48.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.576+0000 7f6e589bd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6e540ca040 con 0x7f6e5414e180 2024-09-10T10:27:48.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.576+0000 7f6e589bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6e540ca320 con 0x7f6e540c6520 2024-09-10T10:27:48.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.576+0000 7f6e52d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6e5414e180 0x7f6e5414e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.576+0000 7f6e5259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 0x7f6e540c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.576+0000 7f6e51d9b640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f6e540c6900 0x7f6e5406ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:50142/0 (socket says 172.21.15.120:50142) 2024-09-10T10:27:48.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.576+0000 7f6e51d9b640 1 -- 172.21.15.120:0/265154399 learned_addr learned my addr 172.21.15.120:0/265154399 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:48.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.577+0000 7f6e5159a640 1 -- 172.21.15.120:0/265154399 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955579463 0 0) 0x7f6e540c9d60 con 0x7f6e540c6900 2024-09-10T10:27:48.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.577+0000 7f6e5159a640 1 -- 172.21.15.120:0/265154399 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e3c003520 con 0x7f6e540c6900 2024-09-10T10:27:48.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.577+0000 7f6e5259c640 1 -- 172.21.15.120:0/265154399 >> v2:172.21.15.134:3300/0 conn(0x7f6e5414e180 msgr2=0x7f6e5414e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:48.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.577+0000 7f6e5259c640 1 --2- 172.21.15.120:0/265154399 >> v2:172.21.15.134:3300/0 conn(0x7f6e5414e180 0x7f6e5414e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.577+0000 7f6e5259c640 1 -- 172.21.15.120:0/265154399 >> v1:172.21.15.134:6789/0 conn(0x7f6e540c6900 legacy=0x7f6e5406ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.577+0000 7f6e5259c640 1 -- 172.21.15.120:0/265154399 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e540ca600 con 0x7f6e540c6520 2024-09-10T10:27:48.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.577+0000 7f6e52d9d640 1 --2- 172.21.15.120:0/265154399 >> v2:172.21.15.134:3300/0 conn(0x7f6e5414e180 0x7f6e5414e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:48.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.577+0000 7f6e5259c640 1 --2- 172.21.15.120:0/265154399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 0x7f6e540c46c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800ba40 tx=0x7f6e4802c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6123314c9668117f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e5159a640 1 -- 172.21.15.120:0/265154399 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e480327d0 con 0x7f6e540c6520 2024-09-10T10:27:48.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e5159a640 1 -- 172.21.15.120:0/265154399 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e48038070 con 0x7f6e540c6520 2024-09-10T10:27:48.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e5159a640 1 -- 172.21.15.120:0/265154399 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48032b70 con 0x7f6e540c6520 2024-09-10T10:27:48.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e589bd640 1 -- 172.21.15.120:0/265154399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 msgr2=0x7f6e540c46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e589bd640 1 --2- 172.21.15.120:0/265154399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 0x7f6e540c46c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800ba40 tx=0x7f6e4802c8d0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e589bd640 1 -- 172.21.15.120:0/265154399 shutdown_connections 2024-09-10T10:27:48.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e589bd640 1 --2- 172.21.15.120:0/265154399 >> v2:172.21.15.134:3300/0 conn(0x7f6e5414e180 0x7f6e5414e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e589bd640 1 --2- 172.21.15.120:0/265154399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 0x7f6e540c46c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e589bd640 1 -- 172.21.15.120:0/265154399 >> 172.21.15.120:0/265154399 conn(0x7f6e54068140 msgr2=0x7f6e54070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:48.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e589bd640 1 -- 172.21.15.120:0/265154399 shutdown_connections 2024-09-10T10:27:48.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.578+0000 7f6e589bd640 1 -- 172.21.15.120:0/265154399 wait complete. 2024-09-10T10:27:48.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e589bd640 1 Processor -- start 2024-09-10T10:27:48.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e589bd640 1 -- start start 2024-09-10T10:27:48.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e589bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 0x7f6e5414e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e589bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e540c6900 0x7f6e541497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e589bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414e180 0x7f6e541542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e589bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e540c7590 con 0x7f6e540c6900 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e589bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e540c4cc0 con 0x7f6e5414e180 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e589bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6e540c7710 con 0x7f6e540c6520 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e51d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e540c6900 0x7f6e541497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e51d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e540c6900 0x7f6e541497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42286/0 (socket says 172.21.15.120:42286) 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.580+0000 7f6e51d9b640 1 -- 172.21.15.120:0/713437177 learned_addr learned my addr 172.21.15.120:0/713437177 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e52d9d640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414e180 0x7f6e541542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e5259c640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 0x7f6e5414e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e51d9b640 1 -- 172.21.15.120:0/713437177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 msgr2=0x7f6e5414e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e51d9b640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 0x7f6e5414e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e51d9b640 1 -- 172.21.15.120:0/713437177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414e180 msgr2=0x7f6e541542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e51d9b640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414e180 0x7f6e541542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e51d9b640 1 -- 172.21.15.120:0/713437177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e540cd9b0 con 0x7f6e540c6900 2024-09-10T10:27:48.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e5259c640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 0x7f6e5414e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:48.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e52d9d640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414e180 0x7f6e541542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:48.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e51d9b640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e540c6900 0x7f6e541497e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c400 tx=0x7f6e4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e42ffd640 1 -- 172.21.15.120:0/713437177 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44017070 con 0x7f6e540c6900 2024-09-10T10:27:48.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e42ffd640 1 -- 172.21.15.120:0/713437177 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e44005d00 con 0x7f6e540c6900 2024-09-10T10:27:48.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e589bd640 1 -- 172.21.15.120:0/713437177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e54151e70 con 0x7f6e540c6900 2024-09-10T10:27:48.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e42ffd640 1 -- 172.21.15.120:0/713437177 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44007310 con 0x7f6e540c6900 2024-09-10T10:27:48.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.581+0000 7f6e589bd640 1 -- 172.21.15.120:0/713437177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e541f1910 con 0x7f6e540c6900 2024-09-10T10:27:48.586 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:27:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:27:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:27:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.586 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 61.801413699999998, 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078690670000000004, 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2200000000000004e-07 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780048+0000", 2024-09-10T10:27:48.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2427999999999999e-05 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780057+0000", 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1090000000000004e-06 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780063+0000", 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9780000000000002e-06 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011213480000000001 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781281+0000", 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6728e-05 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781382+0000", 2024-09-10T10:27:48.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101455 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784820+0000", 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034376200000000002 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787779+0000", 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002959373 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787852+0000", 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3135999999999999e-05 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787894+0000", 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1369999999999999e-05 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.592 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 61.801366158999997, 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079289789999999992, 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780073+0000", 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780075+0000", 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5129999999999999e-06 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780082+0000", 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9410000000000001e-06 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780087+0000", 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1880000000000001e-06 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781620+0000", 2024-09-10T10:27:48.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001532346 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781698+0000", 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8232999999999994e-05 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781816+0000", 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011825800000000001 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784824+0000", 2024-09-10T10:27:48.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030076349999999998 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787937+0000", 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031134829999999998 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787969+0000", 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1745999999999997e-05 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788001+0000", 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2132000000000003e-05 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 61.801335944000002, 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079888619999999994, 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.598 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780103+0000", 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780106+0000", 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1379999999999999e-06 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780115+0000", 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4299999999999995e-06 2024-09-10T10:27:48.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780122+0000", 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3380000000000001e-06 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782041+0000", 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019193700000000001 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782149+0000", 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107493 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782246+0000", 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999996e-05 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785205+0000", 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002958535 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788041+0000", 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028359650000000002 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788069+0000", 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8123000000000002e-05 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788091+0000", 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2506000000000001e-05 2024-09-10T10:27:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 61.801307403000003, 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080531070000000003, 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:27:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780134+0000", 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0570000000000001e-06 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780143+0000", 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-06 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780156+0000", 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3179e-05 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023278280000000001 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782555+0000", 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0494000000000006e-05 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782675+0000", 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120463 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785209+0000", 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002533455 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788127+0000", 2024-09-10T10:27:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029181260000000001 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788155+0000", 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8495000000000001e-05 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788184+0000", 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8736999999999999e-05 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 61.801274886000002, 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008097524, 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:27:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780164+0000", 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.13e-07 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9780000000000001e-06 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780174+0000", 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7510000000000002e-06 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780179+0000", 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0869999999999997e-06 2024-09-10T10:27:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782890+0000", 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027113509999999999 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782985+0000", 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5152e-05 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783090+0000", 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104947 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785222+0000", 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021320269999999999 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788216+0000", 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029940629999999999 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788242+0000", 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5259999999999999e-05 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788261+0000", 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9395e-05 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 61.801252515000002, 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081619529999999996, 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780189+0000", 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.926e-06 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780197+0000", 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2039999999999997e-06 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780202+0000", 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783326+0000", 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003123855 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783402+0000", 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6548e-05 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783558+0000", 2024-09-10T10:27:48.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015539199999999999 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785225+0000", 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001666989 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788290+0000", 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030647890000000001 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788316+0000", 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6662e-05 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788348+0000", 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1417000000000001e-05 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:3 2.10 2:0f65c9b8:::60a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.621 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 39.709659002999999, 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066829990000000002, 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871781+0000", 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2129999999999998e-06 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871790+0000", 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3380000000000005e-06 2024-09-10T10:27:48.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871805+0000", 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5353999999999999e-05 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871817+0000", 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1399999999999999e-05 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871932+0000", 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114982 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872069+0000", 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013713199999999999 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872208+0000", 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013976900000000001 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874946+0000", 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027378789999999999 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878355+0000", 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034089139999999999 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878413+0000", 2024-09-10T10:27:48.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7540000000000001e-05 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878462+0000", 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9478000000000003e-05 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.634215871000002, 2024-09-10T10:27:48.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067332750000000004, 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947223+0000", 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7299999999999995e-07 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947227+0000", 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.895e-06 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947235+0000", 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947246+0000", 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0738e-05 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948505+0000", 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001259402 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948578+0000", 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2788999999999997e-05 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948718+0000", 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013953699999999999 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953391+0000", 2024-09-10T10:27:48.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046736670000000003 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953881+0000", 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048956900000000003 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953923+0000", 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2153000000000001e-05 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953956+0000", 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2526e-05 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.634157082999998, 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066654160000000004, 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.633 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947282+0000", 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947285+0000", 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7689999999999998e-06 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947292+0000", 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6200000000000001e-06 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947304+0000", 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1702000000000001e-05 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947363+0000", 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9426000000000003e-05 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947485+0000", 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012227599999999999 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947588+0000", 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102641 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950690+0000", 2024-09-10T10:27:48.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003102264 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953856+0000", 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031656200000000001 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953907+0000", 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1430000000000001e-05 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953947+0000", 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9164e-05 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.634119910000003, 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071148679999999999, 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.639 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947319+0000", 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947322+0000", 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.523e-06 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947331+0000", 2024-09-10T10:27:48.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3240000000000003e-06 2024-09-10T10:27:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947336+0000", 2024-09-10T10:27:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7189999999999998e-06 2024-09-10T10:27:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947994+0000", 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065770599999999998 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948080+0000", 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5649000000000006e-05 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111783 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953965+0000", 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057732089999999996 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954343+0000", 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037846199999999998 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954395+0000", 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.126e-05 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954433+0000", 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8693999999999999e-05 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:27:48.645 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.633989714000002, 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067554449999999997, 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6899999999999996e-07 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947454+0000", 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947463+0000", 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0310000000000004e-06 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947477+0000", 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4328000000000001e-05 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3869999999999998e-05 2024-09-10T10:27:48.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947631+0000", 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9874e-05 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947750+0000", 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118483 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953644+0000", 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058946889999999998 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954111+0000", 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00046712099999999998 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954162+0000", 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000003e-05 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954204+0000", 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1718000000000003e-05 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.633602676000002, 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064551169999999998, 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.651 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3e-07 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947841+0000", 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3460000000000004e-06 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947856+0000", 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947864+0000", 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4939999999999997e-06 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947997+0000", 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013305399999999999 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948076+0000", 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9146000000000004e-05 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115586 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953707+0000", 2024-09-10T10:27:48.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055153470000000003 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954215+0000", 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050850400000000001 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954258+0000", 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2932000000000002e-05 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954291+0000", 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2322e-05 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.633563356000003, 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088919089999999999, 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.657 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7199999999999999e-07 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947879+0000", 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8380000000000004e-06 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947887+0000", 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5410000000000003e-06 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947893+0000", 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6930000000000004e-06 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948405+0000", 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051236000000000001 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948502+0000", 2024-09-10T10:27:48.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6916999999999994e-05 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948647+0000", 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000145265 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954479+0000", 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058316970000000003 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956680+0000", 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002201494 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956728+0000", 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8161999999999997e-05 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956767+0000", 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.837e-05 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.662 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.633000695, 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083279530000000008, 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8999999999999995e-07 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948443+0000", 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2189999999999998e-06 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948450+0000", 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8879999999999997e-06 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948457+0000", 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0750000000000003e-06 2024-09-10T10:27:48.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948501+0000", 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948594+0000", 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3009000000000003e-05 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948696+0000", 2024-09-10T10:27:48.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101461 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953756+0000", 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050607129999999997 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956690+0000", 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029334109999999999 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956737+0000", 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7080999999999999e-05 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956766+0000", 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8778000000000001e-05 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.668 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.632461020999997, 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0077788589999999999, 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948986+0000", 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948998+0000", 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.19e-05 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949007+0000", 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.081e-06 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949062+0000", 2024-09-10T10:27:48.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754999999999997e-05 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949172+0000", 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110128 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949282+0000", 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109289 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955941+0000", 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066594990000000001 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956654+0000", 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071279399999999999 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956706+0000", 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2027000000000002e-05 2024-09-10T10:27:48.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956756+0000", 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0012999999999997e-05 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.631922926999998, 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073200349999999999, 2024-09-10T10:27:48.674 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949516+0000", 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999995e-07 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949522+0000", 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3980000000000004e-06 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949534+0000", 2024-09-10T10:27:48.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1562e-05 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949541+0000", 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1380000000000001e-06 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950700+0000", 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011593459999999999 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950786+0000", 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5556999999999994e-05 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950898+0000", 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111743 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954326+0000", 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034280510000000001 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956774+0000", 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024477430000000001 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956815+0000", 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1752999999999998e-05 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956835+0000", 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0032999999999999e-05 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.679 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.631880625999997, 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073303700000000001, 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949571+0000", 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2298e-05 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949579+0000", 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5909999999999996e-06 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949585+0000", 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7370000000000003e-06 2024-09-10T10:27:48.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953802+0000", 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042170929999999999 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953915+0000", 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011256699999999999 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956888+0000", 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029734639999999999 2024-09-10T10:27:48.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:24 2.3 e26/18 2:c216c0d1:::mds2_inotable:head v 26'3, mlcod=26'1)", 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.631062624999998, 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0063254399999999999, 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-10T10:27:48.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950380+0000", 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2590000000000002e-06 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950387+0000", 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950392+0000", 2024-09-10T10:27:48.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950438+0000", 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5592e-05 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950457+0000", 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8569000000000001e-05 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956657+0000", 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062003409999999998 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956701+0000", 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4140000000000001e-05 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:29 2.10 2:0e4da237:::203.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.688 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.659401822, 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064228799999999997, 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922037+0000", 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922040+0000", 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.694e-06 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922054+0000", 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906e-05 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922065+0000", 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922125+0000", 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.982e-05 2024-09-10T10:27:48.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922233+0000", 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108175 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928459+0000", 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062264570000000003 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.640138191999998, 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081214100000000008, 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941301+0000", 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.062e-06 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941306+0000", 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0919999999999997e-06 2024-09-10T10:27:48.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941316+0000", 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5640000000000002e-06 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941323+0000", 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941405+0000", 2024-09-10T10:27:48.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1650000000000006e-05 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941549+0000", 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014447599999999999 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012929 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943549+0000", 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018704049999999999 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949343+0000", 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057943300000000003 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949378+0000", 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4474999999999998e-05 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:48.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:48.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:48.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949421+0000", 2024-09-10T10:27:48.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3955999999999997e-05 2024-09-10T10:27:48.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.698 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:48.698 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:48.698 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:48.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.582+0000 7f6e589bd640 1 -- 172.21.15.120:0/713437177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e5414d760 con 0x7f6e540c6900 2024-09-10T10:27:48.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.583+0000 7f6e42ffd640 1 -- 172.21.15.120:0/713437177 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e4401a070 con 0x7f6e540c6900 2024-09-10T10:27:48.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.583+0000 7f6e42ffd640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6e3406be60 0x7f6e3406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.583+0000 7f6e42ffd640 1 -- 172.21.15.120:0/713437177 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6706+0+0 (secure 0 0 0) 0x7f6e4408a6b0 con 0x7f6e540c6900 2024-09-10T10:27:48.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.583+0000 7f6e5259c640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6e3406be60 0x7f6e3406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.584+0000 7f6e5259c640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6e3406be60 0x7f6e3406e320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e48037000 tx=0x7f6e48032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.588+0000 7f6e42ffd640 1 -- 172.21.15.120:0/713437177 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6e5414d760 con 0x7f6e540c6900 2024-09-10T10:27:48.701 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:27:48.701 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:48.699+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:48.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.754+0000 7f6e589bd640 1 -- 172.21.15.120:0/713437177 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e540c46c0 con 0x7f6e3406be60 2024-09-10T10:27:48.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.762+0000 7f6e42ffd640 1 -- 172.21.15.120:0/713437177 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332720 (secure 0 0 0) 0x7f6e540c46c0 con 0x7f6e3406be60 2024-09-10T10:27:48.766 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:48.768 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:27:48.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 -- 172.21.15.120:0/713437177 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6e3406be60 msgr2=0x7f6e3406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6e3406be60 0x7f6e3406e320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e48037000 tx=0x7f6e48032000 comp rx=0 tx=0).stop 2024-09-10T10:27:48.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 -- 172.21.15.120:0/713437177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e540c6900 msgr2=0x7f6e541497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e540c6900 0x7f6e541497e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c400 tx=0x7f6e4400c8c0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 -- 172.21.15.120:0/713437177 shutdown_connections 2024-09-10T10:27:48.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6e3406be60 0x7f6e3406e320 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414e180 0x7f6e541542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e540c6900 0x7f6e541497e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 --2- 172.21.15.120:0/713437177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540c6520 0x7f6e5414e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 -- 172.21.15.120:0/713437177 >> 172.21.15.120:0/713437177 conn(0x7f6e54068140 msgr2=0x7f6e5414c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:48.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 -- 172.21.15.120:0/713437177 shutdown_connections 2024-09-10T10:27:48.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.765+0000 7f6e589bd640 1 -- 172.21.15.120:0/713437177 wait complete. 2024-09-10T10:27:48.778 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-10T10:27:48.104390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603223,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":376,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219272,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755448,"statfs":{"total":773094113280,"available":772869578752,"internally_reserved":0,"allocated":3121152,"data_stored":2382270,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000948"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.994167+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:27:42.994167+0000","last_peered":"2024-09-10T10:27:42.994167+0000","last_clean":"2024-09-10T10:27:42.994167+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:27:42.994167+0000","last_undegraded":"2024-09-10T10:27:42.994167+0000","last_fullsized":"2024-09-10T10:27:42.994167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.004373+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:27:44.004373+0000","last_peered":"2024-09-10T10:27:44.004373+0000","last_clean":"2024-09-10T10:27:44.004373+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:27:44.004373+0000","last_undegraded":"2024-09-10T10:27:44.004373+0000","last_fullsized":"2024-09-10T10:27:44.004373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003551+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:27:44.003551+0000","last_peered":"2024-09-10T10:27:44.003551+0000","last_clean":"2024-09-10T10:27:44.003551+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:27:44.003551+0000","last_undegraded":"2024-09-10T10:27:44.003551+0000","last_fullsized":"2024-09-10T10:27:44.003551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.004550+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:27:44.004550+0000","last_peered":"2024-09-10T10:27:44.004550+0000","last_clean":"2024-09-10T10:27:44.004550+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:27:44.004550+0000","last_undegraded":"2024-09-10T10:27:44.004550+0000","last_fullsized":"2024-09-10T10:27:44.004550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993883+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:27:42.993883+0000","last_peered":"2024-09-10T10:27:42.993883+0000","last_clean":"2024-09-10T10:27:42.993883+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:27:42.993883+0000","last_undegraded":"2024-09-10T10:27:42.993883+0000","last_fullsized":"2024-09-10T10:27:42.993883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002310+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:27:44.002310+0000","last_peered":"2024-09-10T10:27:44.002310+0000","last_clean":"2024-09-10T10:27:44.002310+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:27:44.002310+0000","last_undegraded":"2024-09-10T10:27:44.002310+0000","last_fullsized":"2024-09-10T10:27:44.002310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"28'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287386+0000","last_change":"2024-09-10T10:26:40.796337+0000","last_active":"2024-09-10T10:27:43.287386+0000","last_peered":"2024-09-10T10:27:43.287386+0000","last_clean":"2024-09-10T10:27:43.287386+0000","last_became_active":"2024-09-10T10:26:40.796029+0000","last_became_peered":"2024-09-10T10:26:40.796029+0000","last_unstale":"2024-09-10T10:27:43.287386+0000","last_undegraded":"2024-09-10T10:27:43.287386+0000","last_fullsized":"2024-09-10T10:27:43.287386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002511+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:27:44.002511+0000","last_peered":"2024-09-10T10:27:44.002511+0000","last_clean":"2024-09-10T10:27:44.002511+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:27:44.002511+0000","last_undegraded":"2024-09-10T10:27:44.002511+0000","last_fullsized":"2024-09-10T10:27:44.002511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002608+0000","last_change":"2024-09-10T10:27:23.955254+0000","last_active":"2024-09-10T10:27:44.002608+0000","last_peered":"2024-09-10T10:27:44.002608+0000","last_clean":"2024-09-10T10:27:44.002608+0000","last_became_active":"2024-09-10T10:27:23.954737+0000","last_became_peered":"2024-09-10T10:27:23.954737+0000","last_unstale":"2024-09-10T10:27:44.002608+0000","last_undegraded":"2024-09-10T10:27:44.002608+0000","last_fullsized":"2024-09-10T10:27:44.002608+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287223+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:27:43.287223+0000","last_peered":"2024-09-10T10:27:43.287223+0000","last_clean":"2024-09-10T10:27:43.287223+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:27:43.287223+0000","last_undegraded":"2024-09-10T10:27:43.287223+0000","last_fullsized":"2024-09-10T10:27:43.287223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999505+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:27:42.999505+0000","last_peered":"2024-09-10T10:27:42.999505+0000","last_clean":"2024-09-10T10:27:42.999505+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:27:42.999505+0000","last_undegraded":"2024-09-10T10:27:42.999505+0000","last_fullsized":"2024-09-10T10:27:42.999505+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002609+0000","last_change":"2024-09-10T10:26:45.816587+0000","last_active":"2024-09-10T10:27:44.002609+0000","last_peered":"2024-09-10T10:27:44.002609+0000","last_clean":"2024-09-10T10:27:44.002609+0000","last_became_active":"2024-09-10T10:26:45.816138+0000","last_became_peered":"2024-09-10T10:26:45.816138+0000","last_unstale":"2024-09-10T10:27:44.002609+0000","last_undegraded":"2024-09-10T10:27:44.002609+0000","last_fullsized":"2024-09-10T10:27:44.002609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999069+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:27:42.999069+0000","last_peered":"2024-09-10T10:27:42.999069+0000","last_clean":"2024-09-10T10:27:42.999069+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:27:42.999069+0000","last_undegraded":"2024-09-10T10:27:42.999069+0000","last_fullsized":"2024-09-10T10:27:42.999069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'4","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287232+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:27:43.287232+0000","last_peered":"2024-09-10T10:27:43.287232+0000","last_clean":"2024-09-10T10:27:43.287232+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:27:43.287232+0000","last_undegraded":"2024-09-10T10:27:43.287232+0000","last_fullsized":"2024-09-10T10:27:43.287232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002924+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:27:44.002924+0000","last_peered":"2024-09-10T10:27:44.002924+0000","last_clean":"2024-09-10T10:27:44.002924+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:27:44.002924+0000","last_undegraded":"2024-09-10T10:27:44.002924+0000","last_fullsized":"2024-09-10T10:27:44.002924+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003902+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:27:44.003902+0000","last_peered":"2024-09-10T10:27:44.003902+0000","last_clean":"2024-09-10T10:27:44.003902+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:27:44.003902+0000","last_undegraded":"2024-09-10T10:27:44.003902+0000","last_fullsized":"2024-09-10T10:27:44.003902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'4","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095239+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:27:44.095239+0000","last_peered":"2024-09-10T10:27:44.095239+0000","last_clean":"2024-09-10T10:27:44.095239+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:27:44.095239+0000","last_undegraded":"2024-09-10T10:27:44.095239+0000","last_fullsized":"2024-09-10T10:27:44.095239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002845+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:27:44.002845+0000","last_peered":"2024-09-10T10:27:44.002845+0000","last_clean":"2024-09-10T10:27:44.002845+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:27:44.002845+0000","last_undegraded":"2024-09-10T10:27:44.002845+0000","last_fullsized":"2024-09-10T10:27:44.002845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.004088+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:27:44.004088+0000","last_peered":"2024-09-10T10:27:44.004088+0000","last_clean":"2024-09-10T10:27:44.004088+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:27:44.004088+0000","last_undegraded":"2024-09-10T10:27:44.004088+0000","last_fullsized":"2024-09-10T10:27:44.004088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095008+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:27:44.095008+0000","last_peered":"2024-09-10T10:27:44.095008+0000","last_clean":"2024-09-10T10:27:44.095008+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:27:44.095008+0000","last_undegraded":"2024-09-10T10:27:44.095008+0000","last_fullsized":"2024-09-10T10:27:44.095008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095512+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:27:44.095512+0000","last_peered":"2024-09-10T10:27:44.095512+0000","last_clean":"2024-09-10T10:27:44.095512+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:27:44.095512+0000","last_undegraded":"2024-09-10T10:27:44.095512+0000","last_fullsized":"2024-09-10T10:27:44.095512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003360+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:27:44.003360+0000","last_peered":"2024-09-10T10:27:44.003360+0000","last_clean":"2024-09-10T10:27:44.003360+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:27:44.003360+0000","last_undegraded":"2024-09-10T10:27:44.003360+0000","last_fullsized":"2024-09-10T10:27:44.003360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.286932+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:27:43.286932+0000","last_peered":"2024-09-10T10:27:43.286932+0000","last_clean":"2024-09-10T10:27:43.286932+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:27:43.286932+0000","last_undegraded":"2024-09-10T10:27:43.286932+0000","last_fullsized":"2024-09-10T10:27:43.286932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003166+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:27:44.003166+0000","last_peered":"2024-09-10T10:27:44.003166+0000","last_clean":"2024-09-10T10:27:44.003166+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:27:44.003166+0000","last_undegraded":"2024-09-10T10:27:44.003166+0000","last_fullsized":"2024-09-10T10:27:44.003166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.994036+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:27:42.994036+0000","last_peered":"2024-09-10T10:27:42.994036+0000","last_clean":"2024-09-10T10:27:42.994036+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:27:42.994036+0000","last_undegraded":"2024-09-10T10:27:42.994036+0000","last_fullsized":"2024-09-10T10:27:42.994036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'7","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999716+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:27:42.999716+0000","last_peered":"2024-09-10T10:27:42.999716+0000","last_clean":"2024-09-10T10:27:42.999716+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:27:42.999716+0000","last_undegraded":"2024-09-10T10:27:42.999716+0000","last_fullsized":"2024-09-10T10:27:42.999716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002839+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:27:44.002839+0000","last_peered":"2024-09-10T10:27:44.002839+0000","last_clean":"2024-09-10T10:27:44.002839+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:27:44.002839+0000","last_undegraded":"2024-09-10T10:27:44.002839+0000","last_fullsized":"2024-09-10T10:27:44.002839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.004567+0000","last_change":"2024-09-10T10:26:41.795085+0000","last_active":"2024-09-10T10:27:44.004567+0000","last_peered":"2024-09-10T10:27:44.004567+0000","last_clean":"2024-09-10T10:27:44.004567+0000","last_became_active":"2024-09-10T10:26:41.794819+0000","last_became_peered":"2024-09-10T10:26:41.794819+0000","last_unstale":"2024-09-10T10:27:44.004567+0000","last_undegraded":"2024-09-10T10:27:44.004567+0000","last_fullsized":"2024-09-10T10:27:44.004567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"28'8","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095539+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:27:44.095539+0000","last_peered":"2024-09-10T10:27:44.095539+0000","last_clean":"2024-09-10T10:27:44.095539+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:27:44.095539+0000","last_undegraded":"2024-09-10T10:27:44.095539+0000","last_fullsized":"2024-09-10T10:27:44.095539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003538+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:27:44.003538+0000","last_peered":"2024-09-10T10:27:44.003538+0000","last_clean":"2024-09-10T10:27:44.003538+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:27:44.003538+0000","last_undegraded":"2024-09-10T10:27:44.003538+0000","last_fullsized":"2024-09-10T10:27:44.003538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095367+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:27:44.095367+0000","last_peered":"2024-09-10T10:27:44.095367+0000","last_clean":"2024-09-10T10:27:44.095367+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:27:44.095367+0000","last_undegraded":"2024-09-10T10:27:44.095367+0000","last_fullsized":"2024-09-10T10:27:44.095367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287477+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:27:43.287477+0000","last_peered":"2024-09-10T10:27:43.287477+0000","last_clean":"2024-09-10T10:27:43.287477+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:27:43.287477+0000","last_undegraded":"2024-09-10T10:27:43.287477+0000","last_fullsized":"2024-09-10T10:27:43.287477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993607+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:27:42.993607+0000","last_peered":"2024-09-10T10:27:42.993607+0000","last_clean":"2024-09-10T10:27:42.993607+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:27:42.993607+0000","last_undegraded":"2024-09-10T10:27:42.993607+0000","last_fullsized":"2024-09-10T10:27:42.993607+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002574+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:27:44.002574+0000","last_peered":"2024-09-10T10:27:44.002574+0000","last_clean":"2024-09-10T10:27:44.002574+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:27:44.002574+0000","last_undegraded":"2024-09-10T10:27:44.002574+0000","last_fullsized":"2024-09-10T10:27:44.002574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":37,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095447+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:27:44.095447+0000","last_peered":"2024-09-10T10:27:44.095447+0000","last_clean":"2024-09-10T10:27:44.095447+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:27:44.095447+0000","last_undegraded":"2024-09-10T10:27:44.095447+0000","last_fullsized":"2024-09-10T10:27:44.095447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.094960+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:27:44.094960+0000","last_peered":"2024-09-10T10:27:44.094960+0000","last_clean":"2024-09-10T10:27:44.094960+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:27:44.094960+0000","last_undegraded":"2024-09-10T10:27:44.094960+0000","last_fullsized":"2024-09-10T10:27:44.094960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003031+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:27:44.003031+0000","last_peered":"2024-09-10T10:27:44.003031+0000","last_clean":"2024-09-10T10:27:44.003031+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:27:44.003031+0000","last_undegraded":"2024-09-10T10:27:44.003031+0000","last_fullsized":"2024-09-10T10:27:44.003031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002535+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:27:44.002535+0000","last_peered":"2024-09-10T10:27:44.002535+0000","last_clean":"2024-09-10T10:27:44.002535+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:27:44.002535+0000","last_undegraded":"2024-09-10T10:27:44.002535+0000","last_fullsized":"2024-09-10T10:27:44.002535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095260+0000","last_change":"2024-09-10T10:26:45.814008+0000","last_active":"2024-09-10T10:27:44.095260+0000","last_peered":"2024-09-10T10:27:44.095260+0000","last_clean":"2024-09-10T10:27:44.095260+0000","last_became_active":"2024-09-10T10:26:45.813627+0000","last_became_peered":"2024-09-10T10:26:45.813627+0000","last_unstale":"2024-09-10T10:27:44.095260+0000","last_undegraded":"2024-09-10T10:27:44.095260+0000","last_fullsized":"2024-09-10T10:27:44.095260+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095141+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:27:44.095141+0000","last_peered":"2024-09-10T10:27:44.095141+0000","last_clean":"2024-09-10T10:27:44.095141+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:27:44.095141+0000","last_undegraded":"2024-09-10T10:27:44.095141+0000","last_fullsized":"2024-09-10T10:27:44.095141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"28'5","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999223+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:27:42.999223+0000","last_peered":"2024-09-10T10:27:42.999223+0000","last_clean":"2024-09-10T10:27:42.999223+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:27:42.999223+0000","last_undegraded":"2024-09-10T10:27:42.999223+0000","last_fullsized":"2024-09-10T10:27:42.999223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095517+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:27:44.095517+0000","last_peered":"2024-09-10T10:27:44.095517+0000","last_clean":"2024-09-10T10:27:44.095517+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:27:44.095517+0000","last_undegraded":"2024-09-10T10:27:44.095517+0000","last_fullsized":"2024-09-10T10:27:44.095517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993300+0000","last_change":"2024-09-10T10:26:41.799224+0000","last_active":"2024-09-10T10:27:42.993300+0000","last_peered":"2024-09-10T10:27:42.993300+0000","last_clean":"2024-09-10T10:27:42.993300+0000","last_became_active":"2024-09-10T10:26:41.798948+0000","last_became_peered":"2024-09-10T10:26:41.798948+0000","last_unstale":"2024-09-10T10:27:42.993300+0000","last_undegraded":"2024-09-10T10:27:42.993300+0000","last_fullsized":"2024-09-10T10:27:42.993300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.286949+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:27:43.286949+0000","last_peered":"2024-09-10T10:27:43.286949+0000","last_clean":"2024-09-10T10:27:43.286949+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:27:43.286949+0000","last_undegraded":"2024-09-10T10:27:43.286949+0000","last_fullsized":"2024-09-10T10:27:43.286949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993940+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:27:42.993940+0000","last_peered":"2024-09-10T10:27:42.993940+0000","last_clean":"2024-09-10T10:27:42.993940+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:27:42.993940+0000","last_undegraded":"2024-09-10T10:27:42.993940+0000","last_fullsized":"2024-09-10T10:27:42.993940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003953+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:27:44.003953+0000","last_peered":"2024-09-10T10:27:44.003953+0000","last_clean":"2024-09-10T10:27:44.003953+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:27:44.003953+0000","last_undegraded":"2024-09-10T10:27:44.003953+0000","last_fullsized":"2024-09-10T10:27:44.003953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002408+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:27:44.002408+0000","last_peered":"2024-09-10T10:27:44.002408+0000","last_clean":"2024-09-10T10:27:44.002408+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:27:44.002408+0000","last_undegraded":"2024-09-10T10:27:44.002408+0000","last_fullsized":"2024-09-10T10:27:44.002408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287157+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:27:43.287157+0000","last_peered":"2024-09-10T10:27:43.287157+0000","last_clean":"2024-09-10T10:27:43.287157+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:27:43.287157+0000","last_undegraded":"2024-09-10T10:27:43.287157+0000","last_fullsized":"2024-09-10T10:27:43.287157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999209+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:27:42.999209+0000","last_peered":"2024-09-10T10:27:42.999209+0000","last_clean":"2024-09-10T10:27:42.999209+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:27:42.999209+0000","last_undegraded":"2024-09-10T10:27:42.999209+0000","last_fullsized":"2024-09-10T10:27:42.999209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287278+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:27:43.287278+0000","last_peered":"2024-09-10T10:27:43.287278+0000","last_clean":"2024-09-10T10:27:43.287278+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:27:43.287278+0000","last_undegraded":"2024-09-10T10:27:43.287278+0000","last_fullsized":"2024-09-10T10:27:43.287278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999583+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:27:42.999583+0000","last_peered":"2024-09-10T10:27:42.999583+0000","last_clean":"2024-09-10T10:27:42.999583+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:27:42.999583+0000","last_undegraded":"2024-09-10T10:27:42.999583+0000","last_fullsized":"2024-09-10T10:27:42.999583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999054+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:27:42.999054+0000","last_peered":"2024-09-10T10:27:42.999054+0000","last_clean":"2024-09-10T10:27:42.999054+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:27:42.999054+0000","last_undegraded":"2024-09-10T10:27:42.999054+0000","last_fullsized":"2024-09-10T10:27:42.999054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002606+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:27:44.002606+0000","last_peered":"2024-09-10T10:27:44.002606+0000","last_clean":"2024-09-10T10:27:44.002606+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:27:44.002606+0000","last_undegraded":"2024-09-10T10:27:44.002606+0000","last_fullsized":"2024-09-10T10:27:44.002606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095276+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:27:44.095276+0000","last_peered":"2024-09-10T10:27:44.095276+0000","last_clean":"2024-09-10T10:27:44.095276+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:27:44.095276+0000","last_undegraded":"2024-09-10T10:27:44.095276+0000","last_fullsized":"2024-09-10T10:27:44.095276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993720+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:27:42.993720+0000","last_peered":"2024-09-10T10:27:42.993720+0000","last_clean":"2024-09-10T10:27:42.993720+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:27:42.993720+0000","last_undegraded":"2024-09-10T10:27:42.993720+0000","last_fullsized":"2024-09-10T10:27:42.993720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287623+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:27:43.287623+0000","last_peered":"2024-09-10T10:27:43.287623+0000","last_clean":"2024-09-10T10:27:43.287623+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:27:43.287623+0000","last_undegraded":"2024-09-10T10:27:43.287623+0000","last_fullsized":"2024-09-10T10:27:43.287623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999872+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:27:42.999872+0000","last_peered":"2024-09-10T10:27:42.999872+0000","last_clean":"2024-09-10T10:27:42.999872+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:27:42.999872+0000","last_undegraded":"2024-09-10T10:27:42.999872+0000","last_fullsized":"2024-09-10T10:27:42.999872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.001976+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:27:44.001976+0000","last_peered":"2024-09-10T10:27:44.001976+0000","last_clean":"2024-09-10T10:27:44.001976+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:27:44.001976+0000","last_undegraded":"2024-09-10T10:27:44.001976+0000","last_fullsized":"2024-09-10T10:27:44.001976+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287455+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:27:43.287455+0000","last_peered":"2024-09-10T10:27:43.287455+0000","last_clean":"2024-09-10T10:27:43.287455+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:27:43.287455+0000","last_undegraded":"2024-09-10T10:27:43.287455+0000","last_fullsized":"2024-09-10T10:27:43.287455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003328+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:27:44.003328+0000","last_peered":"2024-09-10T10:27:44.003328+0000","last_clean":"2024-09-10T10:27:44.003328+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:27:44.003328+0000","last_undegraded":"2024-09-10T10:27:44.003328+0000","last_fullsized":"2024-09-10T10:27:44.003328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002715+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:27:44.002715+0000","last_peered":"2024-09-10T10:27:44.002715+0000","last_clean":"2024-09-10T10:27:44.002715+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:27:44.002715+0000","last_undegraded":"2024-09-10T10:27:44.002715+0000","last_fullsized":"2024-09-10T10:27:44.002715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287118+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:27:43.287118+0000","last_peered":"2024-09-10T10:27:43.287118+0000","last_clean":"2024-09-10T10:27:43.287118+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:27:43.287118+0000","last_undegraded":"2024-09-10T10:27:43.287118+0000","last_fullsized":"2024-09-10T10:27:43.287118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"28'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999739+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:27:42.999739+0000","last_peered":"2024-09-10T10:27:42.999739+0000","last_clean":"2024-09-10T10:27:42.999739+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:27:42.999739+0000","last_undegraded":"2024-09-10T10:27:42.999739+0000","last_fullsized":"2024-09-10T10:27:42.999739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993865+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:27:42.993865+0000","last_peered":"2024-09-10T10:27:42.993865+0000","last_clean":"2024-09-10T10:27:42.993865+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:27:42.993865+0000","last_undegraded":"2024-09-10T10:27:42.993865+0000","last_fullsized":"2024-09-10T10:27:42.993865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999592+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:27:42.999592+0000","last_peered":"2024-09-10T10:27:42.999592+0000","last_clean":"2024-09-10T10:27:42.999592+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:27:42.999592+0000","last_undegraded":"2024-09-10T10:27:42.999592+0000","last_fullsized":"2024-09-10T10:27:42.999592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999115+0000","last_change":"2024-09-10T10:26:40.794739+0000","last_active":"2024-09-10T10:27:42.999115+0000","last_peered":"2024-09-10T10:27:42.999115+0000","last_clean":"2024-09-10T10:27:42.999115+0000","last_became_active":"2024-09-10T10:26:40.794430+0000","last_became_peered":"2024-09-10T10:26:40.794430+0000","last_unstale":"2024-09-10T10:27:42.999115+0000","last_undegraded":"2024-09-10T10:27:42.999115+0000","last_fullsized":"2024-09-10T10:27:42.999115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.004388+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:27:44.004388+0000","last_peered":"2024-09-10T10:27:44.004388+0000","last_clean":"2024-09-10T10:27:44.004388+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:27:44.004388+0000","last_undegraded":"2024-09-10T10:27:44.004388+0000","last_fullsized":"2024-09-10T10:27:44.004388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287190+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:27:43.287190+0000","last_peered":"2024-09-10T10:27:43.287190+0000","last_clean":"2024-09-10T10:27:43.287190+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:27:43.287190+0000","last_undegraded":"2024-09-10T10:27:43.287190+0000","last_fullsized":"2024-09-10T10:27:43.287190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"28'5","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002432+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:27:44.002432+0000","last_peered":"2024-09-10T10:27:44.002432+0000","last_clean":"2024-09-10T10:27:44.002432+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:27:44.002432+0000","last_undegraded":"2024-09-10T10:27:44.002432+0000","last_fullsized":"2024-09-10T10:27:44.002432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003961+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:27:44.003961+0000","last_peered":"2024-09-10T10:27:44.003961+0000","last_clean":"2024-09-10T10:27:44.003961+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:27:44.003961+0000","last_undegraded":"2024-09-10T10:27:44.003961+0000","last_fullsized":"2024-09-10T10:27:44.003961+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993526+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:27:42.993526+0000","last_peered":"2024-09-10T10:27:42.993526+0000","last_clean":"2024-09-10T10:27:42.993526+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:27:42.993526+0000","last_undegraded":"2024-09-10T10:27:42.993526+0000","last_fullsized":"2024-09-10T10:27:42.993526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.994046+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:27:42.994046+0000","last_peered":"2024-09-10T10:27:42.994046+0000","last_clean":"2024-09-10T10:27:42.994046+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:27:42.994046+0000","last_undegraded":"2024-09-10T10:27:42.994046+0000","last_fullsized":"2024-09-10T10:27:42.994046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002484+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:27:44.002484+0000","last_peered":"2024-09-10T10:27:44.002484+0000","last_clean":"2024-09-10T10:27:44.002484+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:27:44.002484+0000","last_undegraded":"2024-09-10T10:27:44.002484+0000","last_fullsized":"2024-09-10T10:27:44.002484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287271+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:27:43.287271+0000","last_peered":"2024-09-10T10:27:43.287271+0000","last_clean":"2024-09-10T10:27:43.287271+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:27:43.287271+0000","last_undegraded":"2024-09-10T10:27:43.287271+0000","last_fullsized":"2024-09-10T10:27:43.287271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"28'16","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999329+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:27:42.999329+0000","last_peered":"2024-09-10T10:27:42.999329+0000","last_clean":"2024-09-10T10:27:42.999329+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:27:42.999329+0000","last_undegraded":"2024-09-10T10:27:42.999329+0000","last_fullsized":"2024-09-10T10:27:42.999329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6177,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095217+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:27:44.095217+0000","last_peered":"2024-09-10T10:27:44.095217+0000","last_clean":"2024-09-10T10:27:44.095217+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:27:44.095217+0000","last_undegraded":"2024-09-10T10:27:44.095217+0000","last_fullsized":"2024-09-10T10:27:44.095217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003051+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:27:44.003051+0000","last_peered":"2024-09-10T10:27:44.003051+0000","last_clean":"2024-09-10T10:27:44.003051+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:27:44.003051+0000","last_undegraded":"2024-09-10T10:27:44.003051+0000","last_fullsized":"2024-09-10T10:27:44.003051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'5","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095280+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:27:44.095280+0000","last_peered":"2024-09-10T10:27:44.095280+0000","last_clean":"2024-09-10T10:27:44.095280+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:27:44.095280+0000","last_undegraded":"2024-09-10T10:27:44.095280+0000","last_fullsized":"2024-09-10T10:27:44.095280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002789+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:27:44.002789+0000","last_peered":"2024-09-10T10:27:44.002789+0000","last_clean":"2024-09-10T10:27:44.002789+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:27:44.002789+0000","last_undegraded":"2024-09-10T10:27:44.002789+0000","last_fullsized":"2024-09-10T10:27:44.002789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.994073+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:27:42.994073+0000","last_peered":"2024-09-10T10:27:42.994073+0000","last_clean":"2024-09-10T10:27:42.994073+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:27:42.994073+0000","last_undegraded":"2024-09-10T10:27:42.994073+0000","last_fullsized":"2024-09-10T10:27:42.994073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002490+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:27:44.002490+0000","last_peered":"2024-09-10T10:27:44.002490+0000","last_clean":"2024-09-10T10:27:44.002490+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:27:44.002490+0000","last_undegraded":"2024-09-10T10:27:44.002490+0000","last_fullsized":"2024-09-10T10:27:44.002490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":37,"reported_epoch":32,"state":"activating","last_fresh":"2024-09-10T10:27:44.004506+0000","last_change":"2024-09-10T10:27:44.003776+0000","last_active":"2024-09-10T10:27:42.997367+0000","last_peered":"2024-09-10T10:27:40.485796+0000","last_clean":"2024-09-10T10:27:40.485796+0000","last_became_active":"2024-09-10T10:27:23.957798+0000","last_became_peered":"2024-09-10T10:27:23.957798+0000","last_unstale":"2024-09-10T10:27:44.004506+0000","last_undegraded":"2024-09-10T10:27:44.004506+0000","last_fullsized":"2024-09-10T10:27:44.004506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":["3(0)","1(1)","2(1)","4(3)","6(2)"],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095289+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:27:44.095289+0000","last_peered":"2024-09-10T10:27:44.095289+0000","last_clean":"2024-09-10T10:27:44.095289+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:27:44.095289+0000","last_undegraded":"2024-09-10T10:27:44.095289+0000","last_fullsized":"2024-09-10T10:27:44.095289+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"28'5","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999613+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:27:42.999613+0000","last_peered":"2024-09-10T10:27:42.999613+0000","last_clean":"2024-09-10T10:27:42.999613+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:27:42.999613+0000","last_undegraded":"2024-09-10T10:27:42.999613+0000","last_fullsized":"2024-09-10T10:27:42.999613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003595+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:27:44.003595+0000","last_peered":"2024-09-10T10:27:44.003595+0000","last_clean":"2024-09-10T10:27:44.003595+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:27:44.003595+0000","last_undegraded":"2024-09-10T10:27:44.003595+0000","last_fullsized":"2024-09-10T10:27:44.003595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993830+0000","last_change":"2024-09-10T10:26:46.193656+0000","last_active":"2024-09-10T10:27:42.993830+0000","last_peered":"2024-09-10T10:27:42.993830+0000","last_clean":"2024-09-10T10:27:42.993830+0000","last_became_active":"2024-09-10T10:26:46.193109+0000","last_became_peered":"2024-09-10T10:26:46.193109+0000","last_unstale":"2024-09-10T10:27:42.993830+0000","last_undegraded":"2024-09-10T10:27:42.993830+0000","last_fullsized":"2024-09-10T10:27:42.993830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.094897+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:27:44.094897+0000","last_peered":"2024-09-10T10:27:44.094897+0000","last_clean":"2024-09-10T10:27:44.094897+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:27:44.094897+0000","last_undegraded":"2024-09-10T10:27:44.094897+0000","last_fullsized":"2024-09-10T10:27:44.094897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002159+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:27:44.002159+0000","last_peered":"2024-09-10T10:27:44.002159+0000","last_clean":"2024-09-10T10:27:44.002159+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:27:44.002159+0000","last_undegraded":"2024-09-10T10:27:44.002159+0000","last_fullsized":"2024-09-10T10:27:44.002159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999419+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:27:42.999419+0000","last_peered":"2024-09-10T10:27:42.999419+0000","last_clean":"2024-09-10T10:27:42.999419+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:27:42.999419+0000","last_undegraded":"2024-09-10T10:27:42.999419+0000","last_fullsized":"2024-09-10T10:27:42.999419+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003995+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:27:44.003995+0000","last_peered":"2024-09-10T10:27:44.003995+0000","last_clean":"2024-09-10T10:27:44.003995+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:27:44.003995+0000","last_undegraded":"2024-09-10T10:27:44.003995+0000","last_fullsized":"2024-09-10T10:27:44.003995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.004084+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:27:44.004084+0000","last_peered":"2024-09-10T10:27:44.004084+0000","last_clean":"2024-09-10T10:27:44.004084+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:27:44.004084+0000","last_undegraded":"2024-09-10T10:27:44.004084+0000","last_fullsized":"2024-09-10T10:27:44.004084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999350+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:27:42.999350+0000","last_peered":"2024-09-10T10:27:42.999350+0000","last_clean":"2024-09-10T10:27:42.999350+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:27:42.999350+0000","last_undegraded":"2024-09-10T10:27:42.999350+0000","last_fullsized":"2024-09-10T10:27:42.999350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993856+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:27:42.993856+0000","last_peered":"2024-09-10T10:27:42.993856+0000","last_clean":"2024-09-10T10:27:42.993856+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:27:42.993856+0000","last_undegraded":"2024-09-10T10:27:42.993856+0000","last_fullsized":"2024-09-10T10:27:42.993856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999386+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:27:42.999386+0000","last_peered":"2024-09-10T10:27:42.999386+0000","last_clean":"2024-09-10T10:27:42.999386+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:27:42.999386+0000","last_undegraded":"2024-09-10T10:27:42.999386+0000","last_fullsized":"2024-09-10T10:27:42.999386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'6","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.004010+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:27:44.004010+0000","last_peered":"2024-09-10T10:27:44.004010+0000","last_clean":"2024-09-10T10:27:44.004010+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:27:44.004010+0000","last_undegraded":"2024-09-10T10:27:44.004010+0000","last_fullsized":"2024-09-10T10:27:44.004010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002784+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:27:44.002784+0000","last_peered":"2024-09-10T10:27:44.002784+0000","last_clean":"2024-09-10T10:27:44.002784+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:27:44.002784+0000","last_undegraded":"2024-09-10T10:27:44.002784+0000","last_fullsized":"2024-09-10T10:27:44.002784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003961+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:27:44.003961+0000","last_peered":"2024-09-10T10:27:44.003961+0000","last_clean":"2024-09-10T10:27:44.003961+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:27:44.003961+0000","last_undegraded":"2024-09-10T10:27:44.003961+0000","last_fullsized":"2024-09-10T10:27:44.003961+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999501+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:27:42.999501+0000","last_peered":"2024-09-10T10:27:42.999501+0000","last_clean":"2024-09-10T10:27:42.999501+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:27:42.999501+0000","last_undegraded":"2024-09-10T10:27:42.999501+0000","last_fullsized":"2024-09-10T10:27:42.999501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999199+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:27:42.999199+0000","last_peered":"2024-09-10T10:27:42.999199+0000","last_clean":"2024-09-10T10:27:42.999199+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:27:42.999199+0000","last_undegraded":"2024-09-10T10:27:42.999199+0000","last_fullsized":"2024-09-10T10:27:42.999199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002342+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:27:44.002342+0000","last_peered":"2024-09-10T10:27:44.002342+0000","last_clean":"2024-09-10T10:27:44.002342+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:27:44.002342+0000","last_undegraded":"2024-09-10T10:27:44.002342+0000","last_fullsized":"2024-09-10T10:27:44.002342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.994139+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:27:42.994139+0000","last_peered":"2024-09-10T10:27:42.994139+0000","last_clean":"2024-09-10T10:27:42.994139+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:27:42.994139+0000","last_undegraded":"2024-09-10T10:27:42.994139+0000","last_fullsized":"2024-09-10T10:27:42.994139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.004266+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:27:44.004266+0000","last_peered":"2024-09-10T10:27:44.004266+0000","last_clean":"2024-09-10T10:27:44.004266+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:27:44.004266+0000","last_undegraded":"2024-09-10T10:27:44.004266+0000","last_fullsized":"2024-09-10T10:27:44.004266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095089+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:27:44.095089+0000","last_peered":"2024-09-10T10:27:44.095089+0000","last_clean":"2024-09-10T10:27:44.095089+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:27:44.095089+0000","last_undegraded":"2024-09-10T10:27:44.095089+0000","last_fullsized":"2024-09-10T10:27:44.095089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999574+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:27:42.999574+0000","last_peered":"2024-09-10T10:27:42.999574+0000","last_clean":"2024-09-10T10:27:42.999574+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:27:42.999574+0000","last_undegraded":"2024-09-10T10:27:42.999574+0000","last_fullsized":"2024-09-10T10:27:42.999574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.004480+0000","last_change":"2024-09-10T10:26:41.802903+0000","last_active":"2024-09-10T10:27:44.004480+0000","last_peered":"2024-09-10T10:27:44.004480+0000","last_clean":"2024-09-10T10:27:44.004480+0000","last_became_active":"2024-09-10T10:26:41.802586+0000","last_became_peered":"2024-09-10T10:26:41.802586+0000","last_unstale":"2024-09-10T10:27:44.004480+0000","last_undegraded":"2024-09-10T10:27:44.004480+0000","last_fullsized":"2024-09-10T10:27:44.004480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002102+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:27:44.002102+0000","last_peered":"2024-09-10T10:27:44.002102+0000","last_clean":"2024-09-10T10:27:44.002102+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:27:44.002102+0000","last_undegraded":"2024-09-10T10:27:44.002102+0000","last_fullsized":"2024-09-10T10:27:44.002102+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003821+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:27:44.003821+0000","last_peered":"2024-09-10T10:27:44.003821+0000","last_clean":"2024-09-10T10:27:44.003821+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:27:44.003821+0000","last_undegraded":"2024-09-10T10:27:44.003821+0000","last_fullsized":"2024-09-10T10:27:44.003821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002252+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:27:44.002252+0000","last_peered":"2024-09-10T10:27:44.002252+0000","last_clean":"2024-09-10T10:27:44.002252+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:27:44.002252+0000","last_undegraded":"2024-09-10T10:27:44.002252+0000","last_fullsized":"2024-09-10T10:27:44.002252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993718+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:27:42.993718+0000","last_peered":"2024-09-10T10:27:42.993718+0000","last_clean":"2024-09-10T10:27:42.993718+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:27:42.993718+0000","last_undegraded":"2024-09-10T10:27:42.993718+0000","last_fullsized":"2024-09-10T10:27:42.993718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999235+0000","last_change":"2024-09-10T10:26:41.787435+0000","last_active":"2024-09-10T10:27:42.999235+0000","last_peered":"2024-09-10T10:27:42.999235+0000","last_clean":"2024-09-10T10:27:42.999235+0000","last_became_active":"2024-09-10T10:26:41.787083+0000","last_became_peered":"2024-09-10T10:26:41.787083+0000","last_unstale":"2024-09-10T10:27:42.999235+0000","last_undegraded":"2024-09-10T10:27:42.999235+0000","last_fullsized":"2024-09-10T10:27:42.999235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287436+0000","last_change":"2024-09-10T10:26:41.792587+0000","last_active":"2024-09-10T10:27:43.287436+0000","last_peered":"2024-09-10T10:27:43.287436+0000","last_clean":"2024-09-10T10:27:43.287436+0000","last_became_active":"2024-09-10T10:26:41.792320+0000","last_became_peered":"2024-09-10T10:26:41.792320+0000","last_unstale":"2024-09-10T10:27:43.287436+0000","last_undegraded":"2024-09-10T10:27:43.287436+0000","last_fullsized":"2024-09-10T10:27:43.287436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999867+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:27:42.999867+0000","last_peered":"2024-09-10T10:27:42.999867+0000","last_clean":"2024-09-10T10:27:42.999867+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:27:42.999867+0000","last_undegraded":"2024-09-10T10:27:42.999867+0000","last_fullsized":"2024-09-10T10:27:42.999867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002978+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:27:44.002978+0000","last_peered":"2024-09-10T10:27:44.002978+0000","last_clean":"2024-09-10T10:27:44.002978+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:27:44.002978+0000","last_undegraded":"2024-09-10T10:27:44.002978+0000","last_fullsized":"2024-09-10T10:27:44.002978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002062+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:27:44.002062+0000","last_peered":"2024-09-10T10:27:44.002062+0000","last_clean":"2024-09-10T10:27:44.002062+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:27:44.002062+0000","last_undegraded":"2024-09-10T10:27:44.002062+0000","last_fullsized":"2024-09-10T10:27:44.002062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.094970+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:27:44.094970+0000","last_peered":"2024-09-10T10:27:44.094970+0000","last_clean":"2024-09-10T10:27:44.094970+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:27:44.094970+0000","last_undegraded":"2024-09-10T10:27:44.094970+0000","last_fullsized":"2024-09-10T10:27:44.094970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993525+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:27:42.993525+0000","last_peered":"2024-09-10T10:27:42.993525+0000","last_clean":"2024-09-10T10:27:42.993525+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:27:42.993525+0000","last_undegraded":"2024-09-10T10:27:42.993525+0000","last_fullsized":"2024-09-10T10:27:42.993525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003806+0000","last_change":"2024-09-10T10:26:41.802117+0000","last_active":"2024-09-10T10:27:44.003806+0000","last_peered":"2024-09-10T10:27:44.003806+0000","last_clean":"2024-09-10T10:27:44.003806+0000","last_became_active":"2024-09-10T10:26:41.801798+0000","last_became_peered":"2024-09-10T10:26:41.801798+0000","last_unstale":"2024-09-10T10:27:44.003806+0000","last_undegraded":"2024-09-10T10:27:44.003806+0000","last_fullsized":"2024-09-10T10:27:44.003806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003002+0000","last_change":"2024-09-10T10:26:41.791156+0000","last_active":"2024-09-10T10:27:44.003002+0000","last_peered":"2024-09-10T10:27:44.003002+0000","last_clean":"2024-09-10T10:27:44.003002+0000","last_became_active":"2024-09-10T10:26:41.790893+0000","last_became_peered":"2024-09-10T10:26:41.790893+0000","last_unstale":"2024-09-10T10:27:44.003002+0000","last_undegraded":"2024-09-10T10:27:44.003002+0000","last_fullsized":"2024-09-10T10:27:44.003002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:46.923082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999340+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:27:42.999340+0000","last_peered":"2024-09-10T10:27:42.999340+0000","last_clean":"2024-09-10T10:27:42.999340+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:27:42.999340+0000","last_undegraded":"2024-09-10T10:27:42.999340+0000","last_fullsized":"2024-09-10T10:27:42.999340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.999524+0000","last_change":"2024-09-10T10:27:23.958305+0000","last_active":"2024-09-10T10:27:42.999524+0000","last_peered":"2024-09-10T10:27:42.999524+0000","last_clean":"2024-09-10T10:27:42.999524+0000","last_became_active":"2024-09-10T10:27:23.957954+0000","last_became_peered":"2024-09-10T10:27:23.957954+0000","last_unstale":"2024-09-10T10:27:42.999524+0000","last_undegraded":"2024-09-10T10:27:42.999524+0000","last_fullsized":"2024-09-10T10:27:42.999524+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:32.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287567+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:27:43.287567+0000","last_peered":"2024-09-10T10:27:43.287567+0000","last_clean":"2024-09-10T10:27:43.287567+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:27:43.287567+0000","last_undegraded":"2024-09-10T10:27:43.287567+0000","last_fullsized":"2024-09-10T10:27:43.287567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.003631+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:27:44.003631+0000","last_peered":"2024-09-10T10:27:44.003631+0000","last_clean":"2024-09-10T10:27:44.003631+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:27:44.003631+0000","last_undegraded":"2024-09-10T10:27:44.003631+0000","last_fullsized":"2024-09-10T10:27:44.003631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287077+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:27:43.287077+0000","last_peered":"2024-09-10T10:27:43.287077+0000","last_clean":"2024-09-10T10:27:43.287077+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:27:43.287077+0000","last_undegraded":"2024-09-10T10:27:43.287077+0000","last_fullsized":"2024-09-10T10:27:43.287077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002200+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:27:44.002200+0000","last_peered":"2024-09-10T10:27:44.002200+0000","last_clean":"2024-09-10T10:27:44.002200+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:27:44.002200+0000","last_undegraded":"2024-09-10T10:27:44.002200+0000","last_fullsized":"2024-09-10T10:27:44.002200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287086+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:27:43.287086+0000","last_peered":"2024-09-10T10:27:43.287086+0000","last_clean":"2024-09-10T10:27:43.287086+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:27:43.287086+0000","last_undegraded":"2024-09-10T10:27:43.287086+0000","last_fullsized":"2024-09-10T10:27:43.287086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.002420+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:27:44.002420+0000","last_peered":"2024-09-10T10:27:44.002420+0000","last_clean":"2024-09-10T10:27:44.002420+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:27:44.002420+0000","last_undegraded":"2024-09-10T10:27:44.002420+0000","last_fullsized":"2024-09-10T10:27:44.002420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:42.993763+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:27:42.993763+0000","last_peered":"2024-09-10T10:27:42.993763+0000","last_clean":"2024-09-10T10:27:42.993763+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:27:42.993763+0000","last_undegraded":"2024-09-10T10:27:42.993763+0000","last_fullsized":"2024-09-10T10:27:42.993763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T10:27:44.095478+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:27:44.095478+0000","last_peered":"2024-09-10T10:27:44.095478+0000","last_clean":"2024-09-10T10:27:44.095478+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:27:44.095478+0000","last_undegraded":"2024-09-10T10:27:44.095478+0000","last_fullsized":"2024-09-10T10:27:44.095478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T10:27:43.287534+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:27:43.287534+0000","last_peered":"2024-09-10T10:27:43.287534+0000","last_clean":"2024-09-10T10:27:43.287534+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:27:43.287534+0000","last_undegraded":"2024-09-10T10:27:43.287534+0000","last_fullsized":"2024-09-10T10:27:43.287534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":12855,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":217,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":25710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":189,"ondisk_log_size":189,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":739702,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":153945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":148308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":154235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":148440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":148510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":738220,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":7257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:27:48.783 INFO:tasks.thrashosds.thrasher:Setting 2.1d to [3, 1, 6, 4] 2024-09-10T10:27:48.783 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.1d', '3', '1', '6', '4'] 2024-09-10T10:27:48.783 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.1d 3 1 6 4 2024-09-10T10:27:48.801 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:48.800+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:48.849 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:48.849 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:48.849 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:48.849 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:48.859 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:27:48.901 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:48.900+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:48.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.922+0000 7f2753f91640 1 Processor -- start 2024-09-10T10:27:48.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.923+0000 7f2753f91640 1 -- start start 2024-09-10T10:27:48.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.923+0000 7f2753f91640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f274c070520 0x7f274c154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.923+0000 7f2753f91640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274c06d270 0x7f274c154f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.923+0000 7f2753f91640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274c067ad0 con 0x7f274c06ce90 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.923+0000 7f2753f91640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f274c068050 con 0x7f274c070520 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.923+0000 7f2753f91640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f274c0c91d0 con 0x7f274c06d270 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.924+0000 7f275152c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f274c070520 0x7f274c154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.924+0000 7f275152c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f274c070520 0x7f274c154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42292/0 (socket says 172.21.15.120:42292) 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.924+0000 7f275152c640 1 -- 172.21.15.120:0/2317058451 learned_addr learned my addr 172.21.15.120:0/2317058451 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.924+0000 7f275252e640 1 --2- 172.21.15.120:0/2317058451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274c06d270 0x7f274c154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.924+0000 7f275152c640 1 -- 172.21.15.120:0/2317058451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274c06d270 msgr2=0x7f274c154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.924+0000 7f275152c640 1 --2- 172.21.15.120:0/2317058451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274c06d270 0x7f274c154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.924+0000 7f275152c640 1 -- 172.21.15.120:0/2317058451 >> v1:172.21.15.120:6789/0 conn(0x7f274c06ce90 legacy=0x7f274c15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.924+0000 7f2750d2b640 1 -- 172.21.15.120:0/2317058451 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3315992553 0 0) 0x7f274c067ad0 con 0x7f274c06ce90 2024-09-10T10:27:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.924+0000 7f275152c640 1 -- 172.21.15.120:0/2317058451 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c067d70 con 0x7f274c070520 2024-09-10T10:27:48.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.925+0000 7f275152c640 1 --2- 172.21.15.120:0/2317058451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c070520 0x7f274c154a00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f273c009870 tx=0x7f273c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2c8c95d0d757d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.925+0000 7f2750d2b640 1 -- 172.21.15.120:0/2317058451 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c037070 con 0x7f274c070520 2024-09-10T10:27:48.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.925+0000 7f2750d2b640 1 -- 172.21.15.120:0/2317058451 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c02ae50 con 0x7f274c070520 2024-09-10T10:27:48.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.925+0000 7f2750d2b640 1 -- 172.21.15.120:0/2317058451 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c032860 con 0x7f274c070520 2024-09-10T10:27:48.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.926+0000 7f2753f91640 1 -- 172.21.15.120:0/2317058451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c070520 msgr2=0x7f274c154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.926+0000 7f2753f91640 1 --2- 172.21.15.120:0/2317058451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c070520 0x7f274c154a00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f273c009870 tx=0x7f273c02a300 comp rx=0 tx=0).stop 2024-09-10T10:27:48.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.926+0000 7f2753f91640 1 -- 172.21.15.120:0/2317058451 shutdown_connections 2024-09-10T10:27:48.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.926+0000 7f2753f91640 1 --2- 172.21.15.120:0/2317058451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274c06d270 0x7f274c154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.926+0000 7f2753f91640 1 --2- 172.21.15.120:0/2317058451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c070520 0x7f274c154a00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.926+0000 7f2753f91640 1 -- 172.21.15.120:0/2317058451 >> 172.21.15.120:0/2317058451 conn(0x7f274c06fa00 msgr2=0x7f274c06fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:48.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.926+0000 7f2753f91640 1 -- 172.21.15.120:0/2317058451 shutdown_connections 2024-09-10T10:27:48.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.926+0000 7f2753f91640 1 -- 172.21.15.120:0/2317058451 wait complete. 2024-09-10T10:27:48.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.927+0000 7f2753f91640 1 Processor -- start 2024-09-10T10:27:48.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.927+0000 7f2753f91640 1 -- start start 2024-09-10T10:27:48.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2753f91640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c06ce90 0x7f274c153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2753f91640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f274c06d270 0x7f274c1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2753f91640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274c070520 0x7f274c158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2753f91640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0c8ef0 con 0x7f274c06ce90 2024-09-10T10:27:48.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2753f91640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0c3aa0 con 0x7f274c06d270 2024-09-10T10:27:48.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2753f91640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f274c070960 con 0x7f274c070520 2024-09-10T10:27:48.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2751d2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c06ce90 0x7f274c153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2751d2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c06ce90 0x7f274c153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42308/0 (socket says 172.21.15.120:42308) 2024-09-10T10:27:48.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2751d2d640 1 -- 172.21.15.120:0/1791227070 learned_addr learned my addr 172.21.15.120:0/1791227070 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f275252e640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274c070520 0x7f274c158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f275152c640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f274c06d270 0x7f274c1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2751d2d640 1 -- 172.21.15.120:0/1791227070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274c070520 msgr2=0x7f274c158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2751d2d640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274c070520 0x7f274c158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2751d2d640 1 -- 172.21.15.120:0/1791227070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f274c06d270 msgr2=0x7f274c1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2751d2d640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f274c06d270 0x7f274c1536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2751d2d640 1 -- 172.21.15.120:0/1791227070 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c0cc2e0 con 0x7f274c06ce90 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f275152c640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f274c06d270 0x7f274c1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2751d2d640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c06ce90 0x7f274c153160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f274800c4b0 tx=0x7f274800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f27427fc640 1 -- 172.21.15.120:0/1791227070 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748017070 con 0x7f274c06ce90 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2753f91640 1 -- 172.21.15.120:0/1791227070 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c158900 con 0x7f274c06ce90 2024-09-10T10:27:48.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f27427fc640 1 -- 172.21.15.120:0/1791227070 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2748005480 con 0x7f274c06ce90 2024-09-10T10:27:48.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f27427fc640 1 -- 172.21.15.120:0/1791227070 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748003830 con 0x7f274c06ce90 2024-09-10T10:27:48.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.928+0000 7f2753f91640 1 -- 172.21.15.120:0/1791227070 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c1f0e20 con 0x7f274c06ce90 2024-09-10T10:27:48.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.929+0000 7f2753f91640 1 -- 172.21.15.120:0/1791227070 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c0cc680 con 0x7f274c06ce90 2024-09-10T10:27:48.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.930+0000 7f27427fc640 1 -- 172.21.15.120:0/1791227070 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f274801a070 con 0x7f274c06ce90 2024-09-10T10:27:48.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.931+0000 7f27427fc640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f273006be30 0x7f273006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:48.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.931+0000 7f27427fc640 1 -- 172.21.15.120:0/1791227070 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6706+0+0 (secure 0 0 0) 0x7f274808a600 con 0x7f274c06ce90 2024-09-10T10:27:48.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.931+0000 7f275152c640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f273006be30 0x7f273006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:48.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.931+0000 7f275152c640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f273006be30 0x7f273006e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f273c034000 tx=0x7f273c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:48.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:48.939+0000 7f27427fc640 1 -- 172.21.15.120:0/1791227070 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f274c0cc680 con 0x7f274c06ce90 2024-09-10T10:27:49.002 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:49.002+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:49.010 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:49.010 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:49.010 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:49.010 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:49.010 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:49.020 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:27:49.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.024+0000 7f50ef7fe640 1 -- 172.21.15.120:0/1850893918 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v34) ==== 87+0+0 (secure 0 0 0) 0x7f50f805c690 con 0x7f510806ce90 2024-09-10T10:27:49.026 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-10T10:27:49.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.026+0000 7f510d36d640 1 -- 172.21.15.120:0/1850893918 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f50f406be30 msgr2=0x7f50f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:49.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.026+0000 7f510d36d640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f50f406be30 0x7f50f406e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0090b0 tx=0x7f50fc02e000 comp rx=0 tx=0).stop 2024-09-10T10:27:49.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.026+0000 7f510d36d640 1 -- 172.21.15.120:0/1850893918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f510806ce90 msgr2=0x7f51080c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:49.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.026+0000 7f510d36d640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f510806ce90 0x7f51080c58b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f50f800c400 tx=0x7f50f800c800 comp rx=0 tx=0).stop 2024-09-10T10:27:49.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.027+0000 7f510d36d640 1 -- 172.21.15.120:0/1850893918 shutdown_connections 2024-09-10T10:27:49.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.027+0000 7f510d36d640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f50f406be30 0x7f50f406e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:49.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.027+0000 7f510d36d640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5108070520 0x7f51081f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:49.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.027+0000 7f510d36d640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510806d270 0x7f51080c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:49.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.027+0000 7f510d36d640 1 --2- 172.21.15.120:0/1850893918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f510806ce90 0x7f51080c58b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:49.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.027+0000 7f510d36d640 1 -- 172.21.15.120:0/1850893918 >> 172.21.15.120:0/1850893918 conn(0x7f510806fa00 msgr2=0x7f510814f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:49.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.027+0000 7f510d36d640 1 -- 172.21.15.120:0/1850893918 shutdown_connections 2024-09-10T10:27:49.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.027+0000 7f510d36d640 1 -- 172.21.15.120:0/1850893918 wait complete. 2024-09-10T10:27:49.103 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:49.103+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:49.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:49.129+0000 7f2753f91640 1 -- 172.21.15.120:0/1791227070 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pg-upmap-items", "pgid": "2.1d", "id": [3, 1, 6, 4]} v 0) -- 0x7f274c15eac0 con 0x7f274c06ce90 2024-09-10T10:27:49.132 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:27:49.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.350520574000001, 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054521709999999996, 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780407+0000", 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.517e-06 2024-09-10T10:27:49.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780414+0000", 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3910000000000003e-06 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780429+0000", 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5379999999999998e-05 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780438+0000", 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9230000000000008e-06 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780885+0000", 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044711300000000001 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780979+0000", 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4312000000000001e-05 2024-09-10T10:27:49.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785856+0000", 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048765350000000004 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.350445847000003, 2024-09-10T10:27:49.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028177609999999998, 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780479+0000", 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9800000000000003e-07 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780488+0000", 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3349999999999996e-06 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780512+0000", 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3156999999999999e-05 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780518+0000", 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.764e-06 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780576+0000", 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7813999999999999e-05 2024-09-10T10:27:49.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780700+0000", 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123733 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783296+0000", 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025963599999999998 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:19 2.14 e24/18 2:2f55791f:::606.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.349326687000001, 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003505083, 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:27:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9400000000000005e-07 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781606+0000", 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6559999999999992e-06 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781624+0000", 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7575e-05 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781630+0000", 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8630000000000002e-06 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781683+0000", 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475000000000002e-05 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781700+0000", 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7393000000000001e-05 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785058+0000", 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033575660000000002 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785103+0000", 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-10T10:27:49.146 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:1 2.14 e24/18 2:292cf221:::200.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.349158181, 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033666960000000002, 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.147 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781767+0000", 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781772+0000", 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5539999999999997e-06 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781781+0000", 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6149999999999997e-06 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781787+0000", 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1589999999999998e-06 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2859000000000002e-05 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781876+0000", 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5916000000000001e-05 2024-09-10T10:27:49.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785108+0000", 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003232146 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785133+0000", 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5015999999999999e-05 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:23 2.e e24/18 2:717a0223:::608.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.348543644999999, 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027543580000000002, 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1699999999999998e-07 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782387+0000", 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3229999999999997e-06 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782399+0000", 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2466000000000001e-05 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782406+0000", 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9460000000000001e-06 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782468+0000", 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2347999999999993e-05 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6016e-05 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785096+0000", 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026118629999999999 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785135+0000", 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8779000000000003e-05 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.156 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:24 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.253361939000001, 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062416909999999997, 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877564+0000", 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877576+0000", 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2411e-05 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877592+0000", 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5812e-05 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877600+0000", 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3590000000000002e-06 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877661+0000", 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1749999999999997e-05 2024-09-10T10:27:49.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877785+0000", 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123807 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877936+0000", 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015084300000000001 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.879317+0000", 2024-09-10T10:27:49.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001381415 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883686+0000", 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004368701 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883755+0000", 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9140999999999994e-05 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883804+0000", 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8772e-05 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.252535975000001, 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056063609999999998, 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:49.162 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878389+0000", 2024-09-10T10:27:49.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699999999999999e-07 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878394+0000", 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6120000000000001e-06 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878407+0000", 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3704e-05 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878415+0000", 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7100000000000007e-06 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878475+0000", 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878603+0000", 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127525 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883995+0000", 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053920460000000002 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 40.252485608999997, 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057312819999999999, 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.166 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5000000000000003e-07 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878442+0000", 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.632e-06 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878450+0000", 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0509999999999997e-06 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878456+0000", 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7200000000000003e-06 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878847+0000", 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039182099999999999 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878949+0000", 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101458 2024-09-10T10:27:49.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.884170+0000", 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052210499999999996 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.183820291000004, 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033173069999999998, 2024-09-10T10:27:49.170 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947105+0000", 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4670000000000001e-06 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947111+0000", 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6230000000000002e-06 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947123+0000", 2024-09-10T10:27:49.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1883e-05 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947132+0000", 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8740000000000001e-06 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947220+0000", 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7800000000000006e-05 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947375+0000", 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015566399999999999 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947501+0000", 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125609 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948393+0000", 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00089165399999999997 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950323+0000", 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019308769999999999 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950366+0000", 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2688999999999997e-05 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950421+0000", 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5167000000000002e-05 2024-09-10T10:27:49.175 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.183764738999997, 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0070341989999999997, 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:27:49.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947163+0000", 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3440000000000001e-06 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947174+0000", 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0248000000000001e-05 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947180+0000", 2024-09-10T10:27:49.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0639999999999999e-06 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947958+0000", 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077789700000000005 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948031+0000", 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3503000000000004e-05 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948151+0000", 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119976 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953625+0000", 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054739360000000004 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954124+0000", 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049858900000000002 2024-09-10T10:27:49.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954157+0000", 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3652999999999998e-05 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954194+0000", 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6455000000000002e-05 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.183672895999997, 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032449549999999999, 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947252+0000", 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000003e-07 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947256+0000", 2024-09-10T10:27:49.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3760000000000001e-06 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947267+0000", 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0259e-05 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947273+0000", 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2190000000000001e-06 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947332+0000", 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9086000000000003e-05 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947452+0000", 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120146 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950496+0000", 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003044171 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.185 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:13 2.1 e26/18 2:81735765:::619.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.183262765000002, 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006029375, 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947662+0000", 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7199999999999997e-07 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6990000000000003e-06 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947681+0000", 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0682999999999999e-05 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947687+0000", 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7479999999999996e-06 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947742+0000", 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4487000000000002e-05 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947761+0000", 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9497999999999999e-05 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953653+0000", 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058916530000000002 2024-09-10T10:27:49.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953691+0000", 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7734999999999997e-05 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:14 2.1 e26/18 2:81735765:::619.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.181942427999999, 2024-09-10T10:27:49.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052525849999999997, 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5700000000000002e-07 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948988+0000", 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7490000000000004e-06 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949003+0000", 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4214e-05 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949010+0000", 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3499999999999999e-06 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949059+0000", 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8758000000000003e-05 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949078+0000", 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9806e-05 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954198+0000", 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051191509999999997 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954234+0000", 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.68e-05 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.194 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:1 2.1 e26/18 2:85bbe569:::202.00000000:head v 26'4, mlcod=0'0)", 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.181714063999998, 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005049258, 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:27:49.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949211+0000", 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4999999999999998e-07 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949228+0000", 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7808999999999998e-05 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949240+0000", 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1226000000000001e-05 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949246+0000", 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0689999999999998e-06 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949292+0000", 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6211999999999997e-05 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949311+0000", 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8873e-05 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954236+0000", 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049253980000000001 2024-09-10T10:27:49.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954259+0000", 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3221e-05 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:26 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.188851981999999, 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039596270000000003, 2024-09-10T10:27:49.199 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942075+0000", 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2680000000000001e-06 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942086+0000", 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1361e-05 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942098+0000", 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2221e-05 2024-09-10T10:27:49.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942106+0000", 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0960000000000001e-06 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942209+0000", 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102944 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942346+0000", 2024-09-10T10:27:49.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136976 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942516+0000", 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017015699999999999 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945195+0000", 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026790920000000001 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:49.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945919+0000", 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00072374500000000003 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945983+0000", 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3418999999999997e-05 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946032+0000", 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9348000000000003e-05 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.204 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:2 2.1 e28/18 2:82971f97:::404.00000000:head v 28'5, mlcod=26'3)", 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.187192826, 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054707489999999996, 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943732+0000", 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1999999999999998e-07 2024-09-10T10:27:49.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943740+0000", 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1910000000000003e-06 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943755+0000", 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5014000000000001e-05 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943763+0000", 2024-09-10T10:27:49.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9519999999999994e-06 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5569999999999997e-05 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943862+0000", 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4150000000000003e-05 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949157+0000", 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052949340000000003 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949202+0000", 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5018000000000002e-05 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.209 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:6 2.e e28/18 2:75ee5fd8:::629.00000000:head v 28'5, mlcod=24'2)", 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.187130998000001, 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027501769999999999, 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943794+0000", 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943799+0000", 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5620000000000003e-06 2024-09-10T10:27:49.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943807+0000", 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.486e-06 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943813+0000", 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6419999999999999e-05 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943888+0000", 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9309000000000001e-05 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946498+0000", 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026094669999999999 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946543+0000", 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:17 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'12, mlcod=26'10)", 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.187087237, 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027214219999999998, 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.214 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943849+0000", 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2023e-05 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943857+0000", 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6009999999999997e-06 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943863+0000", 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2899999999999999e-06 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943927+0000", 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3496000000000003e-05 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943949+0000", 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1801e-05 2024-09-10T10:27:49.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946490+0000", 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025409579999999998 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946558+0000", 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8830000000000003e-05 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.218 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:18 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'13, mlcod=26'10)", 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.187041035, 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027193059999999999, 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943884+0000", 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943889+0000", 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.95e-06 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943896+0000", 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3819999999999998e-06 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943901+0000", 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1719999999999998e-06 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944666+0000", 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00076445300000000001 2024-09-10T10:27:49.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944686+0000", 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-05 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018748879999999999 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946603+0000", 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1947999999999999e-05 2024-09-10T10:27:49.222 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:25 2.12 e28/18 2:4dad84a4:::mds4_sessionmap:head v 28'5, mlcod=26'3)", 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.184127071999999, 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002978994, 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946798+0000", 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0100000000000001e-06 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946807+0000", 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9700000000000005e-06 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946820+0000", 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2458e-05 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946856+0000", 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6427999999999997e-05 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946938+0000", 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2106000000000003e-05 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946958+0000", 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0162000000000001e-05 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949723+0000", 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027645899999999999 2024-09-10T10:27:49.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:49.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:49.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:49.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949776+0000", 2024-09-10T10:27:49.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3270000000000003e-05 2024-09-10T10:27:49.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.227 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:49.227 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:49.227 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:49.229 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:27:49.229 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:49.228+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:49.330 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:49.329+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:49.344 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:49.344 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:49.345 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:27:49.345 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:49.353 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:27:49.430 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:49.430+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:49.497 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:49.497 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:49.497 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:49.497 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:49.498 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:49.506 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:27:49.550 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:49.549+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:49.610 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.829496607999999, 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047203150000000001, 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.611 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.612 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779674+0000", 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0359999999999995e-06 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779702+0000", 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.7970000000000002e-05 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779724+0000", 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2544e-05 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779741+0000", 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6781000000000001e-05 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779981+0000", 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000239764 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780147+0000", 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016605100000000001 2024-09-10T10:27:49.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017096200000000001 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781749+0000", 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001431288 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:49.615 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784234+0000", 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024850940000000002 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784338+0000", 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104078 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784385+0000", 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6746999999999998e-05 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.616 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.1a epoch 24/18 rep_tid 2 entries 24'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.828018497999999, 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021396129999999998, 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.617 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781144+0000", 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.393e-06 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781156+0000", 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1514e-05 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8445999999999998e-05 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781202+0000", 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7439e-05 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781286+0000", 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4652000000000002e-05 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783282+0000", 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0019961689999999999 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.620 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:31 2.d e24/18 2:b50e409b:::mds_snaptable:head v 24'1, mlcod=0'0)", 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.827099007000001, 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030643699999999999, 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782063+0000", 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3699999999999999e-07 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782072+0000", 2024-09-10T10:27:49.622 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9430000000000006e-06 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782083+0000", 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0998e-05 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782092+0000", 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.3270000000000001e-06 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.623 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782158+0000", 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6830999999999998e-05 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782176+0000", 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7370000000000001e-05 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785091+0000", 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029157879999999999 2024-09-10T10:27:49.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785127+0000", 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5176000000000003e-05 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.827040115999999, 2024-09-10T10:27:49.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002013515, 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782122+0000", 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.229e-06 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782137+0000", 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4895999999999999e-05 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782151+0000", 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3229e-05 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782198+0000", 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7301e-05 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782271+0000", 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3516000000000006e-05 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.628 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782290+0000", 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8921000000000001e-05 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784094+0000", 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00180331 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784135+0000", 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1112999999999998e-05 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.629 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:28 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.826928502999998, 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028487970000000001, 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:49.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7000000000000005e-07 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782247+0000", 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3740000000000001e-05 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782258+0000", 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0552e-05 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782265+0000", 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0700000000000001e-06 2024-09-10T10:27:49.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782752+0000", 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048707399999999999 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782771+0000", 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9046999999999999e-05 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.633 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785037+0000", 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002265806 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785082+0000", 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4938000000000002e-05 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:29 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:27:49.634 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.825382507999997, 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001932907, 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783780+0000", 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9900000000000009e-07 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783789+0000", 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0140000000000004e-06 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783802+0000", 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2751e-05 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783811+0000", 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3950000000000007e-06 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783894+0000", 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2887999999999997e-05 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783913+0000", 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9052999999999999e-05 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785668+0000", 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001754633 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785712+0000", 2024-09-10T10:27:49.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4174000000000001e-05 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:7 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 40.736750467999997, 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026422899999999998, 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.561e-06 2024-09-10T10:27:49.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872426+0000", 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3577999999999999e-05 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872462+0000", 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6145e-05 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872473+0000", 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1109e-05 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872543+0000", 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9319000000000005e-05 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872566+0000", 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3768999999999999e-05 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875010+0000", 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024433910000000001 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875053+0000", 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3417999999999997e-05 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:8 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:27:49.643 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 40.736062079, 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0044428820000000004, 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:27:49.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873100+0000", 2024-09-10T10:27:49.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3030000000000005e-06 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873121+0000", 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1471000000000001e-05 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873129+0000", 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7339999999999997e-06 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048786499999999998 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873638+0000", 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0735000000000001e-05 2024-09-10T10:27:49.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877493+0000", 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0038550189999999999 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877542+0000", 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9135000000000003e-05 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:17 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 40.735844745999998, 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042663060000000001, 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:27:49.649 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873318+0000", 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7999999999999993e-07 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.650 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873326+0000", 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8279999999999992e-06 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873341+0000", 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4688999999999999e-05 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873348+0000", 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4810000000000003e-06 2024-09-10T10:27:49.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873843+0000", 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000495083 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873860+0000", 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7609999999999999e-05 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877545+0000", 2024-09-10T10:27:49.652 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036848839999999998 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877583+0000", 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7750999999999998e-05 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:18 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:27:49.653 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 40.735122709000002, 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035807780000000002, 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:49.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874045+0000", 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9909999999999999e-06 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874055+0000", 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0147e-05 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874070+0000", 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4184e-05 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874123+0000", 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3504999999999997e-05 2024-09-10T10:27:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874141+0000", 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7524999999999999e-05 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877586+0000", 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034458700000000002 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877619+0000", 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2962000000000001e-05 2024-09-10T10:27:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.18 epoch 26/18 rep_tid 5 entries 26'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.9:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 40.734061064000002, 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048872359999999997, 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.658 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875101+0000", 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875110+0000", 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2609999999999999e-06 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875129+0000", 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8685000000000002e-05 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875143+0000", 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3723000000000001e-05 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.660 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875222+0000", 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9498000000000004e-05 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879987+0000", 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047652800000000002 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:7 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'5, mlcod=26'3)", 2024-09-10T10:27:49.661 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 35.659298767000003, 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031425310000000001, 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:49.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1100000000000005e-07 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949871+0000", 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8620000000000003e-06 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949885+0000", 2024-09-10T10:27:49.663 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4072e-05 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949893+0000", 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5769999999999996e-06 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949953+0000", 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.978e-05 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949972+0000", 2024-09-10T10:27:49.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9188e-05 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.952959+0000", 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029873859999999999 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953005+0000", 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5855000000000001e-05 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:8 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'6, mlcod=26'3)", 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 35.659246177999997, 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034236359999999999, 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949916+0000", 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6199999999999998e-07 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949922+0000", 2024-09-10T10:27:49.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5390000000000004e-06 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949929+0000", 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4339999999999997e-06 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949934+0000", 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2159999999999997e-06 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950192+0000", 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00025864699999999998 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950213+0000", 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0761999999999999e-05 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953297+0000", 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030842999999999999 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953339+0000", 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1276000000000003e-05 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:34 2.11 e26/18 2:8975f766:::202.00000001:head v 26'3, mlcod=26'1)", 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 35.655273338999997, 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020619190000000002, 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.670 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953889+0000", 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.879999999999999e-07 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953895+0000", 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2249999999999997e-06 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953906+0000", 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:27:49.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953912+0000", 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4840000000000003e-06 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953959+0000", 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7768000000000001e-05 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953976+0000", 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6365999999999999e-05 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955917+0000", 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001941524 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955950+0000", 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2573999999999997e-05 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.674 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:4 2.1a e26/18 2:5fdc322d:::61e.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 29.686205811000001, 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0018931340000000001, 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:27:49.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922956+0000", 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-07 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922962+0000", 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1890000000000004e-06 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922971+0000", 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922978+0000", 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7830000000000001e-06 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923032+0000", 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3637999999999997e-05 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923049+0000", 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7249999999999999e-05 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924803+0000", 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001753788 2024-09-10T10:27:49.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924849+0000", 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6050000000000001e-05 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:20 2.11 e26/18 2:890f50f7:::626.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 29.686098104999999, 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003328012, 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.679 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923064+0000", 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9900000000000001e-07 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923070+0000", 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7499999999999997e-06 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923079+0000", 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9080000000000001e-06 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923089+0000", 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0078000000000001e-05 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923150+0000", 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0300999999999997e-05 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923507+0000", 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00035713599999999998 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926368+0000", 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028615469999999999 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926391+0000", 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2793e-05 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:23 2.18 e26/18 2:1c1db442:::103.00000000.inode:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 29.686038635999999, 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032793050000000002, 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923124+0000", 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0599999999999999e-07 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923130+0000", 2024-09-10T10:27:49.685 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.81e-06 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923141+0000", 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923149+0000", 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2679999999999997e-06 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4744000000000002e-05 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923233+0000", 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9190000000000001e-05 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926353+0000", 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031206419999999999 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926402+0000", 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8755000000000003e-05 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:13 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'6, mlcod=26'2)", 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 29.685986006, 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031818969999999999, 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.688 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923176+0000", 2024-09-10T10:27:49.690 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4799999999999998e-07 2024-09-10T10:27:49.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.690 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.690 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923182+0000", 2024-09-10T10:27:49.690 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.592e-06 2024-09-10T10:27:49.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.692 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.692 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923191+0000", 2024-09-10T10:27:49.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4029999999999996e-06 2024-09-10T10:27:49.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.692 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.692 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923197+0000", 2024-09-10T10:27:49.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1850000000000001e-06 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923230+0000", 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3290000000000001e-05 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923247+0000", 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6810999999999999e-05 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926308+0000", 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030606159999999999 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926357+0000", 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9351999999999998e-05 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.694 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.d epoch 26/18 rep_tid 20 entries 26'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.15:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 29.685433507999999, 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034028819999999999, 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:27:49.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923729+0000", 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.5900000000000005e-07 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.696 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923739+0000", 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8439999999999993e-06 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923761+0000", 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2821999999999999e-05 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923776+0000", 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4763000000000001e-05 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923889+0000", 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000112518 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927131+0000", 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032419760000000001 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:14 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'7, mlcod=26'2)", 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 29.684718546999999, 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027184890000000001, 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5099999999999999e-07 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924449+0000", 2024-09-10T10:27:49.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0310000000000004e-06 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924482+0000", 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2845999999999997e-05 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924500+0000", 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.787e-05 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.701 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924594+0000", 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3566999999999994e-05 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924619+0000", 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4978999999999999e-05 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927118+0000", 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024995479999999999 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:49.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:49.703 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:49.703 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927161+0000", 2024-09-10T10:27:49.703 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2997000000000002e-05 2024-09-10T10:27:49.703 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.703 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.703 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.703 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:49.703 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:49.703 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:49.705 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:27:49.705 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:49.703+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:49.805 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:49.804+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:49.851 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:49.851 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:49.851 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:27:49.851 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:49.859 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:27:49.944 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:49.943+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:50.004 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:50.004 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:50.004 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:50.004 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:50.004 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:50.006 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:50.005+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:50.012 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:27:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.021+0000 7f27427fc640 1 -- 172.21.15.120:0/1791227070 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap-items", "pgid": "2.1d", "id": [3, 1, 6, 4]}]=0 set 2.1d pg_upmap_items mapping to [3->1,6->4] v35) ==== 148+0+0 (secure 0 0 0) 0x7f2748058bd0 con 0x7f274c06ce90 2024-09-10T10:27:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:set 2.1d pg_upmap_items mapping to [3->1,6->4] 2024-09-10T10:27:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 -- 172.21.15.120:0/1791227070 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f273006be30 msgr2=0x7f273006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f273006be30 0x7f273006e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f273c034000 tx=0x7f273c02e000 comp rx=0 tx=0).stop 2024-09-10T10:27:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 -- 172.21.15.120:0/1791227070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c06ce90 msgr2=0x7f274c153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c06ce90 0x7f274c153160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f274800c4b0 tx=0x7f274800c970 comp rx=0 tx=0).stop 2024-09-10T10:27:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 -- 172.21.15.120:0/1791227070 shutdown_connections 2024-09-10T10:27:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f273006be30 0x7f273006e2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274c070520 0x7f274c158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f274c06d270 0x7f274c1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 --2- 172.21.15.120:0/1791227070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f274c06ce90 0x7f274c153160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 -- 172.21.15.120:0/1791227070 >> 172.21.15.120:0/1791227070 conn(0x7f274c06fa00 msgr2=0x7f274c14fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 -- 172.21.15.120:0/1791227070 shutdown_connections 2024-09-10T10:27:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.025+0000 7f2753f91640 1 -- 172.21.15.120:0/1791227070 wait complete. 2024-09-10T10:27:50.107 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:50.106+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:50.158 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:50.158 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:50.158 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:50.159 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:50.159 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.159 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.159 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:27:50.159 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.283981060999999, 2024-09-10T10:27:50.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.006807553, 2024-09-10T10:27:50.159 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.159 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.159 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.159 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.161 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.161 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872290+0000", 2024-09-10T10:27:50.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2029999999999999e-06 2024-09-10T10:27:50.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.161 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.161 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872310+0000", 2024-09-10T10:27:50.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0508000000000001e-05 2024-09-10T10:27:50.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872331+0000", 2024-09-10T10:27:50.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0577e-05 2024-09-10T10:27:50.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872341+0000", 2024-09-10T10:27:50.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0421999999999999e-05 2024-09-10T10:27:50.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872457+0000", 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115618 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872575+0000", 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118563 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872690+0000", 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011438900000000001 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875281+0000", 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002590995 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879035+0000", 2024-09-10T10:27:50.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037544879999999998 2024-09-10T10:27:50.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.165 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.165 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879069+0000", 2024-09-10T10:27:50.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4034e-05 2024-09-10T10:27:50.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.165 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.165 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879095+0000", 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5755999999999999e-05 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.28278444, 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0061030149999999998, 2024-09-10T10:27:50.166 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873485+0000", 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2419999999999999e-06 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.669e-06 2024-09-10T10:27:50.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873505+0000", 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.325e-05 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873514+0000", 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1949999999999992e-06 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873575+0000", 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873647+0000", 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2346000000000002e-05 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:50.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873778+0000", 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013125600000000001 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875444+0000", 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001665962 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879528+0000", 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040837809999999999 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879563+0000", 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5329e-05 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879587+0000", 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3947999999999998e-05 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.172 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.282730389999998, 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096274789999999996, 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873539+0000", 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0299999999999998e-07 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873554+0000", 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5481999999999998e-05 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873565+0000", 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0733e-05 2024-09-10T10:27:50.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.176 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.176 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873573+0000", 2024-09-10T10:27:50.176 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9270000000000005e-06 2024-09-10T10:27:50.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.176 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873633+0000", 2024-09-10T10:27:50.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9864000000000001e-05 2024-09-10T10:27:50.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873752+0000", 2024-09-10T10:27:50.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118946 2024-09-10T10:27:50.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873866+0000", 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000113757 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875462+0000", 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015966699999999999 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883084+0000", 2024-09-10T10:27:50.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076214280000000004 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883136+0000", 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2182000000000003e-05 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883166+0000", 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.9787000000000001e-05 2024-09-10T10:27:50.179 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.282672419000001, 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097094669999999994, 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.180 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873597+0000", 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8899999999999998e-07 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873602+0000", 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9230000000000001e-06 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.182 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873611+0000", 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9150000000000002e-06 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5239999999999997e-06 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874719+0000", 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0011012769999999999 2024-09-10T10:27:50.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874791+0000", 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2305000000000007e-05 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874962+0000", 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000171099 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877786+0000", 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028241059999999998 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883211+0000", 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054250019999999999 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883263+0000", 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1563000000000001e-05 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883306+0000", 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2864000000000002e-05 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.281751729, 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005164681, 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.187 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874518+0000", 2024-09-10T10:27:50.188 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3799999999999996e-07 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874521+0000", 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8550000000000004e-06 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874531+0000", 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8079999999999999e-06 2024-09-10T10:27:50.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874538+0000", 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2760000000000003e-06 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874945+0000", 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000406516 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875045+0000", 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9570999999999999e-05 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879682+0000", 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046369169999999999 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.281717833999998, 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056651770000000004, 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.192 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.193 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4600000000000005e-07 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874565+0000", 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4017e-05 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874573+0000", 2024-09-10T10:27:50.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9689999999999994e-06 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874678+0000", 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7483000000000002e-05 2024-09-10T10:27:50.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.196 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.196 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874786+0000", 2024-09-10T10:27:50.196 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107914 2024-09-10T10:27:50.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.196 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.880216+0000", 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005429499 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:24 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 36.207409511000002, 2024-09-10T10:27:50.197 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0079888200000000006, 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:27:50.198 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948861+0000", 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9269999999999999e-06 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948869+0000", 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7749999999999993e-06 2024-09-10T10:27:50.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948884+0000", 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5713e-05 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948894+0000", 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2050000000000008e-06 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.201 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948958+0000", 2024-09-10T10:27:50.201 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4398000000000002e-05 2024-09-10T10:27:50.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.201 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.201 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949093+0000", 2024-09-10T10:27:50.201 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013508300000000001 2024-09-10T10:27:50.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.201 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949223+0000", 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012949500000000001 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953128+0000", 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039058360000000002 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956775+0000", 2024-09-10T10:27:50.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036467439999999999 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956812+0000", 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6510000000000001e-05 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956848+0000", 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6134000000000003e-05 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.203 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:23 2.c e26/18 2:3631e767:::102.00000000.inode:head v 26'3, mlcod=0'0)", 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 36.205925260999997, 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054176620000000002, 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.204 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.205 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950344+0000", 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9400000000000001e-07 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950362+0000", 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8655999999999999e-05 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950384+0000", 2024-09-10T10:27:50.206 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1929999999999998e-05 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950391+0000", 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7020000000000004e-06 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950624+0000", 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000233031 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950643+0000", 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9459000000000001e-05 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955726+0000", 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050826389999999999 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.208 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4851000000000002e-05 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.235276276, 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0070980829999999998, 2024-09-10T10:27:50.209 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:27:50.210 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920994+0000", 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.697e-06 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921001+0000", 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3719999999999999e-06 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921014+0000", 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2529e-05 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921021+0000", 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9519999999999997e-06 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.212 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921097+0000", 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6499000000000001e-05 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921261+0000", 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016399599999999999 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921396+0000", 2024-09-10T10:27:50.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013448400000000001 2024-09-10T10:27:50.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.214 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.214 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922936+0000", 2024-09-10T10:27:50.214 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015405970000000001 2024-09-10T10:27:50.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.214 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.214 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927988+0000", 2024-09-10T10:27:50.214 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00505158 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928047+0000", 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9011000000000001e-05 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928090+0000", 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3365999999999998e-05 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.215 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.235224765000002, 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007356853, 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.217 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5699999999999998e-07 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921048+0000", 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.754e-06 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921056+0000", 2024-09-10T10:27:50.218 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0469999999999994e-06 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921063+0000", 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5980000000000002e-06 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921747+0000", 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00068475800000000005 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.220 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.220 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921852+0000", 2024-09-10T10:27:50.220 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104347 2024-09-10T10:27:50.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.220 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:27:50.220 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922002+0000", 2024-09-10T10:27:50.220 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000150233 2024-09-10T10:27:50.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.221 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.221 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924357+0000", 2024-09-10T10:27:50.221 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0023545599999999999 2024-09-10T10:27:50.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.221 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.221 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928286+0000", 2024-09-10T10:27:50.221 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039299540000000003 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928349+0000", 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2044000000000004e-05 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928401+0000", 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2101000000000001e-05 2024-09-10T10:27:50.222 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.235185306999998, 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005625166, 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.223 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.224 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.224 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:27:50.224 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.224 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.224 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:27:50.224 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921084+0000", 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0100000000000005e-07 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921093+0000", 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9910000000000008e-06 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.225 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921099+0000", 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2269999999999998e-06 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921103+0000", 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0929999999999996e-06 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921167+0000", 2024-09-10T10:27:50.226 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4030999999999997e-05 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921301+0000", 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013388699999999999 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921471+0000", 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017000899999999999 2024-09-10T10:27:50.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924553+0000", 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030818199999999999 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926600+0000", 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020471339999999999 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926663+0000", 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3274000000000004e-05 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926708+0000", 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5099000000000003e-05 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.229 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.212633637, 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010012200000000001, 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.231 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.231 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:27:50.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.231 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.231 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:27:50.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943637+0000", 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5390000000000002e-06 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943646+0000", 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1340000000000004e-06 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943661+0000", 2024-09-10T10:27:50.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4958e-05 2024-09-10T10:27:50.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943669+0000", 2024-09-10T10:27:50.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5629999999999992e-06 2024-09-10T10:27:50.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943785+0000", 2024-09-10T10:27:50.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011584800000000001 2024-09-10T10:27:50.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943960+0000", 2024-09-10T10:27:50.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000174611 2024-09-10T10:27:50.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.235 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:50.235 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944078+0000", 2024-09-10T10:27:50.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118381 2024-09-10T10:27:50.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.235 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948022+0000", 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039442599999999998 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953546+0000", 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055242449999999997 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.236 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953604+0000", 2024-09-10T10:27:50.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7555e-05 2024-09-10T10:27:50.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.237 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953647+0000", 2024-09-10T10:27:50.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3105999999999997e-05 2024-09-10T10:27:50.237 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.237 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.237 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.212440078, 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099609380000000008, 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943829+0000", 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3099999999999996e-07 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943835+0000", 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8610000000000003e-06 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943846+0000", 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0686999999999999e-05 2024-09-10T10:27:50.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943854+0000", 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944413+0000", 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055893399999999997 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.241 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944505+0000", 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2788999999999995e-05 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944639+0000", 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013387699999999999 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950701+0000", 2024-09-10T10:27:50.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0060612360000000002 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953700+0000", 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029997689999999998 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953743+0000", 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3024e-05 2024-09-10T10:27:50.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953789+0000", 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6109999999999997e-05 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:27:50.244 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.212400056, 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054304089999999998, 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.245 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1399999999999996e-07 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.246 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943874+0000", 2024-09-10T10:27:50.247 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3830000000000002e-06 2024-09-10T10:27:50.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.247 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943881+0000", 2024-09-10T10:27:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1369999999999996e-06 2024-09-10T10:27:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943886+0000", 2024-09-10T10:27:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-06 2024-09-10T10:27:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944057+0000", 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000170781 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944182+0000", 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012506299999999999 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944309+0000", 2024-09-10T10:27:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127081 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.947998+0000", 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003689323 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949225+0000", 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001227005 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949267+0000", 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1391999999999999e-05 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949299+0000", 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2169999999999999e-05 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.212339574000001, 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0068819399999999996, 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943930+0000", 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1200000000000002e-07 2024-09-10T10:27:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943934+0000", 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0969999999999999e-06 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944782+0000", 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00084778299999999996 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944793+0000", 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0993e-05 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944864+0000", 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1241000000000005e-05 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944940+0000", 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6044999999999994e-05 2024-09-10T10:27:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945068+0000", 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012846 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949683+0000", 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046151040000000001 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950706+0000", 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010222149999999999 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950764+0000", 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8180999999999997e-05 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950811+0000", 2024-09-10T10:27:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7108999999999999e-05 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:24 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.210965947999998, 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055052950000000003, 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945304+0000", 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4799999999999997e-07 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945311+0000", 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1269999999999997e-06 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945322+0000", 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1134e-05 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945330+0000", 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9039999999999992e-06 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945408+0000", 2024-09-10T10:27:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8200000000000003e-05 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945533+0000", 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000125518 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945659+0000", 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012583599999999999 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949147+0000", 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003487675 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950716+0000", 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015695279999999999 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950767+0000", 2024-09-10T10:27:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0577e-05 2024-09-10T10:27:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950808+0000", 2024-09-10T10:27:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0847999999999999e-05 2024-09-10T10:27:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:25 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.210324579000002, 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050290129999999997, 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:27:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:27:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945945+0000", 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1119999999999999e-06 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945952+0000", 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5189999999999998e-06 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945965+0000", 2024-09-10T10:27:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3643e-05 2024-09-10T10:27:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945974+0000", 2024-09-10T10:27:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5750000000000001e-06 2024-09-10T10:27:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:27:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0532999999999998e-05 2024-09-10T10:27:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946192+0000", 2024-09-10T10:27:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015717699999999999 2024-09-10T10:27:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946341+0000", 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014897200000000001 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949885+0000", 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035449729999999999 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.270 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950896+0000", 2024-09-10T10:27:50.271 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00101023 2024-09-10T10:27:50.271 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.271 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.271 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.271 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950938+0000", 2024-09-10T10:27:50.271 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2284999999999999e-05 2024-09-10T10:27:50.271 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.271 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.271 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.271 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950973+0000", 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4993999999999997e-05 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.210271507000002, 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048920780000000002, 2024-09-10T10:27:50.272 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:27:50.273 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945998+0000", 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3900000000000004e-07 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946012+0000", 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4799e-05 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.275 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.275 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946021+0000", 2024-09-10T10:27:50.275 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.6000000000000007e-06 2024-09-10T10:27:50.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.275 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.275 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.275 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946029+0000", 2024-09-10T10:27:50.275 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9209999999999992e-06 2024-09-10T10:27:50.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.275 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946093+0000", 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3608000000000005e-05 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946242+0000", 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014889699999999999 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.276 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950889+0000", 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046476140000000004 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.21022748, 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047976679999999997, 2024-09-10T10:27:50.277 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:27:50.278 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946042+0000", 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5199999999999996e-07 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946046+0000", 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1439999999999996e-06 2024-09-10T10:27:50.279 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946054+0000", 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1140000000000006e-06 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946070+0000", 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6232e-05 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.280 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946137+0000", 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6956000000000001e-05 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946260+0000", 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012240299999999999 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.281 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950839+0000", 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045791670000000003 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:23 2.1c e28/18 2:399c8d31:::104.00000000.inode:head v 28'5, mlcod=24'1)", 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.209708141, 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042418079999999997, 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.282 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.283 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3e-07 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946574+0000", 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.277e-05 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.284 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946584+0000", 2024-09-10T10:27:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7710000000000001e-06 2024-09-10T10:27:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946590+0000", 2024-09-10T10:27:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6769999999999998e-06 2024-09-10T10:27:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.285 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946666+0000", 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5420999999999995e-05 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946687+0000", 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1665999999999999e-05 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.286 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950762+0000", 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040749030000000004 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950802+0000", 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9969999999999998e-05 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.287 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:50.289 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:50.288+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:50.289 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:27:50.390 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:50.389+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:50.434 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:50.434 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:50.434 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:27:50.434 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:50.442 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:27:50.450 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:27:50.494 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:50.493+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:50.546 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:50.546 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:50.546 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:50.546 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:50.546 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:50.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.553+0000 7fcbecc23640 1 Processor -- start 2024-09-10T10:27:50.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.553+0000 7fcbecc23640 1 -- start start 2024-09-10T10:27:50.555 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:27:50.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbecc23640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbe8152c20 0x7fcbe80c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbecc23640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe8158950 con 0x7fcbe80c91a0 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbecc23640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe806bc10 con 0x7fcbe806d940 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbecc23640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe806d2f0 con 0x7fcbe8152c20 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbe6d9d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fcbe80c91a0 0x7fcbe81631f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57272/0 (socket says 172.21.15.120:57272) 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbe6d9d640 1 -- 172.21.15.120:0/1372710112 learned_addr learned my addr 172.21.15.120:0/1372710112 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbe659c640 1 --2- 172.21.15.120:0/1372710112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbe8152c20 0x7fcbe80c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbe559a640 1 -- 172.21.15.120:0/1372710112 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1985313934 0 0) 0x7fcbe0001490 con 0x7fcbe80c91a0 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbe559a640 1 -- 172.21.15.120:0/1372710112 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbd4003520 con 0x7fcbe80c91a0 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbe559a640 1 -- 172.21.15.120:0/1372710112 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1945824453 0 0) 0x7fcbe806bc10 con 0x7fcbe806d940 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbe559a640 1 -- 172.21.15.120:0/1372710112 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbe0001490 con 0x7fcbe806d940 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbe659c640 1 -- 172.21.15.120:0/1372710112 >> v1:172.21.15.134:6789/0 conn(0x7fcbe806d940 legacy=0x7fcbe80c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbe659c640 1 -- 172.21.15.120:0/1372710112 >> v1:172.21.15.120:6789/0 conn(0x7fcbe80c91a0 legacy=0x7fcbe81631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.554+0000 7fcbe659c640 1 -- 172.21.15.120:0/1372710112 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe806beb0 con 0x7fcbe8152c20 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.555+0000 7fcbe659c640 1 --2- 172.21.15.120:0/1372710112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbe8152c20 0x7fcbe80c2570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80097d0 tx=0x7fcbd802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=884bec62bfeb1352 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.555+0000 7fcbe559a640 1 -- 172.21.15.120:0/1372710112 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8037070 con 0x7fcbe8152c20 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.555+0000 7fcbe559a640 1 -- 172.21.15.120:0/1372710112 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd8032440 con 0x7fcbe8152c20 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.555+0000 7fcbe559a640 1 -- 172.21.15.120:0/1372710112 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd80327f0 con 0x7fcbe8152c20 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.556+0000 7fcbecc23640 1 -- 172.21.15.120:0/1372710112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbe8152c20 msgr2=0x7fcbe80c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.556+0000 7fcbecc23640 1 --2- 172.21.15.120:0/1372710112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbe8152c20 0x7fcbe80c2570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80097d0 tx=0x7fcbd802a290 comp rx=0 tx=0).stop 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.556+0000 7fcbecc23640 1 -- 172.21.15.120:0/1372710112 shutdown_connections 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.556+0000 7fcbecc23640 1 --2- 172.21.15.120:0/1372710112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbe8152c20 0x7fcbe80c2570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.556+0000 7fcbecc23640 1 -- 172.21.15.120:0/1372710112 >> 172.21.15.120:0/1372710112 conn(0x7fcbe80ccb10 msgr2=0x7fcbe80ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.556+0000 7fcbecc23640 1 -- 172.21.15.120:0/1372710112 shutdown_connections 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.556+0000 7fcbecc23640 1 -- 172.21.15.120:0/1372710112 wait complete. 2024-09-10T10:27:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.557+0000 7fcbecc23640 1 Processor -- start 2024-09-10T10:27:50.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.557+0000 7fcbecc23640 1 -- start start 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbecc23640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbe806d940 0x7fcbe81ee630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbecc23640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbe80c91a0 0x7fcbe80cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbecc23640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbe8152c20 0x7fcbe8155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbecc23640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe8158eb0 con 0x7fcbe80c91a0 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbecc23640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe8158bd0 con 0x7fcbe806d940 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbecc23640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe8159190 con 0x7fcbe8152c20 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbe80c91a0 0x7fcbe80cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbe80c91a0 0x7fcbe80cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42322/0 (socket says 172.21.15.120:42322) 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe5d9b640 1 -- 172.21.15.120:0/742470245 learned_addr learned my addr 172.21.15.120:0/742470245 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbe806d940 0x7fcbe81ee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe6d9d640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbe8152c20 0x7fcbe8155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe5d9b640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbe8152c20 msgr2=0x7fcbe8155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe5d9b640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbe8152c20 0x7fcbe8155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe5d9b640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbe806d940 msgr2=0x7fcbe81ee630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe5d9b640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbe806d940 0x7fcbe81ee630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe5d9b640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe81df9e0 con 0x7fcbe80c91a0 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.558+0000 7fcbe5d9b640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbe80c91a0 0x7fcbe80cbcb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000c400 tx=0x7fcbd000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.559+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0017070 con 0x7fcbe80c91a0 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.559+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd00052d0 con 0x7fcbe80c91a0 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.559+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe81f1d20 con 0x7fcbe80c91a0 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.559+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0003810 con 0x7fcbe80c91a0 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.559+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe81dfca0 con 0x7fcbe80c91a0 2024-09-10T10:27:50.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.559+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fcbe8153000 con 0x7fcbe80c91a0 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.560+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbd001a070 con 0x7fcbe80c91a0 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.560+0000 7fcbdeffd640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcbc806be60 0x7fcbc806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.561+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6735+0+0 (secure 0 0 0) 0x7fcbd008a560 con 0x7fcbe80c91a0 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.561+0000 7fcbe659c640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcbc806be60 0x7fcbc806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.561+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v35) ==== 54+0+16 (secure 0 0 0) 0x7fcbd008cbe0 con 0x7fcbe80c91a0 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.561+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fcbe81f2350 0x7fcbe80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.561+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcbe8072b90 con 0x7fcbe81f2350 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.561+0000 7fcbe659c640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcbc806be60 0x7fcbc806e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8034000 tx=0x7fcbd802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.561+0000 7fcbe6d9d640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fcbe81f2350 0x7fcbe80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.561+0000 7fcbe6d9d640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fcbe81f2350 0x7fcbe80730b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.562+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbe8072b90 con 0x7fcbe81f2350 2024-09-10T10:27:50.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.576+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcbe80728b0 con 0x7fcbe81f2350 2024-09-10T10:27:50.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.578+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcbe80728b0 con 0x7fcbe81f2350 2024-09-10T10:27:50.579 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.578+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fcbe81f1ef0 0x7fcbe8156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.578+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fcbe8156190 con 0x7fcbe81f1ef0 2024-09-10T10:27:50.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.578+0000 7fcbe659c640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fcbe81f1ef0 0x7fcbe8156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.578+0000 7fcbe659c640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fcbe81f1ef0 0x7fcbe8156ac0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.579+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbe8156190 con 0x7fcbe81f1ef0 2024-09-10T10:27:50.591 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:50.590+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:50.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.593+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcbe80cc1f0 con 0x7fcbe81f1ef0 2024-09-10T10:27:50.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.595+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcbe80cc1f0 con 0x7fcbe81f1ef0 2024-09-10T10:27:50.596 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:50.597 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:50.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.595+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fcbe81e1790 0x7fcbe8062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.595+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fcbe80cc400 con 0x7fcbe81e1790 2024-09-10T10:27:50.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.595+0000 7fcbe5d9b640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fcbe81e1790 0x7fcbe8062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.595+0000 7fcbe5d9b640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fcbe81e1790 0x7fcbe8062630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.596+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbe80cc400 con 0x7fcbe81e1790 2024-09-10T10:27:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.609+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcbe81f2ab0 con 0x7fcbe81e1790 2024-09-10T10:27:50.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.611+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcbe81f2ab0 con 0x7fcbe81e1790 2024-09-10T10:27:50.612 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.611+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fcbe8063080 0x7fcbe81e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.611+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fcbe80634c0 con 0x7fcbe8063080 2024-09-10T10:27:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.611+0000 7fcbe6d9d640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fcbe8063080 0x7fcbe81e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.611+0000 7fcbe6d9d640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fcbe8063080 0x7fcbe81e4d70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.612+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbe80634c0 con 0x7fcbe8063080 2024-09-10T10:27:50.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.625+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcbe81e52b0 con 0x7fcbe8063080 2024-09-10T10:27:50.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.626+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcbe81e52b0 con 0x7fcbe8063080 2024-09-10T10:27:50.628 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:50.628 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:50.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.627+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fcbe81e59d0 0x7fcbe802c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.627+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fcbe8204630 con 0x7fcbe81e59d0 2024-09-10T10:27:50.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.627+0000 7fcbe659c640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fcbe81e59d0 0x7fcbe802c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.627+0000 7fcbe659c640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fcbe81e59d0 0x7fcbe802c640 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.628+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbe8204630 con 0x7fcbe81e59d0 2024-09-10T10:27:50.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.641+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcbe802cb80 con 0x7fcbe81e59d0 2024-09-10T10:27:50.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.643+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcbe802cb80 con 0x7fcbe81e59d0 2024-09-10T10:27:50.644 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:50.645 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:50.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.643+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fcbe802d2a0 0x7fcbe802d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.643+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fcbe82077c0 con 0x7fcbe802d2a0 2024-09-10T10:27:50.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.644+0000 7fcbe5d9b640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fcbe802d2a0 0x7fcbe802d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.644+0000 7fcbe5d9b640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fcbe802d2a0 0x7fcbe802d6e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.645+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbe82077c0 con 0x7fcbe802d2a0 2024-09-10T10:27:50.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.658+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcbe8204f30 con 0x7fcbe802d2a0 2024-09-10T10:27:50.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.660+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcbe8204f30 con 0x7fcbe802d2a0 2024-09-10T10:27:50.661 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:50.662 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:50.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.660+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fcbe8205650 0x7fcbe8207a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.660+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fcbe820a810 con 0x7fcbe8205650 2024-09-10T10:27:50.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.660+0000 7fcbe6d9d640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fcbe8205650 0x7fcbe8207a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.661+0000 7fcbe6d9d640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fcbe8205650 0x7fcbe8207a40 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.661+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbe820a810 con 0x7fcbe8205650 2024-09-10T10:27:50.666 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:50.666 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:50.666 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.885019264999997, 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:50.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:27:50.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:50.672 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.884688765, 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:27:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.884642221999997, 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:50.678 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.679 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.680 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.79320001, 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:27:50.682 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:27:50.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.685 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:27:50.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.793118667000002, 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:27:50.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:27:50.690 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:50.691 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.793092111999997, 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:27:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:27:50.696 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.697 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.792949462999999, 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:27:50.699 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:27:50.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.702 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.703 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.704 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.790302529999998, 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.706 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:27:50.708 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 41.789451790000001, 2024-09-10T10:27:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:27:50.710 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.710 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.710 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.710 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.710 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.711 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:50.711 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:27:50.712 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.713 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.714 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 36.717300107, 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.715 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.716 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:27:50.718 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.719 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.720 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 36.717250970999999, 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:27:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:27:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 36.717139492999998, 2024-09-10T10:27:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:27:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.743738973999999, 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:27:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:27:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.743708803000001, 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:27:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:27:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:27:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.743652310999998, 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:27:50.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.744 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.747 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 30.743551143000001, 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.748 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.749 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.749 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:50.749 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.749 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.749 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:27:50.749 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:27:50.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:27:50.750 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:50.751 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.753 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.723284046, 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.754 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:27:50.756 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:50.757 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:27:50.758 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.719218826999999, 2024-09-10T10:27:50.759 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:50.760 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.761 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.762 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.763 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.719200956999998, 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.764 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.765 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.766 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:27:50.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 24.719154718999999, 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:27:50.768 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:27:50.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:27:50.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.772 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:50.773 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:50.773 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:50.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.674+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcbe8207f80 con 0x7fcbe8205650 2024-09-10T10:27:50.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.676+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcbe8207f80 con 0x7fcbe8205650 2024-09-10T10:27:50.774 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:50.775 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:27:50.776 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:50.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.677+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fcbe8208600 0x7fcbe820a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:50.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.677+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fcbe820d960 con 0x7fcbe8208600 2024-09-10T10:27:50.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.677+0000 7fcbe659c640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fcbe8208600 0x7fcbe820a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:50.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.677+0000 7fcbe659c640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fcbe8208600 0x7fcbe820a9f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:50.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.678+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbe820d960 con 0x7fcbe8208600 2024-09-10T10:27:50.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.691+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fcbe8000950 con 0x7fcbe8208600 2024-09-10T10:27:50.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbdeffd640 1 -- 172.21.15.120:0/742470245 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcbe8000950 con 0x7fcbe8208600 2024-09-10T10:27:50.777 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:50.778 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:50.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fcbe81f2350 msgr2=0x7fcbe80730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fcbe81f2350 0x7fcbe80730b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fcbe81f1ef0 msgr2=0x7fcbe8156ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fcbe81f1ef0 0x7fcbe8156ac0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fcbe81e1790 msgr2=0x7fcbe8062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fcbe81e1790 0x7fcbe8062630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fcbe8063080 msgr2=0x7fcbe81e4d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fcbe8063080 0x7fcbe81e4d70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fcbe81e59d0 msgr2=0x7fcbe802c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fcbe81e59d0 0x7fcbe802c640 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fcbe802d2a0 msgr2=0x7fcbe802d6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fcbe802d2a0 0x7fcbe802d6e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fcbe8205650 msgr2=0x7fcbe8207a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fcbe8205650 0x7fcbe8207a40 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fcbe8208600 msgr2=0x7fcbe820a9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fcbe8208600 0x7fcbe820a9f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbe6d9d640 1 -- 172.21.15.120:0/742470245 reap_dead start 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbe6d9d640 1 -- 172.21.15.120:0/742470245 reap_dead start 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbe6d9d640 1 -- 172.21.15.120:0/742470245 reap_dead start 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbe6d9d640 1 -- 172.21.15.120:0/742470245 reap_dead start 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbe6d9d640 1 -- 172.21.15.120:0/742470245 reap_dead start 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcbc806be60 msgr2=0x7fcbc806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcbc806be60 0x7fcbc806e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8034000 tx=0x7fcbd802e000 comp rx=0 tx=0).stop 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbe80c91a0 msgr2=0x7fcbe80cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbe80c91a0 0x7fcbe80cbcb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000c400 tx=0x7fcbd000c8c0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 shutdown_connections 2024-09-10T10:27:50.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcbc806be60 0x7fcbc806e320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 --2- 172.21.15.120:0/742470245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbe80c91a0 0x7fcbe80cbcb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:50.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 >> 172.21.15.120:0/742470245 conn(0x7fcbe80ccb10 msgr2=0x7fcbe8150460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:50.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 shutdown_connections 2024-09-10T10:27:50.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:50.693+0000 7fcbecc23640 1 -- 172.21.15.120:0/742470245 wait complete. 2024-09-10T10:27:50.781 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:50.773+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:50.874 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:50.873+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:50.885 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:50.885 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:50.893 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:27:50.975 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:50.974+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:51.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-10T10:27:51.041 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:51.041 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:51.051 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:27:51.085 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:51.084+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:51.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.138+0000 7fec0523c640 1 Processor -- start 2024-09-10T10:27:51.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.138+0000 7fec0523c640 1 -- start start 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7fec0523c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fec000cc0c0 0x7fec0015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7fec0523c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec000cc4a0 0x7fec00165430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7fec0523c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec0015b1d0 con 0x7fec000cb2f0 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7fec0523c640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fec000cbd00 con 0x7fec000cc0c0 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7fec0523c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec0015ba70 con 0x7fec000cc4a0 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7febfed9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fec000cc0c0 0x7fec0015aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7febfe59c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fec000cb2f0 0x7fec0015e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57274/0 (socket says 172.21.15.120:57274) 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7febfed9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fec000cc0c0 0x7fec0015aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42334/0 (socket says 172.21.15.120:42334) 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7febfe59c640 1 -- 172.21.15.120:0/4147679670 learned_addr learned my addr 172.21.15.120:0/4147679670 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7febff59e640 1 --2- 172.21.15.120:0/4147679670 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec000cc4a0 0x7fec00165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:51.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7febfed9d640 1 -- 172.21.15.120:0/4147679670 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec000cc4a0 msgr2=0x7fec00165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:51.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7febfed9d640 1 --2- 172.21.15.120:0/4147679670 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec000cc4a0 0x7fec00165430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:51.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7febfed9d640 1 -- 172.21.15.120:0/4147679670 >> v1:172.21.15.120:6789/0 conn(0x7fec000cb2f0 legacy=0x7fec0015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:51.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7febfed9d640 1 -- 172.21.15.120:0/4147679670 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0015b4b0 con 0x7fec000cc0c0 2024-09-10T10:27:51.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.139+0000 7febfdd9b640 1 -- 172.21.15.120:0/4147679670 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1862841594 0 0) 0x7fec0015b1d0 con 0x7fec000cb2f0 2024-09-10T10:27:51.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.140+0000 7febfed9d640 1 --2- 172.21.15.120:0/4147679670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc0c0 0x7fec0015aab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7febf0009d90 tx=0x7febf002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76aeb26d33cf5e23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:51.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.140+0000 7febfdd9b640 1 -- 172.21.15.120:0/4147679670 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0007740 con 0x7fec000cc0c0 2024-09-10T10:27:51.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.140+0000 7febfdd9b640 1 -- 172.21.15.120:0/4147679670 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf00078e0 con 0x7fec000cc0c0 2024-09-10T10:27:51.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.140+0000 7febfdd9b640 1 -- 172.21.15.120:0/4147679670 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0007cb0 con 0x7fec000cc0c0 2024-09-10T10:27:51.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.141+0000 7fec0523c640 1 -- 172.21.15.120:0/4147679670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc0c0 msgr2=0x7fec0015aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:51.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.141+0000 7fec0523c640 1 --2- 172.21.15.120:0/4147679670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc0c0 0x7fec0015aab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7febf0009d90 tx=0x7febf002a660 comp rx=0 tx=0).stop 2024-09-10T10:27:51.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.141+0000 7fec0523c640 1 -- 172.21.15.120:0/4147679670 shutdown_connections 2024-09-10T10:27:51.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.141+0000 7fec0523c640 1 --2- 172.21.15.120:0/4147679670 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec000cc4a0 0x7fec00165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:51.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.141+0000 7fec0523c640 1 --2- 172.21.15.120:0/4147679670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc0c0 0x7fec0015aab0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:51.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.141+0000 7fec0523c640 1 -- 172.21.15.120:0/4147679670 >> 172.21.15.120:0/4147679670 conn(0x7fec0006d560 msgr2=0x7fec0006d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:51.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.141+0000 7fec0523c640 1 -- 172.21.15.120:0/4147679670 shutdown_connections 2024-09-10T10:27:51.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.141+0000 7fec0523c640 1 -- 172.21.15.120:0/4147679670 wait complete. 2024-09-10T10:27:51.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.142+0000 7fec0523c640 1 Processor -- start 2024-09-10T10:27:51.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.142+0000 7fec0523c640 1 -- start start 2024-09-10T10:27:51.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7fec0523c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec000cb2f0 0x7fec000c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:51.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7fec0523c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec000cc0c0 0x7fec001f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:51.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7fec0523c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc4a0 0x7fec001f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:51.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7fec0523c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec0015e1d0 con 0x7fec000cc4a0 2024-09-10T10:27:51.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7fec0523c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec0015e540 con 0x7fec000cb2f0 2024-09-10T10:27:51.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7fec0523c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec000cc8f0 con 0x7fec000cc0c0 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7febff59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc4a0 0x7fec001f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7febff59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc4a0 0x7fec001f0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42338/0 (socket says 172.21.15.120:42338) 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7febff59e640 1 -- 172.21.15.120:0/2888097555 learned_addr learned my addr 172.21.15.120:0/2888097555 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7febfe59c640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec000cc0c0 0x7fec001f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.143+0000 7febfed9d640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec000cb2f0 0x7fec000c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7febff59e640 1 -- 172.21.15.120:0/2888097555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec000cc0c0 msgr2=0x7fec001f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7febff59e640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec000cc0c0 0x7fec001f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7febff59e640 1 -- 172.21.15.120:0/2888097555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec000cb2f0 msgr2=0x7fec000c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7febff59e640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec000cb2f0 0x7fec000c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7febff59e640 1 -- 172.21.15.120:0/2888097555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec001f4c80 con 0x7fec000cc4a0 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7febff59e640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc4a0 0x7fec001f0a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7febf800d830 tx=0x7febf800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7febe77fe640 1 -- 172.21.15.120:0/2888097555 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf800b6c0 con 0x7fec000cc4a0 2024-09-10T10:27:51.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7febe77fe640 1 -- 172.21.15.120:0/2888097555 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf8019070 con 0x7fec000cc4a0 2024-09-10T10:27:51.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7fec0523c640 1 -- 172.21.15.120:0/2888097555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec000ca750 con 0x7fec000cc4a0 2024-09-10T10:27:51.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7febe77fe640 1 -- 172.21.15.120:0/2888097555 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf800bac0 con 0x7fec000cc4a0 2024-09-10T10:27:51.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.144+0000 7fec0523c640 1 -- 172.21.15.120:0/2888097555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0015a940 con 0x7fec000cc4a0 2024-09-10T10:27:51.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.145+0000 7fec0523c640 1 -- 172.21.15.120:0/2888097555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0014f050 con 0x7fec000cc4a0 2024-09-10T10:27:51.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.145+0000 7febe77fe640 1 -- 172.21.15.120:0/2888097555 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febf801e460 con 0x7fec000cc4a0 2024-09-10T10:27:51.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.146+0000 7febe77fe640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7febec06be60 0x7febec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:51.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.146+0000 7febe77fe640 1 -- 172.21.15.120:0/2888097555 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6735+0+0 (secure 0 0 0) 0x7febf808b6a0 con 0x7fec000cc4a0 2024-09-10T10:27:51.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.146+0000 7febfed9d640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7febec06be60 0x7febec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:51.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.146+0000 7febfed9d640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7febec06be60 0x7febec06e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7febf0034000 tx=0x7febf002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:51.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.150+0000 7febe77fe640 1 -- 172.21.15.120:0/2888097555 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fec0014f050 con 0x7fec000cc4a0 2024-09-10T10:27:51.154 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:51.154 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:51.163 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:27:51.207 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:51.206+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:51.268 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:51.268 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:51.277 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:27:51.278 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:51.276+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:51.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:51.331+0000 7fec0523c640 1 -- 172.21.15.120:0/2888097555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fec000ca550 con 0x7fec000cc4a0 2024-09-10T10:27:51.378 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:51.377+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:51.422 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:51.422 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:51.430 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:27:51.515 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:51.514+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:51.574 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:51.574 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:51.579 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:51.578+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:51.582 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:27:51.680 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:51.678+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:51.725 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:51.725 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:51.733 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:27:51.781 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:51.780+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:51.837 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:51.838 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:51.846 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:27:51.882 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:51.880+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:51.983 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:51.982+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:51.989 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:51.989 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:51.998 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:27:52.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.041+0000 7febe77fe640 1 -- 172.21.15.120:0/2888097555 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v37) ==== 97+0+0 (secure 0 0 0) 0x7febf8059690 con 0x7fec000cc4a0 2024-09-10T10:27:52.042 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-10T10:27:52.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 -- 172.21.15.120:0/2888097555 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7febec06be60 msgr2=0x7febec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7febec06be60 0x7febec06e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7febf0034000 tx=0x7febf002f000 comp rx=0 tx=0).stop 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 -- 172.21.15.120:0/2888097555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc4a0 msgr2=0x7fec001f0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc4a0 0x7fec001f0a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7febf800d830 tx=0x7febf800dcf0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 -- 172.21.15.120:0/2888097555 shutdown_connections 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7febec06be60 0x7febec06e320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec000cc4a0 0x7fec001f0a30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec000cc0c0 0x7fec001f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 --2- 172.21.15.120:0/2888097555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec000cb2f0 0x7fec000c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 -- 172.21.15.120:0/2888097555 >> 172.21.15.120:0/2888097555 conn(0x7fec0006d560 msgr2=0x7fec000cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 -- 172.21.15.120:0/2888097555 shutdown_connections 2024-09-10T10:27:52.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.043+0000 7fec0523c640 1 -- 172.21.15.120:0/2888097555 wait complete. 2024-09-10T10:27:52.084 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:52.082+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:52.143 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:52.143 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:52.152 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:27:52.185 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:52.183+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:52.254 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:52.255 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:52.263 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:27:52.307 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:52.306+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:52.366 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:52.366 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:52.377 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:27:52.386 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:52.385+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:52.487 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:52.486+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:52.521 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:52.521 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:52.530 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:27:52.614 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:52.613+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:52.673 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:52.673 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:52.682 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:27:52.723 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:52.722+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:52.782 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:27:52.826 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:52.825+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:52.827 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:52.828 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:52.838 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:27:52.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.880+0000 7f313f53b640 1 Processor -- start 2024-09-10T10:27:52.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.880+0000 7f313f53b640 1 -- start start 2024-09-10T10:27:52.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313f53b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3138151820 0x7f31380c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313f53b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f31380c7980 0x7f3138071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313f53b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31381585b0 con 0x7f31380c8070 2024-09-10T10:27:52.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313f53b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f31381580a0 con 0x7f31380c7980 2024-09-10T10:27:52.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313f53b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f31380c6680 con 0x7f3138151820 2024-09-10T10:27:52.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313cad6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f31380c7980 0x7f3138071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313d2d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3138151820 0x7f31380c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313dad8640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f31380c8070 0x7f31380c5180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:50156/0 (socket says 172.21.15.120:50156) 2024-09-10T10:27:52.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313dad8640 1 -- 172.21.15.120:0/91509342 learned_addr learned my addr 172.21.15.120:0/91509342 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:52.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313d2d7640 1 -- 172.21.15.120:0/91509342 >> v2:172.21.15.134:3300/0 conn(0x7f31380c7980 msgr2=0x7f3138071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:52.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313d2d7640 1 --2- 172.21.15.120:0/91509342 >> v2:172.21.15.134:3300/0 conn(0x7f31380c7980 0x7f3138071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313d2d7640 1 -- 172.21.15.120:0/91509342 >> v1:172.21.15.134:6789/0 conn(0x7f31380c8070 legacy=0x7f31380c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313d2d7640 1 -- 172.21.15.120:0/91509342 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31381582d0 con 0x7f3138151820 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.881+0000 7f313cad6640 1 --2- 172.21.15.120:0/91509342 >> v2:172.21.15.134:3300/0 conn(0x7f31380c7980 0x7f3138071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.882+0000 7f313d2d7640 1 --2- 172.21.15.120:0/91509342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3138151820 0x7f31380c36b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f313400ba40 tx=0x7f313402c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8944103a01fcb71c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.882+0000 7f312ffff640 1 -- 172.21.15.120:0/91509342 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31340327d0 con 0x7f3138151820 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.882+0000 7f312ffff640 1 -- 172.21.15.120:0/91509342 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3134038070 con 0x7f3138151820 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.882+0000 7f312ffff640 1 -- 172.21.15.120:0/91509342 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134032b90 con 0x7f3138151820 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.883+0000 7f313f53b640 1 -- 172.21.15.120:0/91509342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3138151820 msgr2=0x7f31380c36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.883+0000 7f313f53b640 1 --2- 172.21.15.120:0/91509342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3138151820 0x7f31380c36b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f313400ba40 tx=0x7f313402c8d0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.883+0000 7f313f53b640 1 -- 172.21.15.120:0/91509342 shutdown_connections 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.883+0000 7f313f53b640 1 --2- 172.21.15.120:0/91509342 >> v2:172.21.15.134:3300/0 conn(0x7f31380c7980 0x7f3138071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.883+0000 7f313f53b640 1 --2- 172.21.15.120:0/91509342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3138151820 0x7f31380c36b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.883+0000 7f313f53b640 1 -- 172.21.15.120:0/91509342 >> 172.21.15.120:0/91509342 conn(0x7f313806bc10 msgr2=0x7f313806bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:52.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.883+0000 7f313f53b640 1 -- 172.21.15.120:0/91509342 shutdown_connections 2024-09-10T10:27:52.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.883+0000 7f313f53b640 1 -- 172.21.15.120:0/91509342 wait complete. 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.884+0000 7f313f53b640 1 Processor -- start 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.884+0000 7f313f53b640 1 -- start start 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313f53b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31380c7980 0x7f313814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313f53b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31380c8070 0x7f31380ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313f53b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3138151820 0x7f31380cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313f53b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f31380c6ea0 con 0x7f31380c7980 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313f53b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f31380c74f0 con 0x7f3138151820 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313f53b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f31380c7180 con 0x7f31380c8070 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313d2d7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31380c7980 0x7f313814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313d2d7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31380c7980 0x7f313814e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42344/0 (socket says 172.21.15.120:42344) 2024-09-10T10:27:52.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313d2d7640 1 -- 172.21.15.120:0/2244802438 learned_addr learned my addr 172.21.15.120:0/2244802438 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313dad8640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3138151820 0x7f31380cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313cad6640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31380c8070 0x7f31380ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313d2d7640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31380c8070 msgr2=0x7f31380ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313d2d7640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31380c8070 0x7f31380ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313d2d7640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3138151820 msgr2=0x7f31380cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313d2d7640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3138151820 0x7f31380cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.885+0000 7f313d2d7640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31380cca30 con 0x7f31380c7980 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.886+0000 7f313d2d7640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31380c7980 0x7f313814e4c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3134035040 tx=0x7f3134004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.886+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134033ca0 con 0x7f31380c7980 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.886+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3134042070 con 0x7f31380c7980 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.886+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31381f12e0 con 0x7f31380c7980 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.886+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134033e60 con 0x7f31380c7980 2024-09-10T10:27:52.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.886+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31380bd660 con 0x7f31380c7980 2024-09-10T10:27:52.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.887+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f31380c8520 con 0x7f31380c7980 2024-09-10T10:27:52.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.887+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f313403c630 con 0x7f31380c7980 2024-09-10T10:27:52.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.888+0000 7f312dffb640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f311c06be60 0x7f311c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.888+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6735+0+0 (secure 0 0 0) 0x7f31340ac1d0 con 0x7f31380c7980 2024-09-10T10:27:52.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.888+0000 7f313cad6640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f311c06be60 0x7f311c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.888+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v37) ==== 54+0+16 (secure 0 0 0) 0x7f31340ae850 con 0x7f31380c7980 2024-09-10T10:27:52.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.888+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f31380bd8a0 0x7f31380cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.888+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31380cbbd0 con 0x7f31380bd8a0 2024-09-10T10:27:52.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.888+0000 7f313cad6640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f311c06be60 0x7f311c06e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3128009560 tx=0x7f3128009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:52.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.888+0000 7f313dad8640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f31380bd8a0 0x7f31380cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.889+0000 7f313dad8640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f31380bd8a0 0x7f31380cb690 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:52.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.889+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31380cbbd0 con 0x7f31380bd8a0 2024-09-10T10:27:52.892 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:52.889+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:52.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.903+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3138072730 con 0x7f31380bd8a0 2024-09-10T10:27:52.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.905+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3138072730 con 0x7f31380bd8a0 2024-09-10T10:27:52.906 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:52.906 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:52.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.905+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f31381f3ca0 0x7f31381f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.905+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f3138156740 con 0x7f31381f3ca0 2024-09-10T10:27:52.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.905+0000 7f313d2d7640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f31381f3ca0 0x7f31381f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.905+0000 7f313d2d7640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f31381f3ca0 0x7f31381f4100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:52.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.906+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3138156740 con 0x7f31381f3ca0 2024-09-10T10:27:52.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.920+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3138150d80 con 0x7f31381f3ca0 2024-09-10T10:27:52.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.921+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3138150d80 con 0x7f31381f3ca0 2024-09-10T10:27:52.922 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:52.923 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:52.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.921+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f3138062630 0x7f3138062a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.921+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f31380cabf0 con 0x7f3138062630 2024-09-10T10:27:52.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.922+0000 7f313cad6640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f3138062630 0x7f3138062a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.922+0000 7f313cad6640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f3138062630 0x7f3138062a10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:52.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.922+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31380cabf0 con 0x7f3138062630 2024-09-10T10:27:52.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.936+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3138154e40 con 0x7f3138062630 2024-09-10T10:27:52.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.937+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3138154e40 con 0x7f3138062630 2024-09-10T10:27:52.939 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:52.939 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:52.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.938+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f3138063310 0x7f313814d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.938+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f3138155050 con 0x7f3138063310 2024-09-10T10:27:52.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.938+0000 7f313dad8640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f3138063310 0x7f313814d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.938+0000 7f313dad8640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f3138063310 0x7f313814d5e0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:52.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.939+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3138155050 con 0x7f3138063310 2024-09-10T10:27:52.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.952+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3138155050 con 0x7f3138063310 2024-09-10T10:27:52.953 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:52.953 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:52.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.953+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3138155050 con 0x7f3138063310 2024-09-10T10:27:52.955 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:52.956 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:52.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.953+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f313814dd70 0x7f313802cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.953+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f3138155050 con 0x7f313814dd70 2024-09-10T10:27:52.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.954+0000 7f313d2d7640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f313814dd70 0x7f313802cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.954+0000 7f313d2d7640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f313814dd70 0x7f313802cb50 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:52.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.955+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3138155050 con 0x7f313814dd70 2024-09-10T10:27:52.963 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:27:52.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.968+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3138154ff0 con 0x7f313814dd70 2024-09-10T10:27:52.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.970+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3138154ff0 con 0x7f313814dd70 2024-09-10T10:27:52.971 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:52.971 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:52.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.970+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f313802d5a0 0x7f3138207050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.970+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f313802d9e0 con 0x7f313802d5a0 2024-09-10T10:27:52.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.970+0000 7f313cad6640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f313802d5a0 0x7f3138207050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.971+0000 7f313cad6640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f313802d5a0 0x7f3138207050 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:52.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.971+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f313802d9e0 con 0x7f313802d5a0 2024-09-10T10:27:52.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.984+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f313802d9e0 con 0x7f313802d5a0 2024-09-10T10:27:52.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.986+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f313802d9e0 con 0x7f313802d5a0 2024-09-10T10:27:52.987 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:52.988 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:52.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.986+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f3138207aa0 0x7f3138209e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:52.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.986+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f313820cd60 con 0x7f3138207aa0 2024-09-10T10:27:52.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.987+0000 7f313dad8640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f3138207aa0 0x7f3138209e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:52.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.987+0000 7f313dad8640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f3138207aa0 0x7f3138209e90 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:52.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:52.988+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f313820cd60 con 0x7f3138207aa0 2024-09-10T10:27:53.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.001+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f313820cd30 con 0x7f3138207aa0 2024-09-10T10:27:53.004 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:53.002+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:53.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.003+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f313820cd30 con 0x7f3138207aa0 2024-09-10T10:27:53.004 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:53.005 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:53.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.003+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f313820a8e0 0x7f313820ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:53.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.003+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f3138210130 con 0x7f313820a8e0 2024-09-10T10:27:53.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.003+0000 7f313d2d7640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f313820a8e0 0x7f313820ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:53.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.003+0000 7f313d2d7640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f313820a8e0 0x7f313820ad20 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:53.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.004+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3138210130 con 0x7f313820a8e0 2024-09-10T10:27:53.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.017+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f313820b320 con 0x7f313820a8e0 2024-09-10T10:27:53.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f312dffb640 1 -- 172.21.15.120:0/2244802438 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f313820b320 con 0x7f313820a8e0 2024-09-10T10:27:53.020 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:53.022 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:53.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f31380bd8a0 msgr2=0x7f31380cb690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:53.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f31380bd8a0 0x7f31380cb690 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:53.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f31381f3ca0 msgr2=0x7f31381f4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:53.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f31381f3ca0 0x7f31381f4100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:53.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f3138062630 msgr2=0x7f3138062a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:53.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f3138062630 0x7f3138062a10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:53.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f3138063310 msgr2=0x7f313814d5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:53.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f3138063310 0x7f313814d5e0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f313814dd70 msgr2=0x7f313802cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f313814dd70 0x7f313802cb50 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313dad8640 1 -- 172.21.15.120:0/2244802438 reap_dead start 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f313802d5a0 msgr2=0x7f3138207050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f313802d5a0 0x7f3138207050 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313dad8640 1 -- 172.21.15.120:0/2244802438 reap_dead start 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f3138207aa0 msgr2=0x7f3138209e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f3138207aa0 0x7f3138209e90 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f313820a8e0 msgr2=0x7f313820ad20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f313820a8e0 0x7f313820ad20 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313dad8640 1 -- 172.21.15.120:0/2244802438 reap_dead start 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f311c06be60 msgr2=0x7f311c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f311c06be60 0x7f311c06e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3128009560 tx=0x7f3128009190 comp rx=0 tx=0).stop 2024-09-10T10:27:53.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31380c7980 msgr2=0x7f313814e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:53.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.019+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31380c7980 0x7f313814e4c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3134035040 tx=0x7f3134004730 comp rx=0 tx=0).stop 2024-09-10T10:27:53.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.020+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 shutdown_connections 2024-09-10T10:27:53.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.020+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f311c06be60 0x7f311c06e320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:53.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.020+0000 7f313f53b640 1 --2- 172.21.15.120:0/2244802438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31380c7980 0x7f313814e4c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:53.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.020+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 >> 172.21.15.120:0/2244802438 conn(0x7f313806bc10 msgr2=0x7f31380c5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:53.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.020+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 shutdown_connections 2024-09-10T10:27:53.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:53.020+0000 7f313f53b640 1 -- 172.21.15.120:0/2244802438 wait complete. 2024-09-10T10:27:53.091 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:53.091+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:53.107 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:53.107 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:53.108 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:53.108 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:53.108 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:53.116 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:27:53.193 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:53.191+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:53.221 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:53.221 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:53.221 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:53.221 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.440236213000006, 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047203150000000001, 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779674+0000", 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0359999999999995e-06 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779702+0000", 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.7970000000000002e-05 2024-09-10T10:27:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779724+0000", 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2544e-05 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779741+0000", 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6781000000000001e-05 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779981+0000", 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000239764 2024-09-10T10:27:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780147+0000", 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016605100000000001 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017096200000000001 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781749+0000", 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001431288 2024-09-10T10:27:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784234+0000", 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024850940000000002 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784338+0000", 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104078 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784385+0000", 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6746999999999998e-05 2024-09-10T10:27:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.1a epoch 24/18 rep_tid 2 entries 24'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.438758102999998, 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021396129999999998, 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781144+0000", 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.393e-06 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781156+0000", 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1514e-05 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8445999999999998e-05 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781202+0000", 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7439e-05 2024-09-10T10:27:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781286+0000", 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4652000000000002e-05 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783282+0000", 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0019961689999999999 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:31 2.d e24/18 2:b50e409b:::mds_snaptable:head v 24'1, mlcod=0'0)", 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.437838611999993, 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030643699999999999, 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782063+0000", 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3699999999999999e-07 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782072+0000", 2024-09-10T10:27:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9430000000000006e-06 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782083+0000", 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0998e-05 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782092+0000", 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.3270000000000001e-06 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782158+0000", 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6830999999999998e-05 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782176+0000", 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7370000000000001e-05 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785091+0000", 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029157879999999999 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785127+0000", 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5176000000000003e-05 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.437779720999998, 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002013515, 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782122+0000", 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.229e-06 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782137+0000", 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4895999999999999e-05 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782151+0000", 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3229e-05 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782198+0000", 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7301e-05 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782271+0000", 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3516000000000006e-05 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782290+0000", 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8921000000000001e-05 2024-09-10T10:27:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784094+0000", 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00180331 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784135+0000", 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1112999999999998e-05 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:28 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.437668107999997, 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028487970000000001, 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7000000000000005e-07 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782247+0000", 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3740000000000001e-05 2024-09-10T10:27:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782258+0000", 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0552e-05 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782265+0000", 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0700000000000001e-06 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782752+0000", 2024-09-10T10:27:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048707399999999999 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782771+0000", 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9046999999999999e-05 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785037+0000", 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002265806 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785082+0000", 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4938000000000002e-05 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:29 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.436122112999996, 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001932907, 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.244 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783780+0000", 2024-09-10T10:27:53.245 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9900000000000009e-07 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783789+0000", 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0140000000000004e-06 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783802+0000", 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2751e-05 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.246 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783811+0000", 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3950000000000007e-06 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783894+0000", 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2887999999999997e-05 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783913+0000", 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9052999999999999e-05 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.247 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785668+0000", 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001754633 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785712+0000", 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4174000000000001e-05 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.248 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:7 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.347490073000003, 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026422899999999998, 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.561e-06 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872426+0000", 2024-09-10T10:27:53.250 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3577999999999999e-05 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872462+0000", 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6145e-05 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872473+0000", 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1109e-05 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872543+0000", 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9319000000000005e-05 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872566+0000", 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3768999999999999e-05 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875010+0000", 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024433910000000001 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.252 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875053+0000", 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3417999999999997e-05 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:8 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.346801683999999, 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0044428820000000004, 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.253 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.254 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873100+0000", 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3030000000000005e-06 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873121+0000", 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1471000000000001e-05 2024-09-10T10:27:53.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873129+0000", 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7339999999999997e-06 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048786499999999998 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873638+0000", 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0735000000000001e-05 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877493+0000", 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0038550189999999999 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877542+0000", 2024-09-10T10:27:53.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9135000000000003e-05 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:17 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.346584350999997, 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042663060000000001, 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873318+0000", 2024-09-10T10:27:53.259 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7999999999999993e-07 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873326+0000", 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8279999999999992e-06 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873341+0000", 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4688999999999999e-05 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873348+0000", 2024-09-10T10:27:53.260 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4810000000000003e-06 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873843+0000", 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000495083 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873860+0000", 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7609999999999999e-05 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.261 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877545+0000", 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036848839999999998 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877583+0000", 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7750999999999998e-05 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:18 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.345862314000001, 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035807780000000002, 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:53.263 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874045+0000", 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9909999999999999e-06 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.264 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874055+0000", 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0147e-05 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874070+0000", 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4184e-05 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874123+0000", 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3504999999999997e-05 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874141+0000", 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7524999999999999e-05 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877586+0000", 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034458700000000002 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877619+0000", 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2962000000000001e-05 2024-09-10T10:27:53.266 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.18 epoch 26/18 rep_tid 5 entries 26'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.9:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.344800669000001, 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048872359999999997, 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.267 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875101+0000", 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:27:53.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875110+0000", 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2609999999999999e-06 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875129+0000", 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8685000000000002e-05 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.269 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875143+0000", 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3723000000000001e-05 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875222+0000", 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9498000000000004e-05 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879987+0000", 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047652800000000002 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.270 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:7 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'5, mlcod=26'3)", 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 39.270038372000002, 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031425310000000001, 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.271 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1100000000000005e-07 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949871+0000", 2024-09-10T10:27:53.272 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8620000000000003e-06 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949885+0000", 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4072e-05 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949893+0000", 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5769999999999996e-06 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.273 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949953+0000", 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.978e-05 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949972+0000", 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9188e-05 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.952959+0000", 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029873859999999999 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.274 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953005+0000", 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5855000000000001e-05 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:8 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'6, mlcod=26'3)", 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 39.269985783000003, 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034236359999999999, 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.275 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949916+0000", 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6199999999999998e-07 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.276 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949922+0000", 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5390000000000004e-06 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949929+0000", 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4339999999999997e-06 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949934+0000", 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2159999999999997e-06 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.277 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950192+0000", 2024-09-10T10:27:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00025864699999999998 2024-09-10T10:27:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950213+0000", 2024-09-10T10:27:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0761999999999999e-05 2024-09-10T10:27:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953297+0000", 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030842999999999999 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953339+0000", 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1276000000000003e-05 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:34 2.11 e26/18 2:8975f766:::202.00000001:head v 26'3, mlcod=26'1)", 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 39.266012944000003, 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020619190000000002, 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953889+0000", 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.879999999999999e-07 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953895+0000", 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2249999999999997e-06 2024-09-10T10:27:53.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953906+0000", 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953912+0000", 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4840000000000003e-06 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953959+0000", 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7768000000000001e-05 2024-09-10T10:27:53.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953976+0000", 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6365999999999999e-05 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955917+0000", 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001941524 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955950+0000", 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2573999999999997e-05 2024-09-10T10:27:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:4 2.1a e26/18 2:5fdc322d:::61e.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 33.296945416, 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0018931340000000001, 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.336 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922956+0000", 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-07 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922962+0000", 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1890000000000004e-06 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922971+0000", 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922978+0000", 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7830000000000001e-06 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923032+0000", 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3637999999999997e-05 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923049+0000", 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7249999999999999e-05 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924803+0000", 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001753788 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924849+0000", 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6050000000000001e-05 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:20 2.11 e26/18 2:890f50f7:::626.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 33.296837709999998, 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003328012, 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.340 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923064+0000", 2024-09-10T10:27:53.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9900000000000001e-07 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923070+0000", 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7499999999999997e-06 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923079+0000", 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9080000000000001e-06 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923089+0000", 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0078000000000001e-05 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923150+0000", 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0300999999999997e-05 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923507+0000", 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00035713599999999998 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926368+0000", 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028615469999999999 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926391+0000", 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2793e-05 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.344 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:23 2.18 e26/18 2:1c1db442:::103.00000000.inode:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 33.296778240999998, 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032793050000000002, 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923124+0000", 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0599999999999999e-07 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923130+0000", 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.81e-06 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.346 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923141+0000", 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923149+0000", 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2679999999999997e-06 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4744000000000002e-05 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.347 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923233+0000", 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9190000000000001e-05 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926353+0000", 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031206419999999999 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926402+0000", 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8755000000000003e-05 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.348 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:13 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'6, mlcod=26'2)", 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 33.296725610999999, 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031818969999999999, 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:27:53.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923176+0000", 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4799999999999998e-07 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923182+0000", 2024-09-10T10:27:53.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.592e-06 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923191+0000", 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4029999999999996e-06 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923197+0000", 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1850000000000001e-06 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.351 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923230+0000", 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3290000000000001e-05 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923247+0000", 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6810999999999999e-05 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926308+0000", 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030606159999999999 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.352 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926357+0000", 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9351999999999998e-05 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.d epoch 26/18 rep_tid 20 entries 26'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.15:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 33.296173113000002, 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034028819999999999, 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.353 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923729+0000", 2024-09-10T10:27:53.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.5900000000000005e-07 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923739+0000", 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8439999999999993e-06 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923761+0000", 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2821999999999999e-05 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923776+0000", 2024-09-10T10:27:53.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4763000000000001e-05 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923889+0000", 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000112518 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927131+0000", 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032419760000000001 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.356 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:14 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'7, mlcod=26'2)", 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 33.295458152000002, 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027184890000000001, 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5099999999999999e-07 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.358 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924449+0000", 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0310000000000004e-06 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924482+0000", 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2845999999999997e-05 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924500+0000", 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.787e-05 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924594+0000", 2024-09-10T10:27:53.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3566999999999994e-05 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924619+0000", 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4978999999999999e-05 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927118+0000", 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024995479999999999 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927161+0000", 2024-09-10T10:27:53.361 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2997000000000002e-05 2024-09-10T10:27:53.361 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.361 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.361 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.361 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.361 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.361 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:53.362 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:27:53.363 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:53.361+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:53.463 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:53.461+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:53.507 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:53.507 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:53.507 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:27:53.507 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:53.516 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:27:53.600 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:53.598+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:53.659 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:53.659 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:53.665 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:53.664+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:53.668 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:27:53.766 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:53.765+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:53.815 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.941459481000003, 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.006807553, 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872290+0000", 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2029999999999999e-06 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872310+0000", 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0508000000000001e-05 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872331+0000", 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0577e-05 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872341+0000", 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0421999999999999e-05 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872457+0000", 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115618 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872575+0000", 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118563 2024-09-10T10:27:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872690+0000", 2024-09-10T10:27:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011438900000000001 2024-09-10T10:27:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875281+0000", 2024-09-10T10:27:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002590995 2024-09-10T10:27:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879035+0000", 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037544879999999998 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879069+0000", 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4034e-05 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879095+0000", 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5755999999999999e-05 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.940262859999997, 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0061030149999999998, 2024-09-10T10:27:53.822 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.823 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873485+0000", 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2419999999999999e-06 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.669e-06 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873505+0000", 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.325e-05 2024-09-10T10:27:53.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873514+0000", 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1949999999999992e-06 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873575+0000", 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873647+0000", 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2346000000000002e-05 2024-09-10T10:27:53.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873778+0000", 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013125600000000001 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875444+0000", 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001665962 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879528+0000", 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040837809999999999 2024-09-10T10:27:53.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879563+0000", 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5329e-05 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879587+0000", 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3947999999999998e-05 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.940208810000001, 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096274789999999996, 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873539+0000", 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0299999999999998e-07 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873554+0000", 2024-09-10T10:27:53.829 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5481999999999998e-05 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873565+0000", 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0733e-05 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873573+0000", 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9270000000000005e-06 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.830 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873633+0000", 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9864000000000001e-05 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873752+0000", 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118946 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873866+0000", 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000113757 2024-09-10T10:27:53.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875462+0000", 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015966699999999999 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883084+0000", 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076214280000000004 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883136+0000", 2024-09-10T10:27:53.832 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2182000000000003e-05 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883166+0000", 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.9787000000000001e-05 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:27:53.833 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.940150838999998, 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097094669999999994, 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:27:53.834 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873597+0000", 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8899999999999998e-07 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873602+0000", 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9230000000000001e-06 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873611+0000", 2024-09-10T10:27:53.835 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9150000000000002e-06 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5239999999999997e-06 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874719+0000", 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0011012769999999999 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.836 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874791+0000", 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2305000000000007e-05 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874962+0000", 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000171099 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877786+0000", 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028241059999999998 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883211+0000", 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054250019999999999 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883263+0000", 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1563000000000001e-05 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883306+0000", 2024-09-10T10:27:53.838 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2864000000000002e-05 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.939230148999997, 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005164681, 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.840 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874518+0000", 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3799999999999996e-07 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874521+0000", 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8550000000000004e-06 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874531+0000", 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8079999999999999e-06 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.841 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874538+0000", 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2760000000000003e-06 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874945+0000", 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000406516 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875045+0000", 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9570999999999999e-05 2024-09-10T10:27:53.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879682+0000", 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046369169999999999 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.939196254000002, 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056651770000000004, 2024-09-10T10:27:53.843 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4600000000000005e-07 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874565+0000", 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4017e-05 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874573+0000", 2024-09-10T10:27:53.845 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9689999999999994e-06 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874678+0000", 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7483000000000002e-05 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874786+0000", 2024-09-10T10:27:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107914 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.880216+0000", 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005429499 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:24 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 39.864887930999998, 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0079888200000000006, 2024-09-10T10:27:53.847 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948861+0000", 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9269999999999999e-06 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948869+0000", 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7749999999999993e-06 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948884+0000", 2024-09-10T10:27:53.849 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5713e-05 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948894+0000", 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2050000000000008e-06 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948958+0000", 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4398000000000002e-05 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.850 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949093+0000", 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013508300000000001 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949223+0000", 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012949500000000001 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953128+0000", 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039058360000000002 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956775+0000", 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036467439999999999 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956812+0000", 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6510000000000001e-05 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956848+0000", 2024-09-10T10:27:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6134000000000003e-05 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:23 2.c e26/18 2:3631e767:::102.00000000.inode:head v 26'3, mlcod=0'0)", 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 39.863403681000001, 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054176620000000002, 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950344+0000", 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9400000000000001e-07 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950362+0000", 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8655999999999999e-05 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950384+0000", 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1929999999999998e-05 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950391+0000", 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7020000000000004e-06 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950624+0000", 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000233031 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950643+0000", 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9459000000000001e-05 2024-09-10T10:27:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955726+0000", 2024-09-10T10:27:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050826389999999999 2024-09-10T10:27:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:27:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4851000000000002e-05 2024-09-10T10:27:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 33.892754695999997, 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0070980829999999998, 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:27:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:27:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920994+0000", 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.697e-06 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921001+0000", 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3719999999999999e-06 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921014+0000", 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2529e-05 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921021+0000", 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9519999999999997e-06 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921097+0000", 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6499000000000001e-05 2024-09-10T10:27:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921261+0000", 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016399599999999999 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921396+0000", 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013448400000000001 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922936+0000", 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015405970000000001 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927988+0000", 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00505158 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928047+0000", 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9011000000000001e-05 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928090+0000", 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3365999999999998e-05 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 33.892703185000002, 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007356853, 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5699999999999998e-07 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921048+0000", 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.754e-06 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921056+0000", 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0469999999999994e-06 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921063+0000", 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5980000000000002e-06 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921747+0000", 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00068475800000000005 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921852+0000", 2024-09-10T10:27:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104347 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922002+0000", 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000150233 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924357+0000", 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0023545599999999999 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928286+0000", 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039299540000000003 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928349+0000", 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2044000000000004e-05 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928401+0000", 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2101000000000001e-05 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 33.892663726999999, 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005625166, 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:27:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921084+0000", 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0100000000000005e-07 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921093+0000", 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9910000000000008e-06 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921099+0000", 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2269999999999998e-06 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921103+0000", 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0929999999999996e-06 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921167+0000", 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4030999999999997e-05 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921301+0000", 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013388699999999999 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921471+0000", 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017000899999999999 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924553+0000", 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030818199999999999 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926600+0000", 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020471339999999999 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926663+0000", 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3274000000000004e-05 2024-09-10T10:27:53.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926708+0000", 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5099000000000003e-05 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 27.870112057, 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010012200000000001, 2024-09-10T10:27:53.875 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943637+0000", 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5390000000000002e-06 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943646+0000", 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1340000000000004e-06 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943661+0000", 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4958e-05 2024-09-10T10:27:53.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943669+0000", 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5629999999999992e-06 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943785+0000", 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011584800000000001 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943960+0000", 2024-09-10T10:27:53.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000174611 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944078+0000", 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118381 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948022+0000", 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039442599999999998 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953546+0000", 2024-09-10T10:27:53.879 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055242449999999997 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953604+0000", 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7555e-05 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953647+0000", 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3105999999999997e-05 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 27.869918498000001, 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099609380000000008, 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943829+0000", 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3099999999999996e-07 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943835+0000", 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8610000000000003e-06 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943846+0000", 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0686999999999999e-05 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943854+0000", 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944413+0000", 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055893399999999997 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944505+0000", 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2788999999999995e-05 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944639+0000", 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013387699999999999 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950701+0000", 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0060612360000000002 2024-09-10T10:27:53.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953700+0000", 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029997689999999998 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953743+0000", 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3024e-05 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953789+0000", 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6109999999999997e-05 2024-09-10T10:27:53.885 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 27.869878476, 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054304089999999998, 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.886 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1399999999999996e-07 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943874+0000", 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3830000000000002e-06 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943881+0000", 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1369999999999996e-06 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943886+0000", 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-06 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944057+0000", 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000170781 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944182+0000", 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012506299999999999 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944309+0000", 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127081 2024-09-10T10:27:53.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.947998+0000", 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003689323 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949225+0000", 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001227005 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949267+0000", 2024-09-10T10:27:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1391999999999999e-05 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949299+0000", 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2169999999999999e-05 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 27.869817994000002, 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0068819399999999996, 2024-09-10T10:27:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943930+0000", 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1200000000000002e-07 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943934+0000", 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0969999999999999e-06 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944782+0000", 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00084778299999999996 2024-09-10T10:27:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944793+0000", 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0993e-05 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944864+0000", 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1241000000000005e-05 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944940+0000", 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6044999999999994e-05 2024-09-10T10:27:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945068+0000", 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012846 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949683+0000", 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046151040000000001 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950706+0000", 2024-09-10T10:27:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010222149999999999 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950764+0000", 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8180999999999997e-05 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950811+0000", 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7108999999999999e-05 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:24 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 27.868444367999999, 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055052950000000003, 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945304+0000", 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4799999999999997e-07 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945311+0000", 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1269999999999997e-06 2024-09-10T10:27:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945322+0000", 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1134e-05 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945330+0000", 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9039999999999992e-06 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945408+0000", 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8200000000000003e-05 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945533+0000", 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000125518 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945659+0000", 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012583599999999999 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949147+0000", 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003487675 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950716+0000", 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015695279999999999 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950767+0000", 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0577e-05 2024-09-10T10:27:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950808+0000", 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0847999999999999e-05 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:25 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 27.867802998999998, 2024-09-10T10:27:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050290129999999997, 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945945+0000", 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1119999999999999e-06 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945952+0000", 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5189999999999998e-06 2024-09-10T10:27:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945965+0000", 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3643e-05 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945974+0000", 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5750000000000001e-06 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0532999999999998e-05 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946192+0000", 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015717699999999999 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946341+0000", 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014897200000000001 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949885+0000", 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035449729999999999 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950896+0000", 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00101023 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950938+0000", 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2284999999999999e-05 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950973+0000", 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4993999999999997e-05 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 27.867749926999998, 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048920780000000002, 2024-09-10T10:27:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945998+0000", 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3900000000000004e-07 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946012+0000", 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4799e-05 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946021+0000", 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.6000000000000007e-06 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946029+0000", 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9209999999999992e-06 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946093+0000", 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3608000000000005e-05 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946242+0000", 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014889699999999999 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950889+0000", 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046476140000000004 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 27.867705900000001, 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047976679999999997, 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946042+0000", 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5199999999999996e-07 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946046+0000", 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1439999999999996e-06 2024-09-10T10:27:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946054+0000", 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1140000000000006e-06 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946070+0000", 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6232e-05 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946137+0000", 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6956000000000001e-05 2024-09-10T10:27:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946260+0000", 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012240299999999999 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950839+0000", 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045791670000000003 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:23 2.1c e28/18 2:399c8d31:::104.00000000.inode:head v 28'5, mlcod=24'1)", 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 27.867186561, 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042418079999999997, 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3e-07 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946574+0000", 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.277e-05 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.918 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946584+0000", 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7710000000000001e-06 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946590+0000", 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6769999999999998e-06 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946666+0000", 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5420999999999995e-05 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946687+0000", 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1665999999999999e-05 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950762+0000", 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040749030000000004 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950802+0000", 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9969999999999998e-05 2024-09-10T10:27:53.920 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.921 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.921 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.921 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:53.921 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:53.921 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:53.922 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:27:53.923 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:53.921+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:54.023 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:54.022+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:54.059 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-10T10:27:54.068 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:54.068 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:54.069 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:27:54.069 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:54.078 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:27:54.124 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:54.123+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:54.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.165+0000 7fa717af8640 1 Processor -- start 2024-09-10T10:27:54.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.165+0000 7fa717af8640 1 -- start start 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.166+0000 7fa717af8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa7101529f0 0x7fa710158890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.166+0000 7fa717af8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa710070b90 0x7fa7100ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.166+0000 7fa717af8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa71006ff70 con 0x7fa710152dd0 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.166+0000 7fa717af8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa7101597e0 con 0x7fa7101529f0 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.166+0000 7fa717af8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa710158f40 con 0x7fa710070b90 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.166+0000 7fa715894640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa7101529f0 0x7fa710158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.166+0000 7fa715093640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fa710152dd0 0x7fa71006f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57290/0 (socket says 172.21.15.120:57290) 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.166+0000 7fa715894640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa7101529f0 0x7fa710158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42356/0 (socket says 172.21.15.120:42356) 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.166+0000 7fa715093640 1 -- 172.21.15.120:0/3941087446 learned_addr learned my addr 172.21.15.120:0/3941087446 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.166+0000 7fa716095640 1 --2- 172.21.15.120:0/3941087446 >> v2:172.21.15.134:3300/0 conn(0x7fa710070b90 0x7fa7100ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:54.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.167+0000 7fa715894640 1 -- 172.21.15.120:0/3941087446 >> v2:172.21.15.134:3300/0 conn(0x7fa710070b90 msgr2=0x7fa7100ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:54.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.167+0000 7fa715894640 1 --2- 172.21.15.120:0/3941087446 >> v2:172.21.15.134:3300/0 conn(0x7fa710070b90 0x7fa7100ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:54.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.167+0000 7fa715894640 1 -- 172.21.15.120:0/3941087446 >> v1:172.21.15.120:6789/0 conn(0x7fa710152dd0 legacy=0x7fa71006f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:54.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.167+0000 7fa715894640 1 -- 172.21.15.120:0/3941087446 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa710159220 con 0x7fa7101529f0 2024-09-10T10:27:54.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.167+0000 7fa714892640 1 -- 172.21.15.120:0/3941087446 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 601157673 0 0) 0x7fa71006ff70 con 0x7fa710152dd0 2024-09-10T10:27:54.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.167+0000 7fa715894640 1 --2- 172.21.15.120:0/3941087446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 0x7fa710158890 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa710070af0 tx=0x7fa70c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76fef562dd7f064 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:54.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.167+0000 7fa714892640 1 -- 172.21.15.120:0/3941087446 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c037070 con 0x7fa7101529f0 2024-09-10T10:27:54.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.167+0000 7fa714892640 1 -- 172.21.15.120:0/3941087446 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70c032440 con 0x7fa7101529f0 2024-09-10T10:27:54.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.168+0000 7fa714892640 1 -- 172.21.15.120:0/3941087446 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c032810 con 0x7fa7101529f0 2024-09-10T10:27:54.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.168+0000 7fa717af8640 1 -- 172.21.15.120:0/3941087446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 msgr2=0x7fa710158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:54.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.168+0000 7fa717af8640 1 --2- 172.21.15.120:0/3941087446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 0x7fa710158890 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa710070af0 tx=0x7fa70c02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:27:54.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.168+0000 7fa717af8640 1 -- 172.21.15.120:0/3941087446 shutdown_connections 2024-09-10T10:27:54.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.168+0000 7fa717af8640 1 --2- 172.21.15.120:0/3941087446 >> v2:172.21.15.134:3300/0 conn(0x7fa710070b90 0x7fa7100ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:54.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.168+0000 7fa717af8640 1 --2- 172.21.15.120:0/3941087446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 0x7fa710158890 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:54.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.168+0000 7fa717af8640 1 -- 172.21.15.120:0/3941087446 >> 172.21.15.120:0/3941087446 conn(0x7fa7100715e0 msgr2=0x7fa7100719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:54.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.168+0000 7fa717af8640 1 -- 172.21.15.120:0/3941087446 shutdown_connections 2024-09-10T10:27:54.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.168+0000 7fa717af8640 1 -- 172.21.15.120:0/3941087446 wait complete. 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.169+0000 7fa717af8640 1 Processor -- start 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.169+0000 7fa717af8640 1 -- start start 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa717af8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa710070b90 0x7fa71014d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa717af8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 0x7fa7101f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa717af8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa710152dd0 0x7fa7101f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa717af8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa71006f340 con 0x7fa7101529f0 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa717af8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa71006f060 con 0x7fa710152dd0 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa717af8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa7100c8f20 con 0x7fa710070b90 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa715093640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 0x7fa7101f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa715093640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 0x7fa7101f3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42370/0 (socket says 172.21.15.120:42370) 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa715093640 1 -- 172.21.15.120:0/4080216103 learned_addr learned my addr 172.21.15.120:0/4080216103 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa716095640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa710152dd0 0x7fa7101f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:54.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.170+0000 7fa715894640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa710070b90 0x7fa71014d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa715093640 1 -- 172.21.15.120:0/4080216103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa710070b90 msgr2=0x7fa71014d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa715093640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa710070b90 0x7fa71014d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa715093640 1 -- 172.21.15.120:0/4080216103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa710152dd0 msgr2=0x7fa7101f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa715093640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa710152dd0 0x7fa7101f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa715093640 1 -- 172.21.15.120:0/4080216103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71006d300 con 0x7fa7101529f0 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa715093640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 0x7fa7101f3b00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa70000c430 tx=0x7fa70000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa7067fc640 1 -- 172.21.15.120:0/4080216103 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7000074e0 con 0x7fa7101529f0 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa717af8640 1 -- 172.21.15.120:0/4080216103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7100cd890 con 0x7fa7101529f0 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa7067fc640 1 -- 172.21.15.120:0/4080216103 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7000047e0 con 0x7fa7101529f0 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa7067fc640 1 -- 172.21.15.120:0/4080216103 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700004bd0 con 0x7fa7101529f0 2024-09-10T10:27:54.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.171+0000 7fa717af8640 1 -- 172.21.15.120:0/4080216103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71006ea00 con 0x7fa7101529f0 2024-09-10T10:27:54.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.172+0000 7fa717af8640 1 -- 172.21.15.120:0/4080216103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7100caa40 con 0x7fa7101529f0 2024-09-10T10:27:54.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.173+0000 7fa7067fc640 1 -- 172.21.15.120:0/4080216103 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa700011070 con 0x7fa7101529f0 2024-09-10T10:27:54.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.173+0000 7fa7067fc640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa6f406be30 0x7fa6f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:54.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.174+0000 7fa715894640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa6f406be30 0x7fa6f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:54.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.174+0000 7fa7067fc640 1 -- 172.21.15.120:0/4080216103 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6735+0+0 (secure 0 0 0) 0x7fa70008ae20 con 0x7fa7101529f0 2024-09-10T10:27:54.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.174+0000 7fa715894640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa6f406be30 0x7fa6f406e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa70c034000 tx=0x7fa70c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:54.181 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:54.181 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:54.182 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:54.182 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:54.182 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:54.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.182+0000 7fa7067fc640 1 -- 172.21.15.120:0/4080216103 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa7100caa40 con 0x7fa7101529f0 2024-09-10T10:27:54.190 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:27:54.225 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:54.224+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:54.295 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.513872259999999, 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.513541759999995, 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:27:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:27:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:27:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:27:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:27:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:27:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:27:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:27:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:27:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.308 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.308 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:27:54.308 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:27:54.308 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.308 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.308 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.308 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.308 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:54.308 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.513495216999999, 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.309 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.310 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.310 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:54.310 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.310 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.310 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:27:54.310 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:27:54.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.311 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.311 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.311 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:27:54.311 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:27:54.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.311 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.311 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.311 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:27:54.312 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:27:54.313 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 45.422053005000002, 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.314 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:27:54.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.316 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.317 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:27:54.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.319 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 45.421971661999997, 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.320 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:27:54.321 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:27:54.322 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.323 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.324 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 45.421945106999999, 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.325 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.326 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:27:54.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.328 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.329 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:27:54.330 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 45.421802458000002, 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.331 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:54.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:27:54.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.335 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.336 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 45.419155525000001, 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:27:54.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:27:54.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.340 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:54.341 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 45.418304784999997, 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.344 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.344 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:27:54.344 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:27:54.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.344 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.344 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:27:54.344 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:27:54.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:27:54.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:54.346 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 40.346153102000002, 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.347 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:27:54.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:27:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 40.346103966000001, 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:27:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 40.345992488, 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:27:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:27:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:27:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:27:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 34.372591968999998, 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:27:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 34.372561798, 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:27:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:27:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:27:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:27:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 34.372505306000001, 2024-09-10T10:27:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:27:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.377 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.377 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:27:54.377 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:27:54.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.377 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.377 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:27:54.377 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:27:54.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.378 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:54.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:27:54.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 34.372404138, 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.382 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:27:54.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:27:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:27:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:27:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:27:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:27:54.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.385 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.385 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:27:54.385 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:27:54.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.385 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:27:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:27:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:27:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:27:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.386 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.387 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:27:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 28.352137040999999, 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:27:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:27:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 28.348071822000001, 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:27:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 28.348053952000001, 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:27:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:27:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 28.348007714000001, 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:27:54.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.408 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.409 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.409 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:54.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:54.368+0000 7fa717af8640 1 -- 172.21.15.120:0/4080216103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fa7100c8050 con 0x7fa7101529f0 2024-09-10T10:27:54.410 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:54.409+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:54.411 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:27:54.511 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:54.510+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:54.514 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:54.514 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:54.515 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:27:54.515 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:54.523 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:27:54.612 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:54.611+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:54.626 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:54.626 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:54.626 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:54.626 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:54.626 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:54.635 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:27:54.713 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:54.712+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:54.739 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:54.739 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.641224683999994, 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.17492211699999999, 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 1 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096318+0000", 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.657e-06 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096326+0000", 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0560000000000005e-06 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096339+0000", 2024-09-10T10:27:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2965999999999999e-05 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096349+0000", 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0784e-05 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096432+0000", 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2657999999999994e-05 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op must wait for map", 2024-09-10T10:27:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096441+0000", 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4590000000000004e-06 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096935+0000", 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049331899999999998 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for peered", 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096944+0000", 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9539999999999993e-06 2024-09-10T10:27:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267505+0000", 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.170561622 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267700+0000", 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019420400000000001 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267923+0000", 2024-09-10T10:27:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000223451 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.270575+0000", 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026522030000000001 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271131+0000", 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055599000000000002 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.746 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271191+0000", 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9947000000000001e-05 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271238+0000", 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6847000000000001e-05 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.747 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.252390439999999, 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078035889999999997, 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 115 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.748 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1300000000000001e-07 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.749 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485209+0000", 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8628999999999997e-05 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485214+0000", 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0849999999999996e-06 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485218+0000", 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3840000000000001e-06 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.750 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490185+0000", 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0049675270000000002 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490231+0000", 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5488000000000002e-05 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490307+0000", 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6038000000000006e-05 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491263+0000", 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095625099999999998 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492908+0000", 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001645355 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492934+0000", 2024-09-10T10:27:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5431000000000001e-05 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492954+0000", 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0088000000000001e-05 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:54.753 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.252315951, 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077741099999999999, 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 116 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:54.754 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1100000000000002e-07 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.755 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485227+0000", 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1890000000000001e-06 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485231+0000", 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4010000000000001e-06 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485233+0000", 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.728e-06 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490628+0000", 2024-09-10T10:27:54.756 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0053952109999999996 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490674+0000", 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5946999999999999e-05 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490750+0000", 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5227000000000003e-05 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.757 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491443+0000", 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069346800000000004 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492959+0000", 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015154470000000001 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492983+0000", 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.406e-05 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.758 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492999+0000", 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6121e-05 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.252303823000005, 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078059490000000004, 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.759 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 117 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.23e-07 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485240+0000", 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780000000000001e-06 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.761 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485243+0000", 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8430000000000003e-06 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485247+0000", 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2279999999999998e-06 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490995+0000", 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057480730000000002 2024-09-10T10:27:54.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491043+0000", 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7936000000000003e-05 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491113+0000", 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0872999999999997e-05 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.763 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491636+0000", 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00052242300000000005 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493002+0000", 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001365808 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493025+0000", 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2932999999999999e-05 2024-09-10T10:27:54.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493043+0000", 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8031000000000001e-05 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919948680999994, 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095639920000000003, 2024-09-10T10:27:54.765 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 131 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.766 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1199999999999998e-07 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817663+0000", 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1099000000000005e-05 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.767 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817670+0000", 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.8539999999999999e-06 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817674+0000", 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.574e-06 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818465+0000", 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000791445 2024-09-10T10:27:54.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818509+0000", 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3569000000000002e-05 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818580+0000", 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1503999999999998e-05 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821292+0000", 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027120529999999999 2024-09-10T10:27:54.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827108+0000", 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0058160759999999999 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827134+0000", 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5859000000000001e-05 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827156+0000", 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1546999999999999e-05 2024-09-10T10:27:54.770 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919859473000002, 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095198769999999995, 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.771 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 132 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2899999999999999e-07 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817683+0000", 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0379999999999998e-06 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817690+0000", 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3110000000000002e-06 2024-09-10T10:27:54.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817699+0000", 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820828+0000", 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031296829999999999 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.774 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820873+0000", 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4869e-05 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820943+0000", 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9919999999999995e-05 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823683+0000", 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027395140000000002 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.775 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827161+0000", 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003477857 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827183+0000", 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2612999999999998e-05 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827201+0000", 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7842999999999998e-05 2024-09-10T10:27:54.776 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919792165999993, 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0094944009999999995, 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.777 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 133 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.778 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817749+0000", 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2399999999999998e-07 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817752+0000", 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1970000000000001e-06 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817760+0000", 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8080000000000005e-06 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.779 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817764+0000", 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9419999999999997e-06 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821319+0000", 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035547740000000001 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821364+0000", 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5398000000000001e-05 2024-09-10T10:27:54.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821434+0000", 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0382999999999993e-05 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823717+0000", 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022825089999999998 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827205+0000", 2024-09-10T10:27:54.781 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034879009999999998 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827225+0000", 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0611999999999998e-05 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827243+0000", 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7353000000000001e-05 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919739324999995, 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095863660000000007, 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 134 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.783 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.784 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817802+0000", 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8999999999999996e-07 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817871+0000", 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9572000000000004e-05 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817878+0000", 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2079999999999997e-06 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817882+0000", 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5889999999999996e-06 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821625+0000", 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037431069999999999 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821669+0000", 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3455000000000003e-05 2024-09-10T10:27:54.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821740+0000", 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1390999999999995e-05 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823826+0000", 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020863399999999999 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827343+0000", 2024-09-10T10:27:54.787 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003516267 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827369+0000", 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.6044999999999999e-05 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827388+0000", 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8802000000000001e-05 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919646630000003, 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095738179999999996, 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 135 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.789 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5999999999999999e-07 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817897+0000", 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0149999999999999e-06 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817904+0000", 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5089999999999999e-06 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817909+0000", 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3120000000000001e-06 2024-09-10T10:27:54.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823862+0000", 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059524199999999999 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823911+0000", 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9323000000000002e-05 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823983+0000", 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1736000000000006e-05 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827101+0000", 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031181009999999999 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827392+0000", 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00029090700000000003 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827426+0000", 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4127e-05 2024-09-10T10:27:54.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827468+0000", 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1907999999999999e-05 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919619613999998, 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.009594488, 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.794 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 136 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.795 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3700000000000001e-07 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817925+0000", 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4659999999999999e-06 2024-09-10T10:27:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817931+0000", 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5710000000000003e-06 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817936+0000", 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7550000000000002e-06 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824178+0000", 2024-09-10T10:27:54.797 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062415700000000001 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824225+0000", 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6966999999999999e-05 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824296+0000", 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1284000000000004e-05 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827290+0000", 2024-09-10T10:27:54.798 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002993697 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827473+0000", 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018376699999999999 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827497+0000", 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3445e-05 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.799 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827515+0000", 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8529000000000002e-05 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919587925000002, 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096066220000000004, 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.800 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 137 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.801 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3000000000000001e-07 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818004+0000", 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0902000000000002e-05 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.802 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818013+0000", 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5420000000000005e-06 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818019+0000", 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7939999999999997e-06 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824489+0000", 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0064692580000000003 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824532+0000", 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3631000000000003e-05 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824601+0000", 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9028000000000004e-05 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827334+0000", 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027325409999999998 2024-09-10T10:27:54.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827520+0000", 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018642499999999999 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827541+0000", 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1151000000000001e-05 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.805 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827559+0000", 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7819999999999999e-05 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919508604000001, 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010100303, 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.806 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 138 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818033+0000", 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818037+0000", 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0419999999999996e-06 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.808 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818045+0000", 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9759999999999995e-06 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818052+0000", 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5909999999999998e-06 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826594+0000", 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0085421660000000003 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.809 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826671+0000", 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7121999999999998e-05 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826795+0000", 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012315199999999999 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828081+0000", 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0012869139999999999 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.810 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828088+0000", 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9870000000000002e-06 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828112+0000", 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3405e-05 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828132+0000", 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0394000000000002e-05 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.811 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919478233000007, 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01125819, 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.812 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 139 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818063+0000", 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7800000000000002e-07 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818067+0000", 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8500000000000004e-06 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.814 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818075+0000", 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818081+0000", 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1829999999999999e-06 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827594+0000", 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095129340000000007 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827669+0000", 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5239000000000002e-05 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827789+0000", 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000119757 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829195+0000", 2024-09-10T10:27:54.816 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014064279999999999 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829232+0000", 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7058999999999999e-05 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829281+0000", 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8884000000000002e-05 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.817 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829321+0000", 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9527999999999998e-05 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919444163999998, 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012263886, 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.818 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 140 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818097+0000", 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8999999999999997e-07 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818145+0000", 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8040000000000001e-05 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818160+0000", 2024-09-10T10:27:54.820 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4671999999999999e-05 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818169+0000", 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1900000000000001e-06 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828158+0000", 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099891440000000001 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.821 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828228+0000", 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0407000000000005e-05 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828343+0000", 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000114821 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829914+0000", 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001570866 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.822 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830312+0000", 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00039770499999999999 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830339+0000", 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.69e-05 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830360+0000", 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1651e-05 2024-09-10T10:27:54.823 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.919356922999995, 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012742035000000001, 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.824 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 141 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7700000000000004e-07 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818188+0000", 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8659999999999999e-06 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818196+0000", 2024-09-10T10:27:54.826 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9480000000000008e-06 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818205+0000", 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8570000000000001e-06 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828883+0000", 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010678600999999999 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.827 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828927+0000", 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3887999999999997e-05 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829001+0000", 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3442000000000006e-05 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830548+0000", 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001547194 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.828 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830884+0000", 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00033638100000000002 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830907+0000", 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3051999999999999e-05 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830926+0000", 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8229000000000001e-05 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.829 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.906209185999998, 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097270880000000001, 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 145 2024-09-10T10:27:54.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831332+0000", 2024-09-10T10:27:54.831 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8900000000000001e-07 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831379+0000", 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7134e-05 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831387+0000", 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7409999999999992e-06 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831391+0000", 2024-09-10T10:27:54.832 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6190000000000002e-06 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833155+0000", 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017641969999999999 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833237+0000", 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1957000000000002e-05 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:54.833 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833356+0000", 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011843100000000001 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.834068+0000", 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00071179000000000001 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841004+0000", 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069359749999999996 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.834 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841034+0000", 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.042e-05 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841058+0000", 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4535000000000001e-05 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:20 2.9 e26/18 2:92e99a1d:::612.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:54.835 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 45.862012921999998, 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074964979999999999, 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.836 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875534+0000", 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9109999999999998e-06 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875543+0000", 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8319999999999995e-06 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875549+0000", 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2380000000000002e-06 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875699+0000", 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014945900000000001 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.838 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875717+0000", 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8584e-05 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882982+0000", 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0072648039999999997 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883024+0000", 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2135000000000002e-05 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.839 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:27 2.9 e26/18 2:91bd965c:::501.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 45.861952287999998, 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074993129999999996, 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:27:54.840 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875589+0000", 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3999999999999997e-07 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.841 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875595+0000", 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9680000000000003e-06 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875602+0000", 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875608+0000", 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0430000000000004e-06 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.842 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875887+0000", 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000279578 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875904+0000", 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6439e-05 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883027+0000", 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0071234480000000001 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.843 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883088+0000", 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0424000000000003e-05 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 28.795873211, 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01193195, 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.844 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:27:54.845 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2399999999999999e-07 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941671+0000", 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8329999999999998e-06 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2919999999999998e-06 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.846 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941686+0000", 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944874+0000", 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031881639999999998 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945305+0000", 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00043084599999999997 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945438+0000", 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013344600000000001 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950653+0000", 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005214733 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953496+0000", 2024-09-10T10:27:54.848 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028433989999999999 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953554+0000", 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7933000000000003e-05 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953599+0000", 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5130999999999999e-05 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 28.795847264999999, 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0093592050000000006, 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941694+0000", 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7899999999999999e-07 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941711+0000", 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7530999999999999e-05 2024-09-10T10:27:54.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941720+0000", 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4279999999999996e-06 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941725+0000", 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2390000000000002e-06 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942690+0000", 2024-09-10T10:27:54.852 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00096541400000000005 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942807+0000", 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000116777 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942967+0000", 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015997599999999999 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.853 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944708+0000", 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001740492 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950956+0000", 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062481580000000002 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951016+0000", 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0299e-05 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:54.854 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:54.855 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951052+0000", 2024-09-10T10:27:54.855 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6412000000000003e-05 2024-09-10T10:27:54.855 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.855 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.855 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.855 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:54.855 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:54.855 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:54.856 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:27:54.857 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:54.855+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:54.957 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:54.956+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:54.961 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:54.961 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:54.970 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:27:55.033 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:27:55.039 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:27:55.039 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:27:55.040 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-10T10:27:55.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-10T10:27:55.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.053+0000 7fa7067fc640 1 -- 172.21.15.120:0/4080216103 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v38) ==== 91+0+0 (secure 0 0 0) 0x7fa700058d00 con 0x7fa7101529f0 2024-09-10T10:27:55.054 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-10T10:27:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.056+0000 7fa717af8640 1 -- 172.21.15.120:0/4080216103 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa6f406be30 msgr2=0x7fa6f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.056+0000 7fa717af8640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa6f406be30 0x7fa6f406e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa70c034000 tx=0x7fa70c02e000 comp rx=0 tx=0).stop 2024-09-10T10:27:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.056+0000 7fa717af8640 1 -- 172.21.15.120:0/4080216103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 msgr2=0x7fa7101f3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.056+0000 7fa717af8640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 0x7fa7101f3b00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa70000c430 tx=0x7fa70000c8f0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.056+0000 7fa717af8640 1 -- 172.21.15.120:0/4080216103 shutdown_connections 2024-09-10T10:27:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.056+0000 7fa717af8640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa6f406be30 0x7fa6f406e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.056+0000 7fa717af8640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa710152dd0 0x7fa7101f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.056+0000 7fa717af8640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7101529f0 0x7fa7101f3b00 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa70000c430 tx=0x7fa70000c8f0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.056+0000 7fa717af8640 1 --2- 172.21.15.120:0/4080216103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa710070b90 0x7fa71014d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.056+0000 7fa717af8640 1 -- 172.21.15.120:0/4080216103 >> 172.21.15.120:0/4080216103 conn(0x7fa7100715e0 msgr2=0x7fa710157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:55.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.057+0000 7fa717af8640 1 -- 172.21.15.120:0/4080216103 shutdown_connections 2024-09-10T10:27:55.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.057+0000 7fa717af8640 1 -- 172.21.15.120:0/4080216103 wait complete. 2024-09-10T10:27:55.059 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:55.057+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:55.083 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:55.083 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:55.083 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:55.083 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:55.083 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:55.093 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:27:55.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.139+0000 7fe20802e640 1 Processor -- start 2024-09-10T10:27:55.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.139+0000 7fe20802e640 1 -- start start 2024-09-10T10:27:55.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe20802e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 0x7fe200163150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe20802e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe20006cf70 con 0x7fe2000c71b0 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe20802e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe20006d250 con 0x7fe200154d50 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe20802e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe20006cd20 con 0x7fe2000c4470 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe2055c9640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fe2000c71b0 0x7fe2000ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57306/0 (socket says 172.21.15.120:57306) 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe2055c9640 1 -- 172.21.15.120:0/908641798 learned_addr learned my addr 172.21.15.120:0/908641798 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe2065cb640 1 --2- 172.21.15.120:0/908641798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 0x7fe200163150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe204dc8640 1 -- 172.21.15.120:0/908641798 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2062245655 0 0) 0x7fe20006cf70 con 0x7fe2000c71b0 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe204dc8640 1 -- 172.21.15.120:0/908641798 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1ec003520 con 0x7fe2000c71b0 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe204dc8640 1 -- 172.21.15.120:0/908641798 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3183708678 0 0) 0x7fe20006d250 con 0x7fe200154d50 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe204dc8640 1 -- 172.21.15.120:0/908641798 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe20006cf70 con 0x7fe200154d50 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe2065cb640 1 -- 172.21.15.120:0/908641798 >> v1:172.21.15.134:6789/0 conn(0x7fe200154d50 legacy=0x7fe20015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe2065cb640 1 -- 172.21.15.120:0/908641798 >> v1:172.21.15.120:6789/0 conn(0x7fe2000c71b0 legacy=0x7fe2000ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.140+0000 7fe2065cb640 1 -- 172.21.15.120:0/908641798 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20006d530 con 0x7fe2000c4470 2024-09-10T10:27:55.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.141+0000 7fe2065cb640 1 --2- 172.21.15.120:0/908641798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 0x7fe200163150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009e40 tx=0x7fe1f802cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c1c4cde3b8482b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.141+0000 7fe204dc8640 1 -- 172.21.15.120:0/908641798 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f803a070 con 0x7fe2000c4470 2024-09-10T10:27:55.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.141+0000 7fe204dc8640 1 -- 172.21.15.120:0/908641798 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f802d960 con 0x7fe2000c4470 2024-09-10T10:27:55.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.141+0000 7fe204dc8640 1 -- 172.21.15.120:0/908641798 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f802dd30 con 0x7fe2000c4470 2024-09-10T10:27:55.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.142+0000 7fe20802e640 1 -- 172.21.15.120:0/908641798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 msgr2=0x7fe200163150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.142+0000 7fe20802e640 1 --2- 172.21.15.120:0/908641798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 0x7fe200163150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009e40 tx=0x7fe1f802cc10 comp rx=0 tx=0).stop 2024-09-10T10:27:55.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.142+0000 7fe20802e640 1 -- 172.21.15.120:0/908641798 shutdown_connections 2024-09-10T10:27:55.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.142+0000 7fe20802e640 1 --2- 172.21.15.120:0/908641798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 0x7fe200163150 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.142+0000 7fe20802e640 1 -- 172.21.15.120:0/908641798 >> 172.21.15.120:0/908641798 conn(0x7fe2000728b0 msgr2=0x7fe200072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:55.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.142+0000 7fe20802e640 1 -- 172.21.15.120:0/908641798 shutdown_connections 2024-09-10T10:27:55.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.142+0000 7fe20802e640 1 -- 172.21.15.120:0/908641798 wait complete. 2024-09-10T10:27:55.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.144+0000 7fe20802e640 1 Processor -- start 2024-09-10T10:27:55.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.144+0000 7fe20802e640 1 -- start start 2024-09-10T10:27:55.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe20802e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 0x7fe2000c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe20802e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2000c71b0 0x7fe2000c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe20802e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200154d50 0x7fe2000c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe20802e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe20015afb0 con 0x7fe200154d50 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe20802e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2001578a0 con 0x7fe2000c71b0 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe20802e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe20006eae0 con 0x7fe2000c4470 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe2065cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200154d50 0x7fe2000c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe2065cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200154d50 0x7fe2000c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42376/0 (socket says 172.21.15.120:42376) 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe2065cb640 1 -- 172.21.15.120:0/3682494459 learned_addr learned my addr 172.21.15.120:0/3682494459 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe2055c9640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2000c71b0 0x7fe2000c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe205dca640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 0x7fe2000c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.145+0000 7fe205dca640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2000c71b0 msgr2=0x7fe2000c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe205dca640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2000c71b0 0x7fe2000c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe205dca640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200154d50 msgr2=0x7fe2000c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe205dca640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200154d50 0x7fe2000c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe205dca640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2001ebba0 con 0x7fe2000c4470 2024-09-10T10:27:55.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe2065cb640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200154d50 0x7fe2000c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:55.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe2055c9640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2000c71b0 0x7fe2000c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:55.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe205dca640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 0x7fe2000c1080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00c3f0 tx=0x7fe1fc00c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc017070 con 0x7fe2000c4470 2024-09-10T10:27:55.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1fc0052f0 con 0x7fe2000c4470 2024-09-10T10:27:55.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2001ef150 con 0x7fe2000c4470 2024-09-10T10:27:55.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc003830 con 0x7fe2000c4470 2024-09-10T10:27:55.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.146+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2001ef410 con 0x7fe2000c4470 2024-09-10T10:27:55.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.147+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe20006dcf0 con 0x7fe2000c4470 2024-09-10T10:27:55.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.148+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1fc01a070 con 0x7fe2000c4470 2024-09-10T10:27:55.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.149+0000 7fe1f67fc640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe1e406be60 0x7fe1e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.149+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 6735+0+0 (secure 0 0 0) 0x7fe1fc08abd0 con 0x7fe2000c4470 2024-09-10T10:27:55.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.149+0000 7fe2055c9640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe1e406be60 0x7fe1e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.149+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v38) ==== 54+0+16 (secure 0 0 0) 0x7fe1fc021070 con 0x7fe2000c4470 2024-09-10T10:27:55.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.149+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fe2001ef5e0 0x7fe200150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.149+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2000c4150 con 0x7fe2001ef5e0 2024-09-10T10:27:55.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.149+0000 7fe2055c9640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe1e406be60 0x7fe1e406e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe20006c9f0 tx=0x7fe1f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.150+0000 7fe2065cb640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fe2001ef5e0 0x7fe200150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.150+0000 7fe2065cb640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fe2001ef5e0 0x7fe200150b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.151+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2000c4150 con 0x7fe2001ef5e0 2024-09-10T10:27:55.158 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:55.156+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.167+0000 7fe84416d640 1 Processor -- start 2024-09-10T10:27:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.167+0000 7fe84416d640 1 -- start start 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe84416d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe83c154d70 0x7fe83c0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe84416d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe83c15e220 0x7fe83c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe84416d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe83c155150 con 0x7fe83c06bc10 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe84416d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe83c15ebd0 con 0x7fe83c154d70 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe84416d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe83c0c49f0 con 0x7fe83c15e220 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe841708640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe83c06bc10 0x7fe83c0cd080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:50176/0 (socket says 172.21.15.120:50176) 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe841f09640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe83c154d70 0x7fe83c0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe841708640 1 -- 172.21.15.120:0/1023303020 learned_addr learned my addr 172.21.15.120:0/1023303020 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe84270a640 1 --2- 172.21.15.120:0/1023303020 >> v2:172.21.15.134:3300/0 conn(0x7fe83c15e220 0x7fe83c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe840f07640 1 -- 172.21.15.120:0/1023303020 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211731701 0 0) 0x7fe83c155150 con 0x7fe83c06bc10 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe840f07640 1 -- 172.21.15.120:0/1023303020 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe828003520 con 0x7fe83c06bc10 2024-09-10T10:27:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe84270a640 1 -- 172.21.15.120:0/1023303020 >> v2:172.21.15.120:3300/0 conn(0x7fe83c154d70 msgr2=0x7fe83c0c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.168+0000 7fe84270a640 1 --2- 172.21.15.120:0/1023303020 >> v2:172.21.15.120:3300/0 conn(0x7fe83c154d70 0x7fe83c0c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.169+0000 7fe84270a640 1 -- 172.21.15.120:0/1023303020 >> v1:172.21.15.134:6789/0 conn(0x7fe83c06bc10 legacy=0x7fe83c0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.169+0000 7fe84270a640 1 -- 172.21.15.120:0/1023303020 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c06c4c0 con 0x7fe83c15e220 2024-09-10T10:27:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.169+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe20006c2b0 con 0x7fe2001ef5e0 2024-09-10T10:27:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.170+0000 7fe84270a640 1 --2- 172.21.15.120:0/1023303020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe83c15e220 0x7fe83c15e690 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe83c073590 tx=0x7fe83402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e7893d48298f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.170+0000 7fe840f07640 1 -- 172.21.15.120:0/1023303020 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834007740 con 0x7fe83c15e220 2024-09-10T10:27:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.170+0000 7fe840f07640 1 -- 172.21.15.120:0/1023303020 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8340078e0 con 0x7fe83c15e220 2024-09-10T10:27:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.170+0000 7fe840f07640 1 -- 172.21.15.120:0/1023303020 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834007cb0 con 0x7fe83c15e220 2024-09-10T10:27:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.171+0000 7fe84416d640 1 -- 172.21.15.120:0/1023303020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe83c15e220 msgr2=0x7fe83c15e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.171+0000 7fe84416d640 1 --2- 172.21.15.120:0/1023303020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe83c15e220 0x7fe83c15e690 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe83c073590 tx=0x7fe83402a710 comp rx=0 tx=0).stop 2024-09-10T10:27:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.171+0000 7fe84416d640 1 -- 172.21.15.120:0/1023303020 shutdown_connections 2024-09-10T10:27:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.171+0000 7fe84416d640 1 --2- 172.21.15.120:0/1023303020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe83c15e220 0x7fe83c15e690 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.171+0000 7fe84416d640 1 --2- 172.21.15.120:0/1023303020 >> v2:172.21.15.120:3300/0 conn(0x7fe83c154d70 0x7fe83c0c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.171+0000 7fe84416d640 1 -- 172.21.15.120:0/1023303020 >> 172.21.15.120:0/1023303020 conn(0x7fe83c067d40 msgr2=0x7fe83c070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.171+0000 7fe84416d640 1 -- 172.21.15.120:0/1023303020 shutdown_connections 2024-09-10T10:27:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.171+0000 7fe84416d640 1 -- 172.21.15.120:0/1023303020 wait complete. 2024-09-10T10:27:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.172+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe20006c2b0 con 0x7fe2001ef5e0 2024-09-10T10:27:55.174 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.172+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fe2000c3690 0x7fe2000cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.172+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe2000c3a70 con 0x7fe2000c3690 2024-09-10T10:27:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.172+0000 7fe84416d640 1 Processor -- start 2024-09-10T10:27:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.172+0000 7fe84416d640 1 -- start start 2024-09-10T10:27:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.172+0000 7fe205dca640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fe2000c3690 0x7fe2000cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.173+0000 7fe205dca640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fe2000c3690 0x7fe2000cbe60 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.173+0000 7fe84416d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe83c06bc10 0x7fe83c15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.173+0000 7fe84416d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe83c154d70 0x7fe83c15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.173+0000 7fe84416d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c15e220 0x7fe83c15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.173+0000 7fe84416d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c06cd20 con 0x7fe83c154d70 2024-09-10T10:27:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.173+0000 7fe84416d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c06ca40 con 0x7fe83c06bc10 2024-09-10T10:27:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.173+0000 7fe84416d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c0c8250 con 0x7fe83c15e220 2024-09-10T10:27:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.173+0000 7fe841708640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe83c154d70 0x7fe83c15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.173+0000 7fe841708640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe83c154d70 0x7fe83c15c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42396/0 (socket says 172.21.15.120:42396) 2024-09-10T10:27:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.173+0000 7fe841708640 1 -- 172.21.15.120:0/442434485 learned_addr learned my addr 172.21.15.120:0/442434485 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe84270a640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c15e220 0x7fe83c15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2000c3a70 con 0x7fe2000c3690 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe841f09640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe83c06bc10 0x7fe83c15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe841708640 1 -- 172.21.15.120:0/442434485 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c15e220 msgr2=0x7fe83c15cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe841708640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c15e220 0x7fe83c15cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe841708640 1 -- 172.21.15.120:0/442434485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe83c06bc10 msgr2=0x7fe83c15be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe841708640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe83c06bc10 0x7fe83c15be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe841708640 1 -- 172.21.15.120:0/442434485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c1e5260 con 0x7fe83c154d70 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe841708640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe83c154d70 0x7fe83c15c9e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00c430 tx=0x7fe82c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe8327fc640 1 -- 172.21.15.120:0/442434485 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c017070 con 0x7fe83c154d70 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe8327fc640 1 -- 172.21.15.120:0/442434485 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe82c005ce0 con 0x7fe83c154d70 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe84416d640 1 -- 172.21.15.120:0/442434485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c1e42f0 con 0x7fe83c154d70 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.174+0000 7fe84416d640 1 -- 172.21.15.120:0/442434485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c1e5d00 con 0x7fe83c154d70 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.175+0000 7fe8327fc640 1 -- 172.21.15.120:0/442434485 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0072d0 con 0x7fe83c154d70 2024-09-10T10:27:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.176+0000 7fe84416d640 1 -- 172.21.15.120:0/442434485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe824005290 con 0x7fe83c154d70 2024-09-10T10:27:55.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.176+0000 7fe8327fc640 1 -- 172.21.15.120:0/442434485 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe82c01a070 con 0x7fe83c154d70 2024-09-10T10:27:55.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.177+0000 7fe8327fc640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe82006bf00 0x7fe82006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.177+0000 7fe8327fc640 1 -- 172.21.15.120:0/442434485 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6735+0+0 (secure 0 0 0) 0x7fe82c08a600 con 0x7fe83c154d70 2024-09-10T10:27:55.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.177+0000 7fe841f09640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe82006bf00 0x7fe82006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.177+0000 7fe841f09640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe82006bf00 0x7fe82006e3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe838009ad0 tx=0x7fe838008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.181+0000 7fe8327fc640 1 -- 172.21.15.120:0/442434485 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe82c0584b0 con 0x7fe83c154d70 2024-09-10T10:27:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.203+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe2000c7df0 con 0x7fe2000c3690 2024-09-10T10:27:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.204+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2000c7df0 con 0x7fe2000c3690 2024-09-10T10:27:55.206 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:55.206 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:55.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.205+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fe2000cc8b0 0x7fe200062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.205+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe200062b70 con 0x7fe2000cc8b0 2024-09-10T10:27:55.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.205+0000 7fe2055c9640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fe2000cc8b0 0x7fe200062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.205+0000 7fe2055c9640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fe2000cc8b0 0x7fe200062630 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.206+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe200062b70 con 0x7fe2000cc8b0 2024-09-10T10:27:55.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.233+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe200062fd0 con 0x7fe2000cc8b0 2024-09-10T10:27:55.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.235+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe200062fd0 con 0x7fe2000cc8b0 2024-09-10T10:27:55.236 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:55.237 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:55.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.235+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fe200152690 0x7fe200152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.235+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe2000635a0 con 0x7fe200152690 2024-09-10T10:27:55.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.235+0000 7fe2065cb640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fe200152690 0x7fe200152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.236+0000 7fe2065cb640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fe200152690 0x7fe200152af0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.237+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2000635a0 con 0x7fe200152690 2024-09-10T10:27:55.249 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:55.250 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:55.259 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:27:55.259 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:55.258+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:55.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.265+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe20002c640 con 0x7fe200152690 2024-09-10T10:27:55.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.267+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe20002c640 con 0x7fe200152690 2024-09-10T10:27:55.268 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:55.269 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:55.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.267+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe20002cd60 0x7fe20002d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.268+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe20002c640 con 0x7fe20002cd60 2024-09-10T10:27:55.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.268+0000 7fe205dca640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe20002cd60 0x7fe20002d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.268+0000 7fe205dca640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe20002cd60 0x7fe20002d1e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.269+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe20002c640 con 0x7fe20002cd60 2024-09-10T10:27:55.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.289+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe20002c640 con 0x7fe20002cd60 2024-09-10T10:27:55.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.291+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe20002c640 con 0x7fe20002cd60 2024-09-10T10:27:55.292 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:55.292 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:55.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.291+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe200200510 0x7fe200202900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.291+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe2002056d0 con 0x7fe200200510 2024-09-10T10:27:55.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.291+0000 7fe2055c9640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe200200510 0x7fe200202900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.292+0000 7fe2055c9640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe200200510 0x7fe200202900 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.292+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2002056d0 con 0x7fe200200510 2024-09-10T10:27:55.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.308+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe200202e40 con 0x7fe200200510 2024-09-10T10:27:55.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.310+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe200202e40 con 0x7fe200200510 2024-09-10T10:27:55.311 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:55.311 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:55.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.310+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fe200203560 0x7fe200205950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.310+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe2002087a0 con 0x7fe200203560 2024-09-10T10:27:55.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.311+0000 7fe2065cb640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fe200203560 0x7fe200205950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.311+0000 7fe2065cb640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fe200203560 0x7fe200205950 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.312+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2002087a0 con 0x7fe200203560 2024-09-10T10:27:55.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.339+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe200205f10 con 0x7fe200203560 2024-09-10T10:27:55.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.341+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe200205f10 con 0x7fe200203560 2024-09-10T10:27:55.342 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:55.343 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:55.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.341+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fe200206630 0x7fe200206a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.341+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe20020bbc0 con 0x7fe200206630 2024-09-10T10:27:55.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.341+0000 7fe205dca640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fe200206630 0x7fe200206a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.342+0000 7fe205dca640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fe200206630 0x7fe200206a90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.342+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe20020bbc0 con 0x7fe200206630 2024-09-10T10:27:55.359 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:55.359+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:55.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.366+0000 7fe84416d640 1 -- 172.21.15.120:0/442434485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fe824005580 con 0x7fe83c154d70 2024-09-10T10:27:55.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.368+0000 7fe8327fc640 1 -- 172.21.15.120:0/442434485 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v38) ==== 81+0+38860 (secure 0 0 0) 0x7fe82c05c7b0 con 0x7fe83c154d70 2024-09-10T10:27:55.369 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:55.369 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 38, 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-10T10:26:18.018818+0000", 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-10T10:27:55.050117+0000", 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-10T10:26:29.078139+0000", 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-10T10:26:20.091201+0000", 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5804032, 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "nodeep-scrub", 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-10T10:27:55.370 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 4, 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "luminous", 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-10T10:27:55.371 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:30.118649+0000", 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:55.372 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:55.373 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:55.374 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:27:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-10T10:27:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_metadata", 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:38.936606+0000", 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:27:55.378 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:55.379 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "20", 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:55.380 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:55.381 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:55.382 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_bias": 4, 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 16, 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_priority": 5 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: "metadata": "cephfs" 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.384 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data", 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:40.094780+0000", 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:55.385 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 64, 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 64, 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 64, 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 64, 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 64, 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:55.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:55.387 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:55.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:55.389 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.05000000074505806 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.3799999952316284, 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.3799999952316284, 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:27:55.390 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 4, 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data_ec", 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:43.814005+0000", 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 5, 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,ec_overwrites", 2024-09-10T10:27:55.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 3, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 4, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 3, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 1, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:27:55.392 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "26", 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:27:55.393 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:27:55.394 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "cephfs_data_ec", 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:27:55.395 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 8192, 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:27:55.396 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e3b9c3c3-cf7d-4b33-913e-60e928439d8e", 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:55.397 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-10T10:27:55.398 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-10T10:27:55.399 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.400 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.401 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/3664783490", 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/3664783490", 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/3664783490", 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/3664783490", 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "62c20a9d-e6ab-446d-998c-9ee46e46aa90", 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.404 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/3972761964", 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/3972761964", 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/3972761964", 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/3972761964", 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1528f8e0-5e13-4ec7-9d98-0d94b7a505c3", 2024-09-10T10:27:55.408 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.409 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.410 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.411 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.413 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2916410859", 2024-09-10T10:27:55.415 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2916410859", 2024-09-10T10:27:55.416 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2916410859", 2024-09-10T10:27:55.416 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2916410859", 2024-09-10T10:27:55.416 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:55.416 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:55.416 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:55.416 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.416 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "74942f78-c1fc-4886-abf0-950722c5edc6", 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 31, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1224278950", 2024-09-10T10:27:55.421 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1224278950", 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1224278950", 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1224278950", 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e71319d4-5318-4441-84a8-7665ae037596", 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:55.422 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6816", 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:55.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6817", 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6818", 2024-09-10T10:27:55.424 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6819", 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6822", 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6823", 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6820", 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6821", 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6817/1874635282", 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6819/1874635282", 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6823/1874635282", 2024-09-10T10:27:55.427 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6821/1874635282", 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "29e193cd-37af-43e5-bd0c-2f85ce0631e8", 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:55.428 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6808", 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.429 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6809", 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6810", 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.430 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6811", 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6814", 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6815", 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6812", 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6813", 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6809/457724087", 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6811/457724087", 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6815/457724087", 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6813/457724087", 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "55523ec2-cf12-44b3-aeea-f1cedabc996a", 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:55.434 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6824", 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6825", 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6826", 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6827", 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6830", 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6831", 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6828", 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6829", 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6825/50133702", 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6827/50133702", 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6831/50133702", 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6829/50133702", 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "097b1149-3e3c-4e5a-890b-d4cc4361d51b", 2024-09-10T10:27:55.439 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 35, 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.440 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6800", 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6801", 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.441 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6802", 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6803", 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.442 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6806", 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6807", 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6804", 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6805", 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6801/251960875", 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6803/251960875", 2024-09-10T10:27:55.444 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6807/251960875", 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6805/251960875", 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:55.445 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.143056+0000", 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.423968+0000", 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:55.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.640076+0000", 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:55.447 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:55.448 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:55.448 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:55.448 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:55.448 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.536160+0000", 2024-09-10T10:27:55.448 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:55.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.448 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:27:55.448 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.146033+0000", 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:55.449 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.426011+0000", 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.598992+0000", 2024-09-10T10:27:55.450 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.182929+0000", 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:55.451 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [], 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [ 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "2.1d", 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 6, 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 4 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.16", 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 1 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.1d", 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.28", 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.36", 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:27:55.454 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.d", 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 5 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.455 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.1b", 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-10T10:27:55.456 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_data_ec": { 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-device-class": "", 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-num-failure-domains": "0", 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-osds-per-failure-domain": "0", 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-root": "default", 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "jerasure-per-chunk-alignment": "false", 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "2", 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van", 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: "w": "8" 2024-09-10T10:27:55.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [], 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [], 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-10T10:27:55.458 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-10T10:27:55.459 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-10T10:27:55.459 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-10T10:27:55.459 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-10T10:27:55.459 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-10T10:27:55.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:55.459 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:55.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe2002070e0 con 0x7fe200206630 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 -- 172.21.15.120:0/442434485 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe82006bf00 msgr2=0x7fe82006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe82006bf00 0x7fe82006e3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe838009ad0 tx=0x7fe838008040 comp rx=0 tx=0).stop 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 -- 172.21.15.120:0/442434485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe83c154d70 msgr2=0x7fe83c15c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe83c154d70 0x7fe83c15c9e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00c430 tx=0x7fe82c00c8f0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 -- 172.21.15.120:0/442434485 shutdown_connections 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe82006bf00 0x7fe82006e3c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c15e220 0x7fe83c15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe83c154d70 0x7fe83c15c9e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 --2- 172.21.15.120:0/442434485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe83c06bc10 0x7fe83c15be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 -- 172.21.15.120:0/442434485 >> 172.21.15.120:0/442434485 conn(0x7fe83c067d40 msgr2=0x7fe83c159e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:55.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 -- 172.21.15.120:0/442434485 shutdown_connections 2024-09-10T10:27:55.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.370+0000 7fe84416d640 1 -- 172.21.15.120:0/442434485 wait complete. 2024-09-10T10:27:55.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.371+0000 7fe1f67fc640 1 -- 172.21.15.120:0/3682494459 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2002070e0 con 0x7fe200206630 2024-09-10T10:27:55.462 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:55.463 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:55.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fe2001ef5e0 msgr2=0x7fe200150b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fe2001ef5e0 0x7fe200150b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fe2000c3690 msgr2=0x7fe2000cbe60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fe2000c3690 0x7fe2000cbe60 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fe2000cc8b0 msgr2=0x7fe200062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fe2000cc8b0 0x7fe200062630 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fe200152690 msgr2=0x7fe200152af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fe200152690 0x7fe200152af0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe20002cd60 msgr2=0x7fe20002d1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe20002cd60 0x7fe20002d1e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe2065cb640 1 -- 172.21.15.120:0/3682494459 reap_dead start 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe200200510 msgr2=0x7fe200202900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe200200510 0x7fe200202900 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fe200203560 msgr2=0x7fe200205950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fe200203560 0x7fe200205950 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fe200206630 msgr2=0x7fe200206a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fe200206630 0x7fe200206a90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe2065cb640 1 -- 172.21.15.120:0/3682494459 reap_dead start 2024-09-10T10:27:55.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe2065cb640 1 -- 172.21.15.120:0/3682494459 reap_dead start 2024-09-10T10:27:55.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe2065cb640 1 -- 172.21.15.120:0/3682494459 reap_dead start 2024-09-10T10:27:55.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe2065cb640 1 -- 172.21.15.120:0/3682494459 reap_dead start 2024-09-10T10:27:55.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe1e406be60 msgr2=0x7fe1e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe1e406be60 0x7fe1e406e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe20006c9f0 tx=0x7fe1f0009040 comp rx=0 tx=0).stop 2024-09-10T10:27:55.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 msgr2=0x7fe2000c1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 0x7fe2000c1080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00c3f0 tx=0x7fe1fc00c8b0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 shutdown_connections 2024-09-10T10:27:55.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe1e406be60 0x7fe1e406e320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 --2- 172.21.15.120:0/3682494459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2000c4470 0x7fe2000c1080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 >> 172.21.15.120:0/3682494459 conn(0x7fe2000728b0 msgr2=0x7fe200151410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:55.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 shutdown_connections 2024-09-10T10:27:55.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.372+0000 7fe20802e640 1 -- 172.21.15.120:0/3682494459 wait complete. 2024-09-10T10:27:55.466 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:55.466 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:55.468 INFO:tasks.thrashosds.thrasher:j is {'epoch': 38, 'fsid': '7d0d0830-4370-41c3-9acc-b67b07024f68', 'created': '2024-09-10T10:26:18.018818+0000', 'modified': '2024-09-10T10:27:55.050117+0000', 'last_up_change': '2024-09-10T10:26:29.078139+0000', 'last_in_change': '2024-09-10T10:26:20.091201+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:26:30.118649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:26:38.936606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:26:40.094780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T10:26:43.814005+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'e3b9c3c3-cf7d-4b33-913e-60e928439d8e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 3664783490}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 3664783490}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 3664783490}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 3664783490}]}, 'public_addr': '172.21.15.120:6809/3664783490', 'cluster_addr': '172.21.15.120:6811/3664783490', 'heartbeat_back_addr': '172.21.15.120:6815/3664783490', 'heartbeat_front_addr': '172.21.15.120:6813/3664783490', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '62c20a9d-e6ab-446d-998c-9ee46e46aa90', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 3972761964}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 3972761964}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 3972761964}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 3972761964}]}, 'public_addr': '172.21.15.120:6801/3972761964', 'cluster_addr': '172.21.15.120:6803/3972761964', 'heartbeat_back_addr': '172.21.15.120:6807/3972761964', 'heartbeat_front_addr': '172.21.15.120:6805/3972761964', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1528f8e0-5e13-4ec7-9d98-0d94b7a505c3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2916410859}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2916410859}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2916410859}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2916410859}]}, 'public_addr': '172.21.15.120:6817/2916410859', 'cluster_addr': '172.21.15.120:6819/2916410859', 'heartbeat_back_addr': '172.21.15.120:6823/2916410859', 'heartbeat_front_addr': '172.21.15.120:6821/2916410859', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '74942f78-c1fc-4886-abf0-950722c5edc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1224278950}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1224278950}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1224278950}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1224278950}]}, 'public_addr': '172.21.15.120:6825/1224278950', 'cluster_addr': '172.21.15.120:6827/1224278950', 'heartbeat_back_addr': '172.21.15.120:6831/1224278950', 'heartbeat_front_addr': '172.21.15.120:6829/1224278950', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e71319d4-5318-4441-84a8-7665ae037596', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6816', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6817', 'nonce': 1874635282}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6818', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6819', 'nonce': 1874635282}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6822', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6823', 'nonce': 1874635282}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6820', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6821', 'nonce': 1874635282}]}, 'public_addr': '172.21.15.134:6817/1874635282', 'cluster_addr': '172.21.15.134:6819/1874635282', 'heartbeat_back_addr': '172.21.15.134:6823/1874635282', 'heartbeat_front_addr': '172.21.15.134:6821/1874635282', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '29e193cd-37af-43e5-bd0c-2f85ce0631e8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6808', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6809', 'nonce': 457724087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6810', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6811', 'nonce': 457724087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6814', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6815', 'nonce': 457724087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6812', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6813', 'nonce': 457724087}]}, 'public_addr': '172.21.15.134:6809/457724087', 'cluster_addr': '172.21.15.134:6811/457724087', 'heartbeat_back_addr': '172.21.15.134:6815/457724087', 'heartbeat_front_addr': '172.21.15.134:6813/457724087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '55523ec2-cf12-44b3-aeea-f1cedabc996a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6824', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6825', 'nonce': 50133702}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6826', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6827', 'nonce': 50133702}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6830', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6831', 'nonce': 50133702}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6828', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6829', 'nonce': 50133702}]}, 'public_addr': '172.21.15.134:6825/50133702', 'cluster_addr': '172.21.15.134:6827/50133702', 'heartbeat_back_addr': '172.21.15.134:6831/50133702', 'heartbeat_front_addr': '172.21.15.134:6829/50133702', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '097b1149-3e3c-4e5a-890b-d4cc4361d51b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6800', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6801', 'nonce': 251960875}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6802', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6803', 'nonce': 251960875}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6806', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6807', 'nonce': 251960875}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6804', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6805', 'nonce': 251960875}]}, 'public_addr': '172.21.15.134:6801/251960875', 'cluster_addr': '172.21.15.134:6803/251960875', 'heartbeat_back_addr': '172.21.15.134:6807/251960875', 'heartbeat_front_addr': '172.21.15.134:6805/251960875', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.143056+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.423968+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.640076+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.536160+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.146033+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.426011+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.598992+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.182929+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.1d', 'mappings': [{'from': 6, 'to': 4}]}, {'pgid': '3.16', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.1d', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.28', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.36', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'cephfs_data_ec': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '2', 'plugin': 'jerasure', 'technique': 'reed_sol_van', 'w': '8'}, 'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-10T10:27:55.468 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:27:55.469 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:27:55.470 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:27:55.459+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:27:55.561 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:55.560+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:55.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.608+0000 7f8406fb9640 1 Processor -- start 2024-09-10T10:27:55.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.608+0000 7f8406fb9640 1 -- start start 2024-09-10T10:27:55.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.609+0000 7f8406fb9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84001528f0 0x7f84000c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.609+0000 7f8406fb9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84000c7f90 con 0x7f84001592a0 2024-09-10T10:27:55.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.609+0000 7f8406fb9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f840006f690 con 0x7f8400152cd0 2024-09-10T10:27:55.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.609+0000 7f8406fb9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f840006f970 con 0x7f84001528f0 2024-09-10T10:27:55.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.609+0000 7f8405556640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f84001592a0 0x7f8400163040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57312/0 (socket says 172.21.15.120:57312) 2024-09-10T10:27:55.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.609+0000 7f8405556640 1 -- 172.21.15.120:0/3148475563 learned_addr learned my addr 172.21.15.120:0/3148475563 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:55.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.609+0000 7f8404d55640 1 --2- 172.21.15.120:0/3148475563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84001528f0 0x7f84000c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.610+0000 7f83ff7fe640 1 -- 172.21.15.120:0/3148475563 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3127858970 0 0) 0x7f84000c7f90 con 0x7f84001592a0 2024-09-10T10:27:55.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.610+0000 7f83ff7fe640 1 -- 172.21.15.120:0/3148475563 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83f0003520 con 0x7f84001592a0 2024-09-10T10:27:55.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.610+0000 7f83ff7fe640 1 -- 172.21.15.120:0/3148475563 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853774619 0 0) 0x7f840006f690 con 0x7f8400152cd0 2024-09-10T10:27:55.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.610+0000 7f83ff7fe640 1 -- 172.21.15.120:0/3148475563 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84000c7f90 con 0x7f8400152cd0 2024-09-10T10:27:55.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.610+0000 7f8404d55640 1 -- 172.21.15.120:0/3148475563 >> v1:172.21.15.134:6789/0 conn(0x7f8400152cd0 legacy=0x7f8400158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.610+0000 7f8404d55640 1 -- 172.21.15.120:0/3148475563 >> v1:172.21.15.120:6789/0 conn(0x7f84001592a0 legacy=0x7f8400163040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.610+0000 7f83ff7fe640 1 -- 172.21.15.120:0/3148475563 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1792287746 0 0) 0x7f83f0003520 con 0x7f84001592a0 2024-09-10T10:27:55.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.610+0000 7f8404d55640 1 -- 172.21.15.120:0/3148475563 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840006fc50 con 0x7f84001528f0 2024-09-10T10:27:55.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.611+0000 7f8404d55640 1 --2- 172.21.15.120:0/3148475563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84001528f0 0x7f84000c68d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83f40097d0 tx=0x7f83f402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7df5bcfca8bfe195 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.611+0000 7f83ff7fe640 1 -- 172.21.15.120:0/3148475563 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4037070 con 0x7f84001528f0 2024-09-10T10:27:55.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.611+0000 7f83ff7fe640 1 -- 172.21.15.120:0/3148475563 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f402ac70 con 0x7f84001528f0 2024-09-10T10:27:55.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.611+0000 7f83ff7fe640 1 -- 172.21.15.120:0/3148475563 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f80019f0 con 0x7f84001528f0 2024-09-10T10:27:55.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.611+0000 7f8406fb9640 1 -- 172.21.15.120:0/3148475563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84001528f0 msgr2=0x7f84000c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.611+0000 7f8406fb9640 1 --2- 172.21.15.120:0/3148475563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84001528f0 0x7f84000c68d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83f40097d0 tx=0x7f83f402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.612+0000 7f8406fb9640 1 -- 172.21.15.120:0/3148475563 shutdown_connections 2024-09-10T10:27:55.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.612+0000 7f8406fb9640 1 --2- 172.21.15.120:0/3148475563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84001528f0 0x7f84000c68d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.612+0000 7f8406fb9640 1 -- 172.21.15.120:0/3148475563 >> 172.21.15.120:0/3148475563 conn(0x7f8400068140 msgr2=0x7f8400070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:55.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.612+0000 7f8406fb9640 1 -- 172.21.15.120:0/3148475563 shutdown_connections 2024-09-10T10:27:55.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.612+0000 7f8406fb9640 1 -- 172.21.15.120:0/3148475563 wait complete. 2024-09-10T10:27:55.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.613+0000 7f8406fb9640 1 Processor -- start 2024-09-10T10:27:55.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.613+0000 7f8406fb9640 1 -- start start 2024-09-10T10:27:55.615 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:55.615 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8406fb9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84001528f0 0x7f84001536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8406fb9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8400152cd0 0x7f8400153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8406fb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84001592a0 0x7f840014b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8406fb9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f84000c7d40 con 0x7f84001592a0 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8406fb9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84000c7bc0 con 0x7f84001528f0 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8406fb9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f840006f100 con 0x7f8400152cd0 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8405556640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84001592a0 0x7f840014b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8405556640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84001592a0 0x7f840014b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42400/0 (socket says 172.21.15.120:42400) 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8405556640 1 -- 172.21.15.120:0/1342884081 learned_addr learned my addr 172.21.15.120:0/1342884081 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8404d55640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84001528f0 0x7f84001536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f83fffff640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8400152cd0 0x7f8400153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8405556640 1 -- 172.21.15.120:0/1342884081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8400152cd0 msgr2=0x7f8400153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8405556640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8400152cd0 0x7f8400153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8405556640 1 -- 172.21.15.120:0/1342884081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84001528f0 msgr2=0x7f84001536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8405556640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84001528f0 0x7f84001536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8405556640 1 -- 172.21.15.120:0/1342884081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84000c6280 con 0x7f84001592a0 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8404d55640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84001528f0 0x7f84001536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f83fffff640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8400152cd0 0x7f8400153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.614+0000 7f8405556640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84001592a0 0x7f840014b7c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f83f800cc30 tx=0x7f83f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.615+0000 7f83fd7fa640 1 -- 172.21.15.120:0/1342884081 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8017070 con 0x7f84001592a0 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.615+0000 7f83fd7fa640 1 -- 172.21.15.120:0/1342884081 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f800ce40 con 0x7f84001592a0 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.615+0000 7f83fd7fa640 1 -- 172.21.15.120:0/1342884081 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8012640 con 0x7f84001592a0 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.615+0000 7f8406fb9640 1 -- 172.21.15.120:0/1342884081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84000c6820 con 0x7f84001592a0 2024-09-10T10:27:55.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.615+0000 7f8406fb9640 1 -- 172.21.15.120:0/1342884081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84000c5170 con 0x7f84001592a0 2024-09-10T10:27:55.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.616+0000 7f8406fb9640 1 -- 172.21.15.120:0/1342884081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84000c5940 con 0x7f84001592a0 2024-09-10T10:27:55.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.616+0000 7f83fd7fa640 1 -- 172.21.15.120:0/1342884081 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83f801a070 con 0x7f84001592a0 2024-09-10T10:27:55.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.617+0000 7f83fd7fa640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83e406be30 0x7f83e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.617+0000 7f83fd7fa640 1 -- 172.21.15.120:0/1342884081 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6735+0+0 (secure 0 0 0) 0x7f83f808b0f0 con 0x7f84001592a0 2024-09-10T10:27:55.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.617+0000 7f8404d55640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83e406be30 0x7f83e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.617+0000 7f8404d55640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83e406be30 0x7f83e406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f83f4034000 tx=0x7f83f402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.621+0000 7f83fd7fa640 1 -- 172.21.15.120:0/1342884081 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f83f8058fd0 con 0x7f84001592a0 2024-09-10T10:27:55.626 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:27:55.662 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:55.660+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:55.763 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:55.761+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:55.771 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:55.771 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:55.780 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:27:55.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.801+0000 7f8406fb9640 1 -- 172.21.15.120:0/1342884081 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84000ce2b0 con 0x7f83e406be30 2024-09-10T10:27:55.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.809+0000 7f83fd7fa640 1 -- 172.21.15.120:0/1342884081 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332728 (secure 0 0 0) 0x7f84000ce2b0 con 0x7f83e406be30 2024-09-10T10:27:55.813 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:27:55.815 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:27:55.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 -- 172.21.15.120:0/1342884081 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83e406be30 msgr2=0x7f83e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83e406be30 0x7f83e406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f83f4034000 tx=0x7f83f402e000 comp rx=0 tx=0).stop 2024-09-10T10:27:55.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 -- 172.21.15.120:0/1342884081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84001592a0 msgr2=0x7f840014b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84001592a0 0x7f840014b7c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f83f800cc30 tx=0x7f83f800a430 comp rx=0 tx=0).stop 2024-09-10T10:27:55.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 -- 172.21.15.120:0/1342884081 shutdown_connections 2024-09-10T10:27:55.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83e406be30 0x7f83e406e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84001592a0 0x7f840014b7c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8400152cd0 0x7f8400153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 --2- 172.21.15.120:0/1342884081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84001528f0 0x7f84001536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 -- 172.21.15.120:0/1342884081 >> 172.21.15.120:0/1342884081 conn(0x7f8400068140 msgr2=0x7f840014e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:55.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 -- 172.21.15.120:0/1342884081 shutdown_connections 2024-09-10T10:27:55.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.813+0000 7f8406fb9640 1 -- 172.21.15.120:0/1342884081 wait complete. 2024-09-10T10:27:55.826 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-10T10:27:55.057344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603223,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":376,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219872,"kb_used_data":3624,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754848,"statfs":{"total":773094113280,"available":772868964352,"internally_reserved":0,"allocated":3710976,"data_stored":2714814,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.034549"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":58,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.371560+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:27:52.371560+0000","last_peered":"2024-09-10T10:27:52.371560+0000","last_clean":"2024-09-10T10:27:52.371560+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:27:52.371560+0000","last_undegraded":"2024-09-10T10:27:52.371560+0000","last_fullsized":"2024-09-10T10:27:52.371560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370324+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:27:52.370324+0000","last_peered":"2024-09-10T10:27:52.370324+0000","last_clean":"2024-09-10T10:27:52.370324+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:27:52.370324+0000","last_undegraded":"2024-09-10T10:27:52.370324+0000","last_fullsized":"2024-09-10T10:27:52.370324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369345+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:27:52.369345+0000","last_peered":"2024-09-10T10:27:52.369345+0000","last_clean":"2024-09-10T10:27:52.369345+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:27:52.369345+0000","last_undegraded":"2024-09-10T10:27:52.369345+0000","last_fullsized":"2024-09-10T10:27:52.369345+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370458+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:27:52.370458+0000","last_peered":"2024-09-10T10:27:52.370458+0000","last_clean":"2024-09-10T10:27:52.370458+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:27:52.370458+0000","last_undegraded":"2024-09-10T10:27:52.370458+0000","last_fullsized":"2024-09-10T10:27:52.370458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.371417+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:27:52.371417+0000","last_peered":"2024-09-10T10:27:52.371417+0000","last_clean":"2024-09-10T10:27:52.371417+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:27:52.371417+0000","last_undegraded":"2024-09-10T10:27:52.371417+0000","last_fullsized":"2024-09-10T10:27:52.371417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.046086+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:27:52.046086+0000","last_peered":"2024-09-10T10:27:52.046086+0000","last_clean":"2024-09-10T10:27:52.046086+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:27:52.046086+0000","last_undegraded":"2024-09-10T10:27:52.046086+0000","last_fullsized":"2024-09-10T10:27:52.046086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"28'12","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370415+0000","last_change":"2024-09-10T10:27:51.060242+0000","last_active":"2024-09-10T10:27:52.370415+0000","last_peered":"2024-09-10T10:27:52.370415+0000","last_clean":"2024-09-10T10:27:52.370415+0000","last_became_active":"2024-09-10T10:27:51.049148+0000","last_became_peered":"2024-09-10T10:27:51.049148+0000","last_unstale":"2024-09-10T10:27:52.370415+0000","last_undegraded":"2024-09-10T10:27:52.370415+0000","last_fullsized":"2024-09-10T10:27:52.370415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.604144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045552+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:27:52.045552+0000","last_peered":"2024-09-10T10:27:52.045552+0000","last_clean":"2024-09-10T10:27:52.045552+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:27:52.045552+0000","last_undegraded":"2024-09-10T10:27:52.045552+0000","last_fullsized":"2024-09-10T10:27:52.045552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":46,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.044769+0000","last_change":"2024-09-10T10:27:23.955254+0000","last_active":"2024-09-10T10:27:52.044769+0000","last_peered":"2024-09-10T10:27:52.044769+0000","last_clean":"2024-09-10T10:27:52.044769+0000","last_became_active":"2024-09-10T10:27:23.954737+0000","last_became_peered":"2024-09-10T10:27:23.954737+0000","last_unstale":"2024-09-10T10:27:52.044769+0000","last_undegraded":"2024-09-10T10:27:52.044769+0000","last_fullsized":"2024-09-10T10:27:52.044769+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370044+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:27:52.370044+0000","last_peered":"2024-09-10T10:27:52.370044+0000","last_clean":"2024-09-10T10:27:52.370044+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:27:52.370044+0000","last_undegraded":"2024-09-10T10:27:52.370044+0000","last_fullsized":"2024-09-10T10:27:52.370044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.047178+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:27:52.047178+0000","last_peered":"2024-09-10T10:27:52.047178+0000","last_clean":"2024-09-10T10:27:52.047178+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:27:52.047178+0000","last_undegraded":"2024-09-10T10:27:52.047178+0000","last_fullsized":"2024-09-10T10:27:52.047178+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.044591+0000","last_change":"2024-09-10T10:26:45.816587+0000","last_active":"2024-09-10T10:27:52.044591+0000","last_peered":"2024-09-10T10:27:52.044591+0000","last_clean":"2024-09-10T10:27:52.044591+0000","last_became_active":"2024-09-10T10:26:45.816138+0000","last_became_peered":"2024-09-10T10:26:45.816138+0000","last_unstale":"2024-09-10T10:27:52.044591+0000","last_undegraded":"2024-09-10T10:27:52.044591+0000","last_fullsized":"2024-09-10T10:27:52.044591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369855+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:27:52.369855+0000","last_peered":"2024-09-10T10:27:52.369855+0000","last_clean":"2024-09-10T10:27:52.369855+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:27:52.369855+0000","last_undegraded":"2024-09-10T10:27:52.369855+0000","last_fullsized":"2024-09-10T10:27:52.369855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'4","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370203+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:27:52.370203+0000","last_peered":"2024-09-10T10:27:52.370203+0000","last_clean":"2024-09-10T10:27:52.370203+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:27:52.370203+0000","last_undegraded":"2024-09-10T10:27:52.370203+0000","last_fullsized":"2024-09-10T10:27:52.370203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":46,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045535+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:27:52.045535+0000","last_peered":"2024-09-10T10:27:52.045535+0000","last_clean":"2024-09-10T10:27:52.045535+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:27:52.045535+0000","last_undegraded":"2024-09-10T10:27:52.045535+0000","last_fullsized":"2024-09-10T10:27:52.045535+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369635+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:27:52.369635+0000","last_peered":"2024-09-10T10:27:52.369635+0000","last_clean":"2024-09-10T10:27:52.369635+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:27:52.369635+0000","last_undegraded":"2024-09-10T10:27:52.369635+0000","last_fullsized":"2024-09-10T10:27:52.369635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'4","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370001+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:27:52.370001+0000","last_peered":"2024-09-10T10:27:52.370001+0000","last_clean":"2024-09-10T10:27:52.370001+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:27:52.370001+0000","last_undegraded":"2024-09-10T10:27:52.370001+0000","last_fullsized":"2024-09-10T10:27:52.370001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.044833+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:27:52.044833+0000","last_peered":"2024-09-10T10:27:52.044833+0000","last_clean":"2024-09-10T10:27:52.044833+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:27:52.044833+0000","last_undegraded":"2024-09-10T10:27:52.044833+0000","last_fullsized":"2024-09-10T10:27:52.044833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369337+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:27:52.369337+0000","last_peered":"2024-09-10T10:27:52.369337+0000","last_clean":"2024-09-10T10:27:52.369337+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:27:52.369337+0000","last_undegraded":"2024-09-10T10:27:52.369337+0000","last_fullsized":"2024-09-10T10:27:52.369337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":53,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369576+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:27:52.369576+0000","last_peered":"2024-09-10T10:27:52.369576+0000","last_clean":"2024-09-10T10:27:52.369576+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:27:52.369576+0000","last_undegraded":"2024-09-10T10:27:52.369576+0000","last_fullsized":"2024-09-10T10:27:52.369576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370671+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:27:52.370671+0000","last_peered":"2024-09-10T10:27:52.370671+0000","last_clean":"2024-09-10T10:27:52.370671+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:27:52.370671+0000","last_undegraded":"2024-09-10T10:27:52.370671+0000","last_fullsized":"2024-09-10T10:27:52.370671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045272+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:27:52.045272+0000","last_peered":"2024-09-10T10:27:52.045272+0000","last_clean":"2024-09-10T10:27:52.045272+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:27:52.045272+0000","last_undegraded":"2024-09-10T10:27:52.045272+0000","last_fullsized":"2024-09-10T10:27:52.045272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369711+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:27:52.369711+0000","last_peered":"2024-09-10T10:27:52.369711+0000","last_clean":"2024-09-10T10:27:52.369711+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:27:52.369711+0000","last_undegraded":"2024-09-10T10:27:52.369711+0000","last_fullsized":"2024-09-10T10:27:52.369711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.044786+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:27:52.044786+0000","last_peered":"2024-09-10T10:27:52.044786+0000","last_clean":"2024-09-10T10:27:52.044786+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:27:52.044786+0000","last_undegraded":"2024-09-10T10:27:52.044786+0000","last_fullsized":"2024-09-10T10:27:52.044786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.371146+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:27:52.371146+0000","last_peered":"2024-09-10T10:27:52.371146+0000","last_clean":"2024-09-10T10:27:52.371146+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:27:52.371146+0000","last_undegraded":"2024-09-10T10:27:52.371146+0000","last_fullsized":"2024-09-10T10:27:52.371146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'7","reported_seq":53,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370424+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:27:52.370424+0000","last_peered":"2024-09-10T10:27:52.370424+0000","last_clean":"2024-09-10T10:27:52.370424+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:27:52.370424+0000","last_undegraded":"2024-09-10T10:27:52.370424+0000","last_fullsized":"2024-09-10T10:27:52.370424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.044645+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:27:52.044645+0000","last_peered":"2024-09-10T10:27:52.044645+0000","last_clean":"2024-09-10T10:27:52.044645+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:27:52.044645+0000","last_undegraded":"2024-09-10T10:27:52.044645+0000","last_fullsized":"2024-09-10T10:27:52.044645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370066+0000","last_change":"2024-09-10T10:26:41.795085+0000","last_active":"2024-09-10T10:27:52.370066+0000","last_peered":"2024-09-10T10:27:52.370066+0000","last_clean":"2024-09-10T10:27:52.370066+0000","last_became_active":"2024-09-10T10:26:41.794819+0000","last_became_peered":"2024-09-10T10:26:41.794819+0000","last_unstale":"2024-09-10T10:27:52.370066+0000","last_undegraded":"2024-09-10T10:27:52.370066+0000","last_fullsized":"2024-09-10T10:27:52.370066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"28'8","reported_seq":54,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370471+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:27:52.370471+0000","last_peered":"2024-09-10T10:27:52.370471+0000","last_clean":"2024-09-10T10:27:52.370471+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:27:52.370471+0000","last_undegraded":"2024-09-10T10:27:52.370471+0000","last_fullsized":"2024-09-10T10:27:52.370471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369021+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:27:52.369021+0000","last_peered":"2024-09-10T10:27:52.369021+0000","last_clean":"2024-09-10T10:27:52.369021+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:27:52.369021+0000","last_undegraded":"2024-09-10T10:27:52.369021+0000","last_fullsized":"2024-09-10T10:27:52.369021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370265+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:27:52.370265+0000","last_peered":"2024-09-10T10:27:52.370265+0000","last_clean":"2024-09-10T10:27:52.370265+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:27:52.370265+0000","last_undegraded":"2024-09-10T10:27:52.370265+0000","last_fullsized":"2024-09-10T10:27:52.370265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370637+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:27:52.370637+0000","last_peered":"2024-09-10T10:27:52.370637+0000","last_clean":"2024-09-10T10:27:52.370637+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:27:52.370637+0000","last_undegraded":"2024-09-10T10:27:52.370637+0000","last_fullsized":"2024-09-10T10:27:52.370637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":38,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.371043+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:27:52.371043+0000","last_peered":"2024-09-10T10:27:52.371043+0000","last_clean":"2024-09-10T10:27:52.371043+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:27:52.371043+0000","last_undegraded":"2024-09-10T10:27:52.371043+0000","last_fullsized":"2024-09-10T10:27:52.371043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045777+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:27:52.045777+0000","last_peered":"2024-09-10T10:27:52.045777+0000","last_clean":"2024-09-10T10:27:52.045777+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:27:52.045777+0000","last_undegraded":"2024-09-10T10:27:52.045777+0000","last_fullsized":"2024-09-10T10:27:52.045777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370342+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:27:52.370342+0000","last_peered":"2024-09-10T10:27:52.370342+0000","last_clean":"2024-09-10T10:27:52.370342+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:27:52.370342+0000","last_undegraded":"2024-09-10T10:27:52.370342+0000","last_fullsized":"2024-09-10T10:27:52.370342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369722+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:27:52.369722+0000","last_peered":"2024-09-10T10:27:52.369722+0000","last_clean":"2024-09-10T10:27:52.369722+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:27:52.369722+0000","last_undegraded":"2024-09-10T10:27:52.369722+0000","last_fullsized":"2024-09-10T10:27:52.369722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.046003+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:27:52.046003+0000","last_peered":"2024-09-10T10:27:52.046003+0000","last_clean":"2024-09-10T10:27:52.046003+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:27:52.046003+0000","last_undegraded":"2024-09-10T10:27:52.046003+0000","last_fullsized":"2024-09-10T10:27:52.046003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045533+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:27:52.045533+0000","last_peered":"2024-09-10T10:27:52.045533+0000","last_clean":"2024-09-10T10:27:52.045533+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:27:52.045533+0000","last_undegraded":"2024-09-10T10:27:52.045533+0000","last_fullsized":"2024-09-10T10:27:52.045533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370181+0000","last_change":"2024-09-10T10:26:45.814008+0000","last_active":"2024-09-10T10:27:52.370181+0000","last_peered":"2024-09-10T10:27:52.370181+0000","last_clean":"2024-09-10T10:27:52.370181+0000","last_became_active":"2024-09-10T10:26:45.813627+0000","last_became_peered":"2024-09-10T10:26:45.813627+0000","last_unstale":"2024-09-10T10:27:52.370181+0000","last_undegraded":"2024-09-10T10:27:52.370181+0000","last_fullsized":"2024-09-10T10:27:52.370181+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369701+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:27:52.369701+0000","last_peered":"2024-09-10T10:27:52.369701+0000","last_clean":"2024-09-10T10:27:52.369701+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:27:52.369701+0000","last_undegraded":"2024-09-10T10:27:52.369701+0000","last_fullsized":"2024-09-10T10:27:52.369701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"28'5","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.047199+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:27:52.047199+0000","last_peered":"2024-09-10T10:27:52.047199+0000","last_clean":"2024-09-10T10:27:52.047199+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:27:52.047199+0000","last_undegraded":"2024-09-10T10:27:52.047199+0000","last_fullsized":"2024-09-10T10:27:52.047199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370756+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:27:52.370756+0000","last_peered":"2024-09-10T10:27:52.370756+0000","last_clean":"2024-09-10T10:27:52.370756+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:27:52.370756+0000","last_undegraded":"2024-09-10T10:27:52.370756+0000","last_fullsized":"2024-09-10T10:27:52.370756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369443+0000","last_change":"2024-09-10T10:26:41.799224+0000","last_active":"2024-09-10T10:27:52.369443+0000","last_peered":"2024-09-10T10:27:52.369443+0000","last_clean":"2024-09-10T10:27:52.369443+0000","last_became_active":"2024-09-10T10:26:41.798948+0000","last_became_peered":"2024-09-10T10:26:41.798948+0000","last_unstale":"2024-09-10T10:27:52.369443+0000","last_undegraded":"2024-09-10T10:27:52.369443+0000","last_fullsized":"2024-09-10T10:27:52.369443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369856+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:27:52.369856+0000","last_peered":"2024-09-10T10:27:52.369856+0000","last_clean":"2024-09-10T10:27:52.369856+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:27:52.369856+0000","last_undegraded":"2024-09-10T10:27:52.369856+0000","last_fullsized":"2024-09-10T10:27:52.369856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":38,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370869+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:27:52.370869+0000","last_peered":"2024-09-10T10:27:52.370869+0000","last_clean":"2024-09-10T10:27:52.370869+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:27:52.370869+0000","last_undegraded":"2024-09-10T10:27:52.370869+0000","last_fullsized":"2024-09-10T10:27:52.370869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369669+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:27:52.369669+0000","last_peered":"2024-09-10T10:27:52.369669+0000","last_clean":"2024-09-10T10:27:52.369669+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:27:52.369669+0000","last_undegraded":"2024-09-10T10:27:52.369669+0000","last_fullsized":"2024-09-10T10:27:52.369669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":52,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045465+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:27:52.045465+0000","last_peered":"2024-09-10T10:27:52.045465+0000","last_clean":"2024-09-10T10:27:52.045465+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:27:52.045465+0000","last_undegraded":"2024-09-10T10:27:52.045465+0000","last_fullsized":"2024-09-10T10:27:52.045465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369973+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:27:52.369973+0000","last_peered":"2024-09-10T10:27:52.369973+0000","last_clean":"2024-09-10T10:27:52.369973+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:27:52.369973+0000","last_undegraded":"2024-09-10T10:27:52.369973+0000","last_fullsized":"2024-09-10T10:27:52.369973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369811+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:27:52.369811+0000","last_peered":"2024-09-10T10:27:52.369811+0000","last_clean":"2024-09-10T10:27:52.369811+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:27:52.369811+0000","last_undegraded":"2024-09-10T10:27:52.369811+0000","last_fullsized":"2024-09-10T10:27:52.369811+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370208+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:27:52.370208+0000","last_peered":"2024-09-10T10:27:52.370208+0000","last_clean":"2024-09-10T10:27:52.370208+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:27:52.370208+0000","last_undegraded":"2024-09-10T10:27:52.370208+0000","last_fullsized":"2024-09-10T10:27:52.370208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.047447+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:27:52.047447+0000","last_peered":"2024-09-10T10:27:52.047447+0000","last_clean":"2024-09-10T10:27:52.047447+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:27:52.047447+0000","last_undegraded":"2024-09-10T10:27:52.047447+0000","last_fullsized":"2024-09-10T10:27:52.047447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369456+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:27:52.369456+0000","last_peered":"2024-09-10T10:27:52.369456+0000","last_clean":"2024-09-10T10:27:52.369456+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:27:52.369456+0000","last_undegraded":"2024-09-10T10:27:52.369456+0000","last_fullsized":"2024-09-10T10:27:52.369456+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045674+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:27:52.045674+0000","last_peered":"2024-09-10T10:27:52.045674+0000","last_clean":"2024-09-10T10:27:52.045674+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:27:52.045674+0000","last_undegraded":"2024-09-10T10:27:52.045674+0000","last_fullsized":"2024-09-10T10:27:52.045674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370655+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:27:52.370655+0000","last_peered":"2024-09-10T10:27:52.370655+0000","last_clean":"2024-09-10T10:27:52.370655+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:27:52.370655+0000","last_undegraded":"2024-09-10T10:27:52.370655+0000","last_fullsized":"2024-09-10T10:27:52.370655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.371348+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:27:52.371348+0000","last_peered":"2024-09-10T10:27:52.371348+0000","last_clean":"2024-09-10T10:27:52.371348+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:27:52.371348+0000","last_undegraded":"2024-09-10T10:27:52.371348+0000","last_fullsized":"2024-09-10T10:27:52.371348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370746+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:27:52.370746+0000","last_peered":"2024-09-10T10:27:52.370746+0000","last_clean":"2024-09-10T10:27:52.370746+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:27:52.370746+0000","last_undegraded":"2024-09-10T10:27:52.370746+0000","last_fullsized":"2024-09-10T10:27:52.370746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.047651+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:27:52.047651+0000","last_peered":"2024-09-10T10:27:52.047651+0000","last_clean":"2024-09-10T10:27:52.047651+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:27:52.047651+0000","last_undegraded":"2024-09-10T10:27:52.047651+0000","last_fullsized":"2024-09-10T10:27:52.047651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.044769+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:27:52.044769+0000","last_peered":"2024-09-10T10:27:52.044769+0000","last_clean":"2024-09-10T10:27:52.044769+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:27:52.044769+0000","last_undegraded":"2024-09-10T10:27:52.044769+0000","last_fullsized":"2024-09-10T10:27:52.044769+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370533+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:27:52.370533+0000","last_peered":"2024-09-10T10:27:52.370533+0000","last_clean":"2024-09-10T10:27:52.370533+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:27:52.370533+0000","last_undegraded":"2024-09-10T10:27:52.370533+0000","last_fullsized":"2024-09-10T10:27:52.370533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045833+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:27:52.045833+0000","last_peered":"2024-09-10T10:27:52.045833+0000","last_clean":"2024-09-10T10:27:52.045833+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:27:52.045833+0000","last_undegraded":"2024-09-10T10:27:52.045833+0000","last_fullsized":"2024-09-10T10:27:52.045833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.044515+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:27:52.044515+0000","last_peered":"2024-09-10T10:27:52.044515+0000","last_clean":"2024-09-10T10:27:52.044515+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:27:52.044515+0000","last_undegraded":"2024-09-10T10:27:52.044515+0000","last_fullsized":"2024-09-10T10:27:52.044515+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369895+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:27:52.369895+0000","last_peered":"2024-09-10T10:27:52.369895+0000","last_clean":"2024-09-10T10:27:52.369895+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:27:52.369895+0000","last_undegraded":"2024-09-10T10:27:52.369895+0000","last_fullsized":"2024-09-10T10:27:52.369895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"28'17","reported_seq":63,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.047072+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:27:52.047072+0000","last_peered":"2024-09-10T10:27:52.047072+0000","last_clean":"2024-09-10T10:27:52.047072+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:27:52.047072+0000","last_undegraded":"2024-09-10T10:27:52.047072+0000","last_fullsized":"2024-09-10T10:27:52.047072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":38,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370736+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:27:52.370736+0000","last_peered":"2024-09-10T10:27:52.370736+0000","last_clean":"2024-09-10T10:27:52.370736+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:27:52.370736+0000","last_undegraded":"2024-09-10T10:27:52.370736+0000","last_fullsized":"2024-09-10T10:27:52.370736+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369388+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:27:52.369388+0000","last_peered":"2024-09-10T10:27:52.369388+0000","last_clean":"2024-09-10T10:27:52.369388+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:27:52.369388+0000","last_undegraded":"2024-09-10T10:27:52.369388+0000","last_fullsized":"2024-09-10T10:27:52.369388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369462+0000","last_change":"2024-09-10T10:26:40.794739+0000","last_active":"2024-09-10T10:27:52.369462+0000","last_peered":"2024-09-10T10:27:52.369462+0000","last_clean":"2024-09-10T10:27:52.369462+0000","last_became_active":"2024-09-10T10:26:40.794430+0000","last_became_peered":"2024-09-10T10:26:40.794430+0000","last_unstale":"2024-09-10T10:27:52.369462+0000","last_undegraded":"2024-09-10T10:27:52.369462+0000","last_fullsized":"2024-09-10T10:27:52.369462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369893+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:27:52.369893+0000","last_peered":"2024-09-10T10:27:52.369893+0000","last_clean":"2024-09-10T10:27:52.369893+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:27:52.369893+0000","last_undegraded":"2024-09-10T10:27:52.369893+0000","last_fullsized":"2024-09-10T10:27:52.369893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370197+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:27:52.370197+0000","last_peered":"2024-09-10T10:27:52.370197+0000","last_clean":"2024-09-10T10:27:52.370197+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:27:52.370197+0000","last_undegraded":"2024-09-10T10:27:52.370197+0000","last_fullsized":"2024-09-10T10:27:52.370197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"28'5","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045413+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:27:52.045413+0000","last_peered":"2024-09-10T10:27:52.045413+0000","last_clean":"2024-09-10T10:27:52.045413+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:27:52.045413+0000","last_undegraded":"2024-09-10T10:27:52.045413+0000","last_fullsized":"2024-09-10T10:27:52.045413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369901+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:27:52.369901+0000","last_peered":"2024-09-10T10:27:52.369901+0000","last_clean":"2024-09-10T10:27:52.369901+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:27:52.369901+0000","last_undegraded":"2024-09-10T10:27:52.369901+0000","last_fullsized":"2024-09-10T10:27:52.369901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370065+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:27:52.370065+0000","last_peered":"2024-09-10T10:27:52.370065+0000","last_clean":"2024-09-10T10:27:52.370065+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:27:52.370065+0000","last_undegraded":"2024-09-10T10:27:52.370065+0000","last_fullsized":"2024-09-10T10:27:52.370065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":54,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.371560+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:27:52.371560+0000","last_peered":"2024-09-10T10:27:52.371560+0000","last_clean":"2024-09-10T10:27:52.371560+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:27:52.371560+0000","last_undegraded":"2024-09-10T10:27:52.371560+0000","last_fullsized":"2024-09-10T10:27:52.371560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045721+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:27:52.045721+0000","last_peered":"2024-09-10T10:27:52.045721+0000","last_clean":"2024-09-10T10:27:52.045721+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:27:52.045721+0000","last_undegraded":"2024-09-10T10:27:52.045721+0000","last_fullsized":"2024-09-10T10:27:52.045721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370636+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:27:52.370636+0000","last_peered":"2024-09-10T10:27:52.370636+0000","last_clean":"2024-09-10T10:27:52.370636+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:27:52.370636+0000","last_undegraded":"2024-09-10T10:27:52.370636+0000","last_fullsized":"2024-09-10T10:27:52.370636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"28'16","reported_seq":62,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370344+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:27:52.370344+0000","last_peered":"2024-09-10T10:27:52.370344+0000","last_clean":"2024-09-10T10:27:52.370344+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:27:52.370344+0000","last_undegraded":"2024-09-10T10:27:52.370344+0000","last_fullsized":"2024-09-10T10:27:52.370344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6177,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369863+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:27:52.369863+0000","last_peered":"2024-09-10T10:27:52.369863+0000","last_clean":"2024-09-10T10:27:52.369863+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:27:52.369863+0000","last_undegraded":"2024-09-10T10:27:52.369863+0000","last_fullsized":"2024-09-10T10:27:52.369863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045718+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:27:52.045718+0000","last_peered":"2024-09-10T10:27:52.045718+0000","last_clean":"2024-09-10T10:27:52.045718+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:27:52.045718+0000","last_undegraded":"2024-09-10T10:27:52.045718+0000","last_fullsized":"2024-09-10T10:27:52.045718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'5","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370565+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:27:52.370565+0000","last_peered":"2024-09-10T10:27:52.370565+0000","last_clean":"2024-09-10T10:27:52.370565+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:27:52.370565+0000","last_undegraded":"2024-09-10T10:27:52.370565+0000","last_fullsized":"2024-09-10T10:27:52.370565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045852+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:27:52.045852+0000","last_peered":"2024-09-10T10:27:52.045852+0000","last_clean":"2024-09-10T10:27:52.045852+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:27:52.045852+0000","last_undegraded":"2024-09-10T10:27:52.045852+0000","last_fullsized":"2024-09-10T10:27:52.045852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.371517+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:27:52.371517+0000","last_peered":"2024-09-10T10:27:52.371517+0000","last_clean":"2024-09-10T10:27:52.371517+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:27:52.371517+0000","last_undegraded":"2024-09-10T10:27:52.371517+0000","last_fullsized":"2024-09-10T10:27:52.371517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045657+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:27:52.045657+0000","last_peered":"2024-09-10T10:27:52.045657+0000","last_clean":"2024-09-10T10:27:52.045657+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:27:52.045657+0000","last_undegraded":"2024-09-10T10:27:52.045657+0000","last_fullsized":"2024-09-10T10:27:52.045657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":53,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369155+0000","last_change":"2024-09-10T10:27:44.443362+0000","last_active":"2024-09-10T10:27:52.369155+0000","last_peered":"2024-09-10T10:27:52.369155+0000","last_clean":"2024-09-10T10:27:52.369155+0000","last_became_active":"2024-09-10T10:27:44.442834+0000","last_became_peered":"2024-09-10T10:27:44.442834+0000","last_unstale":"2024-09-10T10:27:52.369155+0000","last_undegraded":"2024-09-10T10:27:52.369155+0000","last_fullsized":"2024-09-10T10:27:52.369155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:53.600117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370099+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:27:52.370099+0000","last_peered":"2024-09-10T10:27:52.370099+0000","last_clean":"2024-09-10T10:27:52.370099+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:27:52.370099+0000","last_undegraded":"2024-09-10T10:27:52.370099+0000","last_fullsized":"2024-09-10T10:27:52.370099+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"28'5","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370284+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:27:52.370284+0000","last_peered":"2024-09-10T10:27:52.370284+0000","last_clean":"2024-09-10T10:27:52.370284+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:27:52.370284+0000","last_undegraded":"2024-09-10T10:27:52.370284+0000","last_fullsized":"2024-09-10T10:27:52.370284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369435+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:27:52.369435+0000","last_peered":"2024-09-10T10:27:52.369435+0000","last_clean":"2024-09-10T10:27:52.369435+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:27:52.369435+0000","last_undegraded":"2024-09-10T10:27:52.369435+0000","last_fullsized":"2024-09-10T10:27:52.369435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":38,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370602+0000","last_change":"2024-09-10T10:26:46.193656+0000","last_active":"2024-09-10T10:27:52.370602+0000","last_peered":"2024-09-10T10:27:52.370602+0000","last_clean":"2024-09-10T10:27:52.370602+0000","last_became_active":"2024-09-10T10:26:46.193109+0000","last_became_peered":"2024-09-10T10:26:46.193109+0000","last_unstale":"2024-09-10T10:27:52.370602+0000","last_undegraded":"2024-09-10T10:27:52.370602+0000","last_fullsized":"2024-09-10T10:27:52.370602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369400+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:27:52.369400+0000","last_peered":"2024-09-10T10:27:52.369400+0000","last_clean":"2024-09-10T10:27:52.369400+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:27:52.369400+0000","last_undegraded":"2024-09-10T10:27:52.369400+0000","last_fullsized":"2024-09-10T10:27:52.369400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045179+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:27:52.045179+0000","last_peered":"2024-09-10T10:27:52.045179+0000","last_clean":"2024-09-10T10:27:52.045179+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:27:52.045179+0000","last_undegraded":"2024-09-10T10:27:52.045179+0000","last_fullsized":"2024-09-10T10:27:52.045179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370186+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:27:52.370186+0000","last_peered":"2024-09-10T10:27:52.370186+0000","last_clean":"2024-09-10T10:27:52.370186+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:27:52.370186+0000","last_undegraded":"2024-09-10T10:27:52.370186+0000","last_fullsized":"2024-09-10T10:27:52.370186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369250+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:27:52.369250+0000","last_peered":"2024-09-10T10:27:52.369250+0000","last_clean":"2024-09-10T10:27:52.369250+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:27:52.369250+0000","last_undegraded":"2024-09-10T10:27:52.369250+0000","last_fullsized":"2024-09-10T10:27:52.369250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369671+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:27:52.369671+0000","last_peered":"2024-09-10T10:27:52.369671+0000","last_clean":"2024-09-10T10:27:52.369671+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:27:52.369671+0000","last_undegraded":"2024-09-10T10:27:52.369671+0000","last_fullsized":"2024-09-10T10:27:52.369671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":53,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369675+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:27:52.369675+0000","last_peered":"2024-09-10T10:27:52.369675+0000","last_clean":"2024-09-10T10:27:52.369675+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:27:52.369675+0000","last_undegraded":"2024-09-10T10:27:52.369675+0000","last_fullsized":"2024-09-10T10:27:52.369675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.371470+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:27:52.371470+0000","last_peered":"2024-09-10T10:27:52.371470+0000","last_clean":"2024-09-10T10:27:52.371470+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:27:52.371470+0000","last_undegraded":"2024-09-10T10:27:52.371470+0000","last_fullsized":"2024-09-10T10:27:52.371470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.047262+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:27:52.047262+0000","last_peered":"2024-09-10T10:27:52.047262+0000","last_clean":"2024-09-10T10:27:52.047262+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:27:52.047262+0000","last_undegraded":"2024-09-10T10:27:52.047262+0000","last_fullsized":"2024-09-10T10:27:52.047262+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'6","reported_seq":52,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369789+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:27:52.369789+0000","last_peered":"2024-09-10T10:27:52.369789+0000","last_clean":"2024-09-10T10:27:52.369789+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:27:52.369789+0000","last_undegraded":"2024-09-10T10:27:52.369789+0000","last_fullsized":"2024-09-10T10:27:52.369789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045001+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:27:52.045001+0000","last_peered":"2024-09-10T10:27:52.045001+0000","last_clean":"2024-09-10T10:27:52.045001+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:27:52.045001+0000","last_undegraded":"2024-09-10T10:27:52.045001+0000","last_fullsized":"2024-09-10T10:27:52.045001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370016+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:27:52.370016+0000","last_peered":"2024-09-10T10:27:52.370016+0000","last_clean":"2024-09-10T10:27:52.370016+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:27:52.370016+0000","last_undegraded":"2024-09-10T10:27:52.370016+0000","last_fullsized":"2024-09-10T10:27:52.370016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369304+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:27:52.369304+0000","last_peered":"2024-09-10T10:27:52.369304+0000","last_clean":"2024-09-10T10:27:52.369304+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:27:52.369304+0000","last_undegraded":"2024-09-10T10:27:52.369304+0000","last_fullsized":"2024-09-10T10:27:52.369304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.047116+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:27:52.047116+0000","last_peered":"2024-09-10T10:27:52.047116+0000","last_clean":"2024-09-10T10:27:52.047116+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:27:52.047116+0000","last_undegraded":"2024-09-10T10:27:52.047116+0000","last_fullsized":"2024-09-10T10:27:52.047116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045318+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:27:52.045318+0000","last_peered":"2024-09-10T10:27:52.045318+0000","last_clean":"2024-09-10T10:27:52.045318+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:27:52.045318+0000","last_undegraded":"2024-09-10T10:27:52.045318+0000","last_fullsized":"2024-09-10T10:27:52.045318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.371341+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:27:52.371341+0000","last_peered":"2024-09-10T10:27:52.371341+0000","last_clean":"2024-09-10T10:27:52.371341+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:27:52.371341+0000","last_undegraded":"2024-09-10T10:27:52.371341+0000","last_fullsized":"2024-09-10T10:27:52.371341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370134+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:27:52.370134+0000","last_peered":"2024-09-10T10:27:52.370134+0000","last_clean":"2024-09-10T10:27:52.370134+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:27:52.370134+0000","last_undegraded":"2024-09-10T10:27:52.370134+0000","last_fullsized":"2024-09-10T10:27:52.370134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370431+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:27:52.370431+0000","last_peered":"2024-09-10T10:27:52.370431+0000","last_clean":"2024-09-10T10:27:52.370431+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:27:52.370431+0000","last_undegraded":"2024-09-10T10:27:52.370431+0000","last_fullsized":"2024-09-10T10:27:52.370431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370085+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:27:52.370085+0000","last_peered":"2024-09-10T10:27:52.370085+0000","last_clean":"2024-09-10T10:27:52.370085+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:27:52.370085+0000","last_undegraded":"2024-09-10T10:27:52.370085+0000","last_fullsized":"2024-09-10T10:27:52.370085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369986+0000","last_change":"2024-09-10T10:26:41.802903+0000","last_active":"2024-09-10T10:27:52.369986+0000","last_peered":"2024-09-10T10:27:52.369986+0000","last_clean":"2024-09-10T10:27:52.369986+0000","last_became_active":"2024-09-10T10:26:41.802586+0000","last_became_peered":"2024-09-10T10:26:41.802586+0000","last_unstale":"2024-09-10T10:27:52.369986+0000","last_undegraded":"2024-09-10T10:27:52.369986+0000","last_fullsized":"2024-09-10T10:27:52.369986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045065+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:27:52.045065+0000","last_peered":"2024-09-10T10:27:52.045065+0000","last_clean":"2024-09-10T10:27:52.045065+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:27:52.045065+0000","last_undegraded":"2024-09-10T10:27:52.045065+0000","last_fullsized":"2024-09-10T10:27:52.045065+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369635+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:27:52.369635+0000","last_peered":"2024-09-10T10:27:52.369635+0000","last_clean":"2024-09-10T10:27:52.369635+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:27:52.369635+0000","last_undegraded":"2024-09-10T10:27:52.369635+0000","last_fullsized":"2024-09-10T10:27:52.369635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045213+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:27:52.045213+0000","last_peered":"2024-09-10T10:27:52.045213+0000","last_clean":"2024-09-10T10:27:52.045213+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:27:52.045213+0000","last_undegraded":"2024-09-10T10:27:52.045213+0000","last_fullsized":"2024-09-10T10:27:52.045213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.371155+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:27:52.371155+0000","last_peered":"2024-09-10T10:27:52.371155+0000","last_clean":"2024-09-10T10:27:52.371155+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:27:52.371155+0000","last_undegraded":"2024-09-10T10:27:52.371155+0000","last_fullsized":"2024-09-10T10:27:52.371155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369806+0000","last_change":"2024-09-10T10:26:41.787435+0000","last_active":"2024-09-10T10:27:52.369806+0000","last_peered":"2024-09-10T10:27:52.369806+0000","last_clean":"2024-09-10T10:27:52.369806+0000","last_became_active":"2024-09-10T10:26:41.787083+0000","last_became_peered":"2024-09-10T10:26:41.787083+0000","last_unstale":"2024-09-10T10:27:52.369806+0000","last_undegraded":"2024-09-10T10:27:52.369806+0000","last_fullsized":"2024-09-10T10:27:52.369806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370523+0000","last_change":"2024-09-10T10:26:41.792587+0000","last_active":"2024-09-10T10:27:52.370523+0000","last_peered":"2024-09-10T10:27:52.370523+0000","last_clean":"2024-09-10T10:27:52.370523+0000","last_became_active":"2024-09-10T10:26:41.792320+0000","last_became_peered":"2024-09-10T10:26:41.792320+0000","last_unstale":"2024-09-10T10:27:52.370523+0000","last_undegraded":"2024-09-10T10:27:52.370523+0000","last_fullsized":"2024-09-10T10:27:52.370523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.047573+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:27:52.047573+0000","last_peered":"2024-09-10T10:27:52.047573+0000","last_clean":"2024-09-10T10:27:52.047573+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:27:52.047573+0000","last_undegraded":"2024-09-10T10:27:52.047573+0000","last_fullsized":"2024-09-10T10:27:52.047573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045976+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:27:52.045976+0000","last_peered":"2024-09-10T10:27:52.045976+0000","last_clean":"2024-09-10T10:27:52.045976+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:27:52.045976+0000","last_undegraded":"2024-09-10T10:27:52.045976+0000","last_fullsized":"2024-09-10T10:27:52.045976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.044967+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:27:52.044967+0000","last_peered":"2024-09-10T10:27:52.044967+0000","last_clean":"2024-09-10T10:27:52.044967+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:27:52.044967+0000","last_undegraded":"2024-09-10T10:27:52.044967+0000","last_fullsized":"2024-09-10T10:27:52.044967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369493+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:27:52.369493+0000","last_peered":"2024-09-10T10:27:52.369493+0000","last_clean":"2024-09-10T10:27:52.369493+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:27:52.369493+0000","last_undegraded":"2024-09-10T10:27:52.369493+0000","last_fullsized":"2024-09-10T10:27:52.369493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369951+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:27:52.369951+0000","last_peered":"2024-09-10T10:27:52.369951+0000","last_clean":"2024-09-10T10:27:52.369951+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:27:52.369951+0000","last_undegraded":"2024-09-10T10:27:52.369951+0000","last_fullsized":"2024-09-10T10:27:52.369951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369554+0000","last_change":"2024-09-10T10:26:41.802117+0000","last_active":"2024-09-10T10:27:52.369554+0000","last_peered":"2024-09-10T10:27:52.369554+0000","last_clean":"2024-09-10T10:27:52.369554+0000","last_became_active":"2024-09-10T10:26:41.801798+0000","last_became_peered":"2024-09-10T10:26:41.801798+0000","last_unstale":"2024-09-10T10:27:52.369554+0000","last_undegraded":"2024-09-10T10:27:52.369554+0000","last_fullsized":"2024-09-10T10:27:52.369554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045615+0000","last_change":"2024-09-10T10:26:41.791156+0000","last_active":"2024-09-10T10:27:52.045615+0000","last_peered":"2024-09-10T10:27:52.045615+0000","last_clean":"2024-09-10T10:27:52.045615+0000","last_became_active":"2024-09-10T10:26:41.790893+0000","last_became_peered":"2024-09-10T10:26:41.790893+0000","last_unstale":"2024-09-10T10:27:52.045615+0000","last_undegraded":"2024-09-10T10:27:52.045615+0000","last_fullsized":"2024-09-10T10:27:52.045615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:46.923082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369944+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:27:52.369944+0000","last_peered":"2024-09-10T10:27:52.369944+0000","last_clean":"2024-09-10T10:27:52.369944+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:27:52.369944+0000","last_undegraded":"2024-09-10T10:27:52.369944+0000","last_fullsized":"2024-09-10T10:27:52.369944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370021+0000","last_change":"2024-09-10T10:27:23.958305+0000","last_active":"2024-09-10T10:27:52.370021+0000","last_peered":"2024-09-10T10:27:52.370021+0000","last_clean":"2024-09-10T10:27:52.370021+0000","last_became_active":"2024-09-10T10:27:23.957954+0000","last_became_peered":"2024-09-10T10:27:23.957954+0000","last_unstale":"2024-09-10T10:27:52.370021+0000","last_undegraded":"2024-09-10T10:27:52.370021+0000","last_fullsized":"2024-09-10T10:27:52.370021+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:32.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370659+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:27:52.370659+0000","last_peered":"2024-09-10T10:27:52.370659+0000","last_clean":"2024-09-10T10:27:52.370659+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:27:52.370659+0000","last_undegraded":"2024-09-10T10:27:52.370659+0000","last_fullsized":"2024-09-10T10:27:52.370659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369059+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:27:52.369059+0000","last_peered":"2024-09-10T10:27:52.369059+0000","last_clean":"2024-09-10T10:27:52.369059+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:27:52.369059+0000","last_undegraded":"2024-09-10T10:27:52.369059+0000","last_fullsized":"2024-09-10T10:27:52.369059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.369778+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:27:52.369778+0000","last_peered":"2024-09-10T10:27:52.369778+0000","last_clean":"2024-09-10T10:27:52.369778+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:27:52.369778+0000","last_undegraded":"2024-09-10T10:27:52.369778+0000","last_fullsized":"2024-09-10T10:27:52.369778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045291+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:27:52.045291+0000","last_peered":"2024-09-10T10:27:52.045291+0000","last_clean":"2024-09-10T10:27:52.045291+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:27:52.045291+0000","last_undegraded":"2024-09-10T10:27:52.045291+0000","last_fullsized":"2024-09-10T10:27:52.045291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370044+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:27:52.370044+0000","last_peered":"2024-09-10T10:27:52.370044+0000","last_clean":"2024-09-10T10:27:52.370044+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:27:52.370044+0000","last_undegraded":"2024-09-10T10:27:52.370044+0000","last_fullsized":"2024-09-10T10:27:52.370044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.045905+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:27:52.045905+0000","last_peered":"2024-09-10T10:27:52.045905+0000","last_clean":"2024-09-10T10:27:52.045905+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:27:52.045905+0000","last_undegraded":"2024-09-10T10:27:52.045905+0000","last_fullsized":"2024-09-10T10:27:52.045905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370459+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:27:52.370459+0000","last_peered":"2024-09-10T10:27:52.370459+0000","last_clean":"2024-09-10T10:27:52.370459+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:27:52.370459+0000","last_undegraded":"2024-09-10T10:27:52.370459+0000","last_fullsized":"2024-09-10T10:27:52.370459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370382+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:27:52.370382+0000","last_peered":"2024-09-10T10:27:52.370382+0000","last_clean":"2024-09-10T10:27:52.370382+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:27:52.370382+0000","last_undegraded":"2024-09-10T10:27:52.370382+0000","last_fullsized":"2024-09-10T10:27:52.370382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T10:27:52.370539+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:27:52.370539+0000","last_peered":"2024-09-10T10:27:52.370539+0000","last_clean":"2024-09-10T10:27:52.370539+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:27:52.370539+0000","last_undegraded":"2024-09-10T10:27:52.370539+0000","last_fullsized":"2024-09-10T10:27:52.370539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":12855,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":217,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":25710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":189,"ondisk_log_size":189,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":516096,"data_stored":781270,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":195513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":192478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":438272,"data_stored":189876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":450560,"data_stored":195803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":190008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":190078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":507904,"data_stored":779788,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":7257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:27:55.831 INFO:tasks.thrashosds.thrasher:Setting 3.34 to [3, 1] 2024-09-10T10:27:55.831 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.34', '3', '1'] 2024-09-10T10:27:55.831 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.34 3 1 2024-09-10T10:27:55.867 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:55.866+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:55.891 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:55.891 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:55.901 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:27:55.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.971+0000 7ff1b2fd8640 1 Processor -- start 2024-09-10T10:27:55.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.971+0000 7ff1b2fd8640 1 -- start start 2024-09-10T10:27:55.972 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:55.971+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1b2fd8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1ac158140 0x7ff1ac06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1b2fd8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff1ac15b440 0x7ff1ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1b2fd8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1ac06c5a0 con 0x7ff1ac0c6d00 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1b2fd8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff1ac0c8040 con 0x7ff1ac15b440 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1b2fd8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ac0c4b60 con 0x7ff1ac158140 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1abfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff1ac15b440 0x7ff1ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1b1575640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff1ac0c6d00 0x7ff1ac15df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57320/0 (socket says 172.21.15.120:57320) 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1b1575640 1 -- 172.21.15.120:0/2768572292 learned_addr learned my addr 172.21.15.120:0/2768572292 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1b0d74640 1 --2- 172.21.15.120:0/2768572292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1ac158140 0x7ff1ac06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1abfff640 1 -- 172.21.15.120:0/2768572292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1ac158140 msgr2=0x7ff1ac06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1abfff640 1 --2- 172.21.15.120:0/2768572292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1ac158140 0x7ff1ac06cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1abfff640 1 -- 172.21.15.120:0/2768572292 >> v1:172.21.15.120:6789/0 conn(0x7ff1ac0c6d00 legacy=0x7ff1ac15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.972+0000 7ff1abfff640 1 -- 172.21.15.120:0/2768572292 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac0c7a80 con 0x7ff1ac15b440 2024-09-10T10:27:55.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.973+0000 7ff1abfff640 1 --2- 172.21.15.120:0/2768572292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 0x7ff1ac06e770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff1980097a0 tx=0x7ff19802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a08fc061fb807a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.973+0000 7ff1ab7fe640 1 -- 172.21.15.120:0/2768572292 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff198037070 con 0x7ff1ac15b440 2024-09-10T10:27:55.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.973+0000 7ff1ab7fe640 1 -- 172.21.15.120:0/2768572292 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff198032440 con 0x7ff1ac15b440 2024-09-10T10:27:55.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.973+0000 7ff1ab7fe640 1 -- 172.21.15.120:0/2768572292 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff198032810 con 0x7ff1ac15b440 2024-09-10T10:27:55.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.974+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2768572292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 msgr2=0x7ff1ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.974+0000 7ff1b2fd8640 1 --2- 172.21.15.120:0/2768572292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 0x7ff1ac06e770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff1980097a0 tx=0x7ff19802a210 comp rx=0 tx=0).stop 2024-09-10T10:27:55.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.974+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2768572292 shutdown_connections 2024-09-10T10:27:55.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.974+0000 7ff1b2fd8640 1 --2- 172.21.15.120:0/2768572292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 0x7ff1ac06e770 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.974+0000 7ff1b2fd8640 1 --2- 172.21.15.120:0/2768572292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1ac158140 0x7ff1ac06cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.974+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2768572292 >> 172.21.15.120:0/2768572292 conn(0x7ff1ac0728b0 msgr2=0x7ff1ac072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:55.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.974+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2768572292 shutdown_connections 2024-09-10T10:27:55.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.974+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2768572292 wait complete. 2024-09-10T10:27:55.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.975+0000 7ff1b2fd8640 1 Processor -- start 2024-09-10T10:27:55.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.975+0000 7ff1b2fd8640 1 -- start start 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1b2fd8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1ac0c6d00 0x7ff1ac1f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1b2fd8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ac158140 0x7ff1ac0c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1b2fd8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 0x7ff1ac1f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1b2fd8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac0735e0 con 0x7ff1ac15b440 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1b2fd8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac06c840 con 0x7ff1ac158140 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1b2fd8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ac06d2a0 con 0x7ff1ac0c6d00 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1b1575640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 0x7ff1ac1f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1b1575640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 0x7ff1ac1f0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42414/0 (socket says 172.21.15.120:42414) 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1b1575640 1 -- 172.21.15.120:0/2969587170 learned_addr learned my addr 172.21.15.120:0/2969587170 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1b0d74640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1ac0c6d00 0x7ff1ac1f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.976+0000 7ff1abfff640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ac158140 0x7ff1ac0c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1b1575640 1 -- 172.21.15.120:0/2969587170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1ac0c6d00 msgr2=0x7ff1ac1f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1b1575640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1ac0c6d00 0x7ff1ac1f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1b1575640 1 -- 172.21.15.120:0/2969587170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ac158140 msgr2=0x7ff1ac0c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1b1575640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ac158140 0x7ff1ac0c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1b1575640 1 -- 172.21.15.120:0/2969587170 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac0c38e0 con 0x7ff1ac15b440 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1b1575640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 0x7ff1ac1f0ff0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400caa0 tx=0x7ff1a400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1a97fa640 1 -- 172.21.15.120:0/2969587170 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4017070 con 0x7ff1ac15b440 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1a97fa640 1 -- 172.21.15.120:0/2969587170 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a400ac00 con 0x7ff1ac15b440 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2969587170 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac1e6700 con 0x7ff1ac15b440 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1a97fa640 1 -- 172.21.15.120:0/2969587170 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4012400 con 0x7ff1ac15b440 2024-09-10T10:27:55.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.977+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2969587170 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac0c3e30 con 0x7ff1ac15b440 2024-09-10T10:27:55.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.978+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2969587170 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ac1f08c0 con 0x7ff1ac15b440 2024-09-10T10:27:55.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.979+0000 7ff1a97fa640 1 -- 172.21.15.120:0/2969587170 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1a401a070 con 0x7ff1ac15b440 2024-09-10T10:27:55.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.980+0000 7ff1a97fa640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff19006be30 0x7ff19006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:55.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.980+0000 7ff1a97fa640 1 -- 172.21.15.120:0/2969587170 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6735+0+0 (secure 0 0 0) 0x7ff1a408ad50 con 0x7ff1ac15b440 2024-09-10T10:27:55.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.980+0000 7ff1b0d74640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff19006be30 0x7ff19006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:55.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.980+0000 7ff1b0d74640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff19006be30 0x7ff19006e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0006fd0 tx=0x7ff1a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:55.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:55.984+0000 7ff1a97fa640 1 -- 172.21.15.120:0/2969587170 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff1a408e030 con 0x7ff1ac15b440 2024-09-10T10:27:56.053 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:56.053 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:56.064 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:27:56.105 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:56.104+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:56.166 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:56.166+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:56.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:56.176+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2969587170 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pg-upmap", "pgid": "3.34", "id": [3, 1]} v 0) -- 0x7ff1ac0c83f0 con 0x7ff1ac15b440 2024-09-10T10:27:56.211 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:56.211 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:56.220 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:27:56.268 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:56.266+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:56.366 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:56.366 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:56.368 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:56.367+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:56.375 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:27:56.469 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:56.468+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:56.480 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:56.480 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:56.488 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:27:56.570 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:56.569+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:56.593 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:56.593 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:56.602 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:27:56.671 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:56.669+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:56.718 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:56.718 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:56.727 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:27:56.772 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:56.771+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:56.871 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:56.871 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:56.872 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:56.870+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:56.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:27:56.973 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:56.971+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:57.023 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:57.024 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:57.032 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:27:57.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.061+0000 7ff1a97fa640 1 -- 172.21.15.120:0/2969587170 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap", "pgid": "3.34", "id": [3, 1]}]=0 set 3.34 pg_upmap mapping to [3,1] v39) ==== 124+0+0 (secure 0 0 0) 0x7ff1a4058d40 con 0x7ff1ac15b440 2024-09-10T10:27:57.062 INFO:teuthology.orchestra.run.smithi120.stderr:set 3.34 pg_upmap mapping to [3,1] 2024-09-10T10:27:57.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2969587170 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff19006be30 msgr2=0x7ff19006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff19006be30 0x7ff19006e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0006fd0 tx=0x7ff1a0008040 comp rx=0 tx=0).stop 2024-09-10T10:27:57.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2969587170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 msgr2=0x7ff1ac1f0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 0x7ff1ac1f0ff0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400caa0 tx=0x7ff1a400cf60 comp rx=0 tx=0).stop 2024-09-10T10:27:57.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2969587170 shutdown_connections 2024-09-10T10:27:57.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff19006be30 0x7ff19006e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1ac15b440 0x7ff1ac1f0ff0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ac158140 0x7ff1ac0c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 --2- 172.21.15.120:0/2969587170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1ac0c6d00 0x7ff1ac1f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2969587170 >> 172.21.15.120:0/2969587170 conn(0x7ff1ac0728b0 msgr2=0x7ff1ac070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:57.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2969587170 shutdown_connections 2024-09-10T10:27:57.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.063+0000 7ff1b2fd8640 1 -- 172.21.15.120:0/2969587170 wait complete. 2024-09-10T10:27:57.073 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-10T10:27:57.074 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:57.072+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.173+0000 7f51f3d42640 1 Processor -- start 2024-09-10T10:27:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.173+0000 7f51f3d42640 1 -- start start 2024-09-10T10:27:57.174 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:27:57.173+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:27:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f3d42640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f51ec15aab0 0x7f51ec0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f3d42640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51ec15b1d0 con 0x7f51ec0cc0c0 2024-09-10T10:27:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f3d42640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51ec15b4b0 con 0x7f51ec0cb2f0 2024-09-10T10:27:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f3d42640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f51ec0cbd00 con 0x7f51ec15aab0 2024-09-10T10:27:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f22df640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f51ec15aab0 0x7f51ec0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f1ade640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f51ec0cc0c0 0x7f51ec15e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:57322/0 (socket says 172.21.15.120:57322) 2024-09-10T10:27:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f22df640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f51ec15aab0 0x7f51ec0cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42422/0 (socket says 172.21.15.120:42422) 2024-09-10T10:27:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f1ade640 1 -- 172.21.15.120:0/408573475 learned_addr learned my addr 172.21.15.120:0/408573475 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f22df640 1 -- 172.21.15.120:0/408573475 >> v1:172.21.15.134:6789/0 conn(0x7f51ec0cb2f0 legacy=0x7f51ec163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f22df640 1 -- 172.21.15.120:0/408573475 >> v1:172.21.15.120:6789/0 conn(0x7f51ec0cc0c0 legacy=0x7f51ec15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.174+0000 7f51f22df640 1 -- 172.21.15.120:0/408573475 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec15b790 con 0x7f51ec15aab0 2024-09-10T10:27:57.177 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:57.177 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.175+0000 7f51f22df640 1 --2- 172.21.15.120:0/408573475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 0x7f51ec0cc4a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009d90 tx=0x7f51e402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f364ef6b01d5532 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.175+0000 7f51f0adc640 1 -- 172.21.15.120:0/408573475 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4007740 con 0x7f51ec15aab0 2024-09-10T10:27:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.175+0000 7f51f0adc640 1 -- 172.21.15.120:0/408573475 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e40078e0 con 0x7f51ec15aab0 2024-09-10T10:27:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.175+0000 7f51f0adc640 1 -- 172.21.15.120:0/408573475 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4007cb0 con 0x7f51ec15aab0 2024-09-10T10:27:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.176+0000 7f51f3d42640 1 -- 172.21.15.120:0/408573475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 msgr2=0x7f51ec0cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.176+0000 7f51f3d42640 1 --2- 172.21.15.120:0/408573475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 0x7f51ec0cc4a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009d90 tx=0x7f51e402a6f0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.176+0000 7f51f3d42640 1 -- 172.21.15.120:0/408573475 shutdown_connections 2024-09-10T10:27:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.176+0000 7f51f3d42640 1 --2- 172.21.15.120:0/408573475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 0x7f51ec0cc4a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.176+0000 7f51f3d42640 1 -- 172.21.15.120:0/408573475 >> 172.21.15.120:0/408573475 conn(0x7f51ec06d560 msgr2=0x7f51ec06d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.176+0000 7f51f3d42640 1 -- 172.21.15.120:0/408573475 shutdown_connections 2024-09-10T10:27:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.176+0000 7f51f3d42640 1 -- 172.21.15.120:0/408573475 wait complete. 2024-09-10T10:27:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.177+0000 7f51f3d42640 1 Processor -- start 2024-09-10T10:27:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.177+0000 7f51f3d42640 1 -- start start 2024-09-10T10:27:57.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f3d42640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51ec0cb2f0 0x7f51ec14ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f3d42640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51ec0cc0c0 0x7f51ec153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f3d42640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 0x7f51ec0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f3d42640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec15e540 con 0x7f51ec15aab0 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f3d42640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec15ba70 con 0x7f51ec0cc0c0 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f3d42640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51ec15aec0 con 0x7f51ec0cb2f0 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f22df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 0x7f51ec0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f22df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 0x7f51ec0c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42430/0 (socket says 172.21.15.120:42430) 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f22df640 1 -- 172.21.15.120:0/4045085083 learned_addr learned my addr 172.21.15.120:0/4045085083 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f12dd640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51ec0cc0c0 0x7f51ec153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f1ade640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51ec0cb2f0 0x7f51ec14ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f22df640 1 -- 172.21.15.120:0/4045085083 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51ec0cb2f0 msgr2=0x7f51ec14ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f22df640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51ec0cb2f0 0x7f51ec14ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f22df640 1 -- 172.21.15.120:0/4045085083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51ec0cc0c0 msgr2=0x7f51ec153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f22df640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51ec0cc0c0 0x7f51ec153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f22df640 1 -- 172.21.15.120:0/4045085083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec0c8020 con 0x7f51ec15aab0 2024-09-10T10:27:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51f22df640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 0x7f51ec0c9e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f51e402ace0 tx=0x7f51e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.178+0000 7f51e27fc640 1 -- 172.21.15.120:0/4045085083 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4040070 con 0x7f51ec15aab0 2024-09-10T10:27:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.179+0000 7f51e27fc640 1 -- 172.21.15.120:0/4045085083 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e4031c80 con 0x7f51ec15aab0 2024-09-10T10:27:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.179+0000 7f51f3d42640 1 -- 172.21.15.120:0/4045085083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec151470 con 0x7f51ec15aab0 2024-09-10T10:27:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.179+0000 7f51e27fc640 1 -- 172.21.15.120:0/4045085083 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4045680 con 0x7f51ec15aab0 2024-09-10T10:27:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.179+0000 7f51f3d42640 1 -- 172.21.15.120:0/4045085083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec0c7d40 con 0x7f51ec15aab0 2024-09-10T10:27:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.179+0000 7f51f3d42640 1 -- 172.21.15.120:0/4045085083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec0c5560 con 0x7f51ec15aab0 2024-09-10T10:27:57.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.180+0000 7f51e27fc640 1 -- 172.21.15.120:0/4045085083 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51e4039040 con 0x7f51ec15aab0 2024-09-10T10:27:57.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.181+0000 7f51e27fc640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f51d006be60 0x7f51d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.181+0000 7f51f1ade640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f51d006be60 0x7f51d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.181+0000 7f51e27fc640 1 -- 172.21.15.120:0/4045085083 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6764+0+0 (secure 0 0 0) 0x7f51e40ac9d0 con 0x7f51ec15aab0 2024-09-10T10:27:57.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.181+0000 7f51f1ade640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f51d006be60 0x7f51d006e320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005880 tx=0x7f51e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.185 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:27:57.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.185+0000 7f51e27fc640 1 -- 172.21.15.120:0/4045085083 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f51e407aac0 con 0x7f51ec15aab0 2024-09-10T10:27:57.275 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:57.273+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:57.289 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:57.290 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:57.299 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:27:57.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.372+0000 7f51f3d42640 1 -- 172.21.15.120:0/4045085083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f51ec0cbea0 con 0x7f51ec15aab0 2024-09-10T10:27:57.383 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:57.381+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:57.442 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:57.442 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:57.452 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:27:57.470 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:27:57.493 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:57.492+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:57.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.566+0000 7fb42e34b640 1 Processor -- start 2024-09-10T10:27:57.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.566+0000 7fb42e34b640 1 -- start start 2024-09-10T10:27:57.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb42e34b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb4280cca70 0x7fb42815b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb42e34b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4280cafa0 0x7fb4280cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb42e34b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb42815e0b0 con 0x7fb428154c10 2024-09-10T10:27:57.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb42e34b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb42815de60 con 0x7fb4280cca70 2024-09-10T10:27:57.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb42e34b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb42815e390 con 0x7fb4280cafa0 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb427fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fb428154c10 0x7fb428073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:53062/0 (socket says 172.21.15.120:53062) 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb427fff640 1 -- 172.21.15.120:0/1309390776 learned_addr learned my addr 172.21.15.120:0/1309390776 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb4277fe640 1 --2- 172.21.15.120:0/1309390776 >> v2:172.21.15.134:3300/0 conn(0x7fb4280cca70 0x7fb42815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb42c8e8640 1 --2- 172.21.15.120:0/1309390776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4280cafa0 0x7fb4280cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb426ffd640 1 -- 172.21.15.120:0/1309390776 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4114386588 0 0) 0x7fb42815e0b0 con 0x7fb428154c10 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.567+0000 7fb426ffd640 1 -- 172.21.15.120:0/1309390776 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb410003520 con 0x7fb428154c10 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.568+0000 7fb4277fe640 1 -- 172.21.15.120:0/1309390776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4280cafa0 msgr2=0x7fb4280cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.568+0000 7fb4277fe640 1 --2- 172.21.15.120:0/1309390776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4280cafa0 0x7fb4280cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.568+0000 7fb4277fe640 1 -- 172.21.15.120:0/1309390776 >> v1:172.21.15.120:6789/0 conn(0x7fb428154c10 legacy=0x7fb428073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.568+0000 7fb4277fe640 1 -- 172.21.15.120:0/1309390776 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42815e670 con 0x7fb4280cca70 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.568+0000 7fb42c8e8640 1 --2- 172.21.15.120:0/1309390776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4280cafa0 0x7fb4280cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:27:57.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.568+0000 7fb4277fe640 1 --2- 172.21.15.120:0/1309390776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4280cca70 0x7fb42815b370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009790 tx=0x7fb41c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c157e9c1600aab99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.568+0000 7fb426ffd640 1 -- 172.21.15.120:0/1309390776 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c037070 con 0x7fb4280cca70 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.569+0000 7fb426ffd640 1 -- 172.21.15.120:0/1309390776 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb41c032440 con 0x7fb4280cca70 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.569+0000 7fb426ffd640 1 -- 172.21.15.120:0/1309390776 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c032810 con 0x7fb4280cca70 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.569+0000 7fb42e34b640 1 -- 172.21.15.120:0/1309390776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4280cca70 msgr2=0x7fb42815b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.569+0000 7fb42e34b640 1 --2- 172.21.15.120:0/1309390776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4280cca70 0x7fb42815b370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009790 tx=0x7fb41c02a180 comp rx=0 tx=0).stop 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.569+0000 7fb42e34b640 1 -- 172.21.15.120:0/1309390776 shutdown_connections 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.569+0000 7fb42e34b640 1 --2- 172.21.15.120:0/1309390776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4280cafa0 0x7fb4280cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.569+0000 7fb42e34b640 1 --2- 172.21.15.120:0/1309390776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4280cca70 0x7fb42815b370 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.569+0000 7fb42e34b640 1 -- 172.21.15.120:0/1309390776 >> 172.21.15.120:0/1309390776 conn(0x7fb42806ce80 msgr2=0x7fb42806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.569+0000 7fb42e34b640 1 -- 172.21.15.120:0/1309390776 shutdown_connections 2024-09-10T10:27:57.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.569+0000 7fb42e34b640 1 -- 172.21.15.120:0/1309390776 wait complete. 2024-09-10T10:27:57.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.570+0000 7fb42e34b640 1 Processor -- start 2024-09-10T10:27:57.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb42e34b640 1 -- start start 2024-09-10T10:27:57.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb42e34b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4280cafa0 0x7fb4280c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb42e34b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4280cca70 0x7fb4281f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb42e34b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb428154c10 0x7fb4280c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb42e34b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb428154ff0 con 0x7fb4280cafa0 2024-09-10T10:27:57.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb42e34b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb42815ec30 con 0x7fb428154c10 2024-09-10T10:27:57.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb42e34b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb428073a50 con 0x7fb4280cca70 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb427fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4280cafa0 0x7fb4280c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb427fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4280cafa0 0x7fb4280c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34922/0 (socket says 172.21.15.120:34922) 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb427fff640 1 -- 172.21.15.120:0/4057557237 learned_addr learned my addr 172.21.15.120:0/4057557237 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb42c8e8640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb428154c10 0x7fb4280c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.571+0000 7fb4277fe640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4280cca70 0x7fb4281f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb427fff640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4280cca70 msgr2=0x7fb4281f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb427fff640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4280cca70 0x7fb4281f10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb427fff640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb428154c10 msgr2=0x7fb4280c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb427fff640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb428154c10 0x7fb4280c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb427fff640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb428158200 con 0x7fb4280cafa0 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb427fff640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4280cafa0 0x7fb4280c7c70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb41800c3c0 tx=0x7fb41800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418017070 con 0x7fb4280cafa0 2024-09-10T10:27:57.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb418005ce0 con 0x7fb4280cafa0 2024-09-10T10:27:57.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4280ca970 con 0x7fb4280cafa0 2024-09-10T10:27:57.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4180072d0 con 0x7fb4280cafa0 2024-09-10T10:27:57.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.572+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb428157f20 con 0x7fb4280cafa0 2024-09-10T10:27:57.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.573+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb428153de0 con 0x7fb4280cafa0 2024-09-10T10:27:57.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.574+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb41801a070 con 0x7fb4280cafa0 2024-09-10T10:27:57.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.574+0000 7fb424ff9640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb41406be60 0x7fb41406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.574+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6764+0+0 (secure 0 0 0) 0x7fb41808a550 con 0x7fb4280cafa0 2024-09-10T10:27:57.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.574+0000 7fb4277fe640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb41406be60 0x7fb41406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.574+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v39) ==== 54+0+16 (secure 0 0 0) 0x7fb428153de0 con 0x7fb4280cafa0 2024-09-10T10:27:57.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.575+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fb4281f07b0 0x7fb4280c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.575+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb428157cd0 con 0x7fb4281f07b0 2024-09-10T10:27:57.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.575+0000 7fb4277fe640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb41406be60 0x7fb41406e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb41c034000 tx=0x7fb41c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.575+0000 7fb42c8e8640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fb4281f07b0 0x7fb4280c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.575+0000 7fb42c8e8640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fb4281f07b0 0x7fb4280c55c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.576+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb428157cd0 con 0x7fb4281f07b0 2024-09-10T10:27:57.579 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:57.577+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:57.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.591+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb4280c6990 con 0x7fb4281f07b0 2024-09-10T10:27:57.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.592+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb4280c6990 con 0x7fb4281f07b0 2024-09-10T10:27:57.593 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:57.594 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:57.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.593+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fb4281f4230 0x7fb4281f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.593+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb4281521d0 con 0x7fb4281f4230 2024-09-10T10:27:57.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.593+0000 7fb427fff640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fb4281f4230 0x7fb4281f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.593+0000 7fb427fff640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fb4281f4230 0x7fb4281f4670 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.594+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4281521d0 con 0x7fb4281f4230 2024-09-10T10:27:57.596 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:57.596 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:57.604 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:27:57.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.608+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb42815a080 con 0x7fb4281f4230 2024-09-10T10:27:57.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.610+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb42815a080 con 0x7fb4281f4230 2024-09-10T10:27:57.611 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:57.612 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:57.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.610+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fb42814e160 0x7fb42814e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.610+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb42814eaa0 con 0x7fb42814e160 2024-09-10T10:27:57.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.610+0000 7fb4277fe640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fb42814e160 0x7fb42814e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.611+0000 7fb4277fe640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fb42814e160 0x7fb42814e560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.611+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb41c0324c0 con 0x7fb42814e160 2024-09-10T10:27:57.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.627+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb4280c81b0 con 0x7fb42814e160 2024-09-10T10:27:57.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.629+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb4280c81b0 con 0x7fb42814e160 2024-09-10T10:27:57.630 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:57.631 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:57.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.629+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fb428062cb0 0x7fb4280630f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.629+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb428203310 con 0x7fb428062cb0 2024-09-10T10:27:57.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.629+0000 7fb42c8e8640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fb428062cb0 0x7fb4280630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.629+0000 7fb42c8e8640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fb428062cb0 0x7fb4280630f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.630+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb428203310 con 0x7fb428062cb0 2024-09-10T10:27:57.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.643+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb42815bca0 con 0x7fb428062cb0 2024-09-10T10:27:57.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.645+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb42815bca0 con 0x7fb428062cb0 2024-09-10T10:27:57.646 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:57.647 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:57.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.645+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fb42815c780 0x7fb42802c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.645+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb42815bca0 con 0x7fb42815c780 2024-09-10T10:27:57.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.645+0000 7fb427fff640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fb42815c780 0x7fb42802c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.646+0000 7fb427fff640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fb42815c780 0x7fb42802c640 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.646+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb42815bca0 con 0x7fb42815c780 2024-09-10T10:27:57.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.660+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb42815bca0 con 0x7fb42815c780 2024-09-10T10:27:57.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.662+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb42815bca0 con 0x7fb42815c780 2024-09-10T10:27:57.663 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:57.664 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:57.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.662+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fb42802d090 0x7fb42802d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.662+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb42802dab0 con 0x7fb42802d090 2024-09-10T10:27:57.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.663+0000 7fb4277fe640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fb42802d090 0x7fb42802d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.663+0000 7fb4277fe640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fb42802d090 0x7fb42802d4f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.664+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb42802dab0 con 0x7fb42802d090 2024-09-10T10:27:57.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.677+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb42815be80 con 0x7fb42802d090 2024-09-10T10:27:57.679 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:57.677+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:57.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.679+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb42815be80 con 0x7fb42802d090 2024-09-10T10:27:57.680 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:57.680 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:57.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.679+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fb4282079b0 0x7fb428209da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.679+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb42820cdc0 con 0x7fb4282079b0 2024-09-10T10:27:57.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.679+0000 7fb42c8e8640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fb4282079b0 0x7fb428209da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.679+0000 7fb42c8e8640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fb4282079b0 0x7fb428209da0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.680+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb42820cdc0 con 0x7fb4282079b0 2024-09-10T10:27:57.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.694+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb428075960 con 0x7fb4282079b0 2024-09-10T10:27:57.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.697+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb428075960 con 0x7fb4282079b0 2024-09-10T10:27:57.698 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:57.698 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:57.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.697+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb42820c0a0 0x7fb42820c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:57.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.697+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb428210190 con 0x7fb42820c0a0 2024-09-10T10:27:57.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.697+0000 7fb427fff640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb42820c0a0 0x7fb42820c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:57.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.697+0000 7fb427fff640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb42820c0a0 0x7fb42820c4e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:57.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.698+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb428210190 con 0x7fb42820c0a0 2024-09-10T10:27:57.709 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:57.709 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:57.709 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:57.709 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:57.709 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:57.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.711+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb42820f010 con 0x7fb42820c0a0 2024-09-10T10:27:57.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb424ff9640 1 -- 172.21.15.120:0/4057557237 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb42820f010 con 0x7fb42820c0a0 2024-09-10T10:27:57.714 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:57.716 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:57.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fb4281f07b0 msgr2=0x7fb4280c55c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fb4281f07b0 0x7fb4280c55c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fb4281f4230 msgr2=0x7fb4281f4670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fb4281f4230 0x7fb4281f4670 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fb42814e160 msgr2=0x7fb42814e560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fb42814e160 0x7fb42814e560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fb428062cb0 msgr2=0x7fb4280630f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fb428062cb0 0x7fb4280630f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fb42815c780 msgr2=0x7fb42802c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fb42815c780 0x7fb42802c640 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fb42802d090 msgr2=0x7fb42802d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fb42802d090 0x7fb42802d4f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fb4282079b0 msgr2=0x7fb428209da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fb4282079b0 0x7fb428209da0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb42820c0a0 msgr2=0x7fb42820c4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb42820c0a0 0x7fb42820c4e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42c8e8640 1 -- 172.21.15.120:0/4057557237 reap_dead start 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42c8e8640 1 -- 172.21.15.120:0/4057557237 reap_dead start 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42c8e8640 1 -- 172.21.15.120:0/4057557237 reap_dead start 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.713+0000 7fb42c8e8640 1 -- 172.21.15.120:0/4057557237 reap_dead start 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42c8e8640 1 -- 172.21.15.120:0/4057557237 reap_dead start 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb41406be60 msgr2=0x7fb41406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb41406be60 0x7fb41406e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb41c034000 tx=0x7fb41c02e000 comp rx=0 tx=0).stop 2024-09-10T10:27:57.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4280cafa0 msgr2=0x7fb4280c7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:57.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4280cafa0 0x7fb4280c7c70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb41800c3c0 tx=0x7fb41800c7c0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 shutdown_connections 2024-09-10T10:27:57.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb41406be60 0x7fb41406e320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42e34b640 1 --2- 172.21.15.120:0/4057557237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4280cafa0 0x7fb4280c7c70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:57.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 >> 172.21.15.120:0/4057557237 conn(0x7fb42806ce80 msgr2=0x7fb4280cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:57.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 shutdown_connections 2024-09-10T10:27:57.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:57.714+0000 7fb42e34b640 1 -- 172.21.15.120:0/4057557237 wait complete. 2024-09-10T10:27:57.719 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:27:57.803 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:57.801+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:57.866 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:57.866 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:57.866 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.084739651000007, 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:57.867 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.868 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.869 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:27:57.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.084409151000003, 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:27:57.872 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.875 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:27:57.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:27:57.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:27:57.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.084362608000006, 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:27:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:27:57.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.992920396000002, 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:27:57.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:27:57.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.888 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.992839052999997, 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.891 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:27:57.893 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.992812497999999, 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:27:57.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:27:57.897 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:27:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.992669849000002, 2024-09-10T10:27:57.899 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:27:57.901 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.902 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.903 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.904 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.990022916000001, 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:57.905 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.989172175999997, 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.913 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 43.917020493000003, 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:27:57.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:27:57.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.917 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.918 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 43.916971357000001, 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.919 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:27:57.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:27:57.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:57.922 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.923 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.924 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 43.916859879, 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.925 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.927 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.928 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 37.943459359999999, 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.929 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.932 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.933 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 37.943429189, 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:57.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.936 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:27:57.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 37.943372697000001, 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.939 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:27:57.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:27:57.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 37.943271529, 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:27:57.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:27:57.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 31.923004431999999, 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.950 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:27:57.951 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:27:57.955 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 31.918939213000002, 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.959 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 31.918921343000001, 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:57.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 31.918875105000001, 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:57.964 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:57.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.966 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:57.968 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:57.970 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:27:57.970 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:57.968+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:58.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.064+0000 7f51e27fc640 1 -- 172.21.15.120:0/4045085083 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v40) ==== 101+0+0 (secure 0 0 0) 0x7f51e407edc0 con 0x7f51ec15aab0 2024-09-10T10:27:58.065 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.067+0000 7f51f3d42640 1 -- 172.21.15.120:0/4045085083 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f51d006be60 msgr2=0x7f51d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.067+0000 7f51f3d42640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f51d006be60 0x7f51d006e320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005880 tx=0x7f51e8009040 comp rx=0 tx=0).stop 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.067+0000 7f51f3d42640 1 -- 172.21.15.120:0/4045085083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 msgr2=0x7f51ec0c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.067+0000 7f51f3d42640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 0x7f51ec0c9e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f51e402ace0 tx=0x7f51e4004a40 comp rx=0 tx=0).stop 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.067+0000 7f51f3d42640 1 -- 172.21.15.120:0/4045085083 shutdown_connections 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.067+0000 7f51f3d42640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f51d006be60 0x7f51d006e320 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.067+0000 7f51f3d42640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51ec15aab0 0x7f51ec0c9e40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.067+0000 7f51f3d42640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51ec0cc0c0 0x7f51ec153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.067+0000 7f51f3d42640 1 --2- 172.21.15.120:0/4045085083 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51ec0cb2f0 0x7f51ec14ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.067+0000 7f51f3d42640 1 -- 172.21.15.120:0/4045085083 >> 172.21.15.120:0/4045085083 conn(0x7f51ec06d560 msgr2=0x7f51ec0cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.068+0000 7f51f3d42640 1 -- 172.21.15.120:0/4045085083 shutdown_connections 2024-09-10T10:27:58.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:58.068+0000 7f51f3d42640 1 -- 172.21.15.120:0/4045085083 wait complete. 2024-09-10T10:27:58.070 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:58.068+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:58.117 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:58.117 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:58.117 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:27:58.118 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:58.126 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:27:58.210 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:58.208+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:58.270 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:58.270 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:27:58.270 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:27:58.270 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:27:58.270 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:58.272 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:58.270+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:58.279 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:27:58.373 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:27:58.372+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:27:58.427 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:27:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:27:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:27:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:27:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:27:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.328703757, 2024-09-10T10:27:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.17492211699999999, 2024-09-10T10:27:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 1 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096318+0000", 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.657e-06 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096326+0000", 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0560000000000005e-06 2024-09-10T10:27:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096339+0000", 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2965999999999999e-05 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096349+0000", 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0784e-05 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096432+0000", 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2657999999999994e-05 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op must wait for map", 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096441+0000", 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4590000000000004e-06 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096935+0000", 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049331899999999998 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for peered", 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096944+0000", 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9539999999999993e-06 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267505+0000", 2024-09-10T10:27:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.170561622 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267700+0000", 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019420400000000001 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267923+0000", 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000223451 2024-09-10T10:27:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.270575+0000", 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026522030000000001 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271131+0000", 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055599000000000002 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271191+0000", 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9947000000000001e-05 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271238+0000", 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6847000000000001e-05 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.939869513000005, 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078035889999999997, 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 115 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1300000000000001e-07 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485209+0000", 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8628999999999997e-05 2024-09-10T10:27:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485214+0000", 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0849999999999996e-06 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485218+0000", 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3840000000000001e-06 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490185+0000", 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0049675270000000002 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490231+0000", 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5488000000000002e-05 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490307+0000", 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6038000000000006e-05 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491263+0000", 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095625099999999998 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492908+0000", 2024-09-10T10:27:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001645355 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492934+0000", 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5431000000000001e-05 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492954+0000", 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0088000000000001e-05 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.939795024000006, 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077741099999999999, 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 116 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1100000000000002e-07 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485227+0000", 2024-09-10T10:27:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1890000000000001e-06 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485231+0000", 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4010000000000001e-06 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485233+0000", 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.728e-06 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490628+0000", 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0053952109999999996 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490674+0000", 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5946999999999999e-05 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490750+0000", 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5227000000000003e-05 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491443+0000", 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069346800000000004 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492959+0000", 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015154470000000001 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492983+0000", 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.406e-05 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492999+0000", 2024-09-10T10:27:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6121e-05 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.939782895999997, 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078059490000000004, 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 117 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.23e-07 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485240+0000", 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780000000000001e-06 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485243+0000", 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8430000000000003e-06 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485247+0000", 2024-09-10T10:27:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2279999999999998e-06 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490995+0000", 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057480730000000002 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491043+0000", 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7936000000000003e-05 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491113+0000", 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0872999999999997e-05 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491636+0000", 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00052242300000000005 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493002+0000", 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001365808 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493025+0000", 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2932999999999999e-05 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493043+0000", 2024-09-10T10:27:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8031000000000001e-05 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.607427754, 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095639920000000003, 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 131 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1199999999999998e-07 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817663+0000", 2024-09-10T10:27:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1099000000000005e-05 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817670+0000", 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.8539999999999999e-06 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817674+0000", 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.574e-06 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818465+0000", 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000791445 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818509+0000", 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3569000000000002e-05 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818580+0000", 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1503999999999998e-05 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821292+0000", 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027120529999999999 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827108+0000", 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0058160759999999999 2024-09-10T10:27:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827134+0000", 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5859000000000001e-05 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827156+0000", 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1546999999999999e-05 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.607338545999994, 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095198769999999995, 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 132 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2899999999999999e-07 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817683+0000", 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0379999999999998e-06 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817690+0000", 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3110000000000002e-06 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817699+0000", 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:27:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820828+0000", 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031296829999999999 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820873+0000", 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4869e-05 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820943+0000", 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9919999999999995e-05 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823683+0000", 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027395140000000002 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827161+0000", 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003477857 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827183+0000", 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2612999999999998e-05 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827201+0000", 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7842999999999998e-05 2024-09-10T10:27:58.471 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.607271238999999, 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0094944009999999995, 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.472 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 133 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817749+0000", 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2399999999999998e-07 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817752+0000", 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1970000000000001e-06 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817760+0000", 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8080000000000005e-06 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.475 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817764+0000", 2024-09-10T10:27:58.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9419999999999997e-06 2024-09-10T10:27:58.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.476 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821319+0000", 2024-09-10T10:27:58.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035547740000000001 2024-09-10T10:27:58.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821364+0000", 2024-09-10T10:27:58.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5398000000000001e-05 2024-09-10T10:27:58.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821434+0000", 2024-09-10T10:27:58.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0382999999999993e-05 2024-09-10T10:27:58.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823717+0000", 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022825089999999998 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827205+0000", 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034879009999999998 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827225+0000", 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0611999999999998e-05 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827243+0000", 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7353000000000001e-05 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.479 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.607218398000001, 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095863660000000007, 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.480 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 134 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817802+0000", 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8999999999999996e-07 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817871+0000", 2024-09-10T10:27:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9572000000000004e-05 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817878+0000", 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2079999999999997e-06 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817882+0000", 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5889999999999996e-06 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821625+0000", 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037431069999999999 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821669+0000", 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3455000000000003e-05 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821740+0000", 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1390999999999995e-05 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823826+0000", 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020863399999999999 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827343+0000", 2024-09-10T10:27:58.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003516267 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827369+0000", 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.6044999999999999e-05 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827388+0000", 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8802000000000001e-05 2024-09-10T10:27:58.486 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.487 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.487 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.607125702999994, 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095738179999999996, 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.488 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 135 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5999999999999999e-07 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817897+0000", 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0149999999999999e-06 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817904+0000", 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5089999999999999e-06 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817909+0000", 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3120000000000001e-06 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823862+0000", 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059524199999999999 2024-09-10T10:27:58.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823911+0000", 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9323000000000002e-05 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823983+0000", 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1736000000000006e-05 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.492 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827101+0000", 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031181009999999999 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827392+0000", 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00029090700000000003 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827426+0000", 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4127e-05 2024-09-10T10:27:58.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827468+0000", 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1907999999999999e-05 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:58.494 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.607098687000004, 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.009594488, 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 136 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3700000000000001e-07 2024-09-10T10:27:58.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817925+0000", 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4659999999999999e-06 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817931+0000", 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5710000000000003e-06 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817936+0000", 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7550000000000002e-06 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824178+0000", 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062415700000000001 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.498 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824225+0000", 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6966999999999999e-05 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824296+0000", 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1284000000000004e-05 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827290+0000", 2024-09-10T10:27:58.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002993697 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827473+0000", 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018376699999999999 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827497+0000", 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3445e-05 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827515+0000", 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8529000000000002e-05 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:58.501 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.607066997999993, 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096066220000000004, 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 137 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.502 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3000000000000001e-07 2024-09-10T10:27:58.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818004+0000", 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0902000000000002e-05 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818013+0000", 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5420000000000005e-06 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.504 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818019+0000", 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7939999999999997e-06 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824489+0000", 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0064692580000000003 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824532+0000", 2024-09-10T10:27:58.505 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3631000000000003e-05 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824601+0000", 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9028000000000004e-05 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827334+0000", 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027325409999999998 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827520+0000", 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018642499999999999 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827541+0000", 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1151000000000001e-05 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827559+0000", 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7819999999999999e-05 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.606987677000006, 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010100303, 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.508 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 138 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818033+0000", 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.510 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818037+0000", 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0419999999999996e-06 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818045+0000", 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9759999999999995e-06 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818052+0000", 2024-09-10T10:27:58.511 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5909999999999998e-06 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826594+0000", 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0085421660000000003 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826671+0000", 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7121999999999998e-05 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826795+0000", 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012315199999999999 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828081+0000", 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0012869139999999999 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828088+0000", 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9870000000000002e-06 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828112+0000", 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3405e-05 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828132+0000", 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0394000000000002e-05 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.514 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.606957305999998, 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01125819, 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.515 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 139 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818063+0000", 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7800000000000002e-07 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818067+0000", 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8500000000000004e-06 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818075+0000", 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818081+0000", 2024-09-10T10:27:58.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1829999999999999e-06 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827594+0000", 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095129340000000007 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827669+0000", 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5239000000000002e-05 2024-09-10T10:27:58.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827789+0000", 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000119757 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829195+0000", 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014064279999999999 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.520 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829232+0000", 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7058999999999999e-05 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829281+0000", 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8884000000000002e-05 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.521 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829321+0000", 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9527999999999998e-05 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.606923237000004, 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012263886, 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.522 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 140 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818097+0000", 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8999999999999997e-07 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818145+0000", 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8040000000000001e-05 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818160+0000", 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4671999999999999e-05 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.525 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818169+0000", 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1900000000000001e-06 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828158+0000", 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099891440000000001 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828228+0000", 2024-09-10T10:27:58.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0407000000000005e-05 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828343+0000", 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000114821 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829914+0000", 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001570866 2024-09-10T10:27:58.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830312+0000", 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00039770499999999999 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830339+0000", 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.69e-05 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.528 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830360+0000", 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1651e-05 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.606835996000001, 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012742035000000001, 2024-09-10T10:27:58.529 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 141 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7700000000000004e-07 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818188+0000", 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8659999999999999e-06 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818196+0000", 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9480000000000008e-06 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818205+0000", 2024-09-10T10:27:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8570000000000001e-06 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828883+0000", 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010678600999999999 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828927+0000", 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3887999999999997e-05 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829001+0000", 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3442000000000006e-05 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830548+0000", 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001547194 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830884+0000", 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00033638100000000002 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830907+0000", 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3051999999999999e-05 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830926+0000", 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8229000000000001e-05 2024-09-10T10:27:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.593688259000004, 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097270880000000001, 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 145 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831332+0000", 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8900000000000001e-07 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831379+0000", 2024-09-10T10:27:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7134e-05 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831387+0000", 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7409999999999992e-06 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831391+0000", 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6190000000000002e-06 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833155+0000", 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017641969999999999 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833237+0000", 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1957000000000002e-05 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833356+0000", 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011843100000000001 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.834068+0000", 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00071179000000000001 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841004+0000", 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069359749999999996 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841034+0000", 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.042e-05 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841058+0000", 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4535000000000001e-05 2024-09-10T10:27:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:20 2.9 e26/18 2:92e99a1d:::612.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 49.549491994999997, 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074964979999999999, 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875534+0000", 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9109999999999998e-06 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875543+0000", 2024-09-10T10:27:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8319999999999995e-06 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875549+0000", 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2380000000000002e-06 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875699+0000", 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014945900000000001 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875717+0000", 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8584e-05 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882982+0000", 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0072648039999999997 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883024+0000", 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2135000000000002e-05 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:27 2.9 e26/18 2:91bd965c:::501.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 49.549431361000003, 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074993129999999996, 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875589+0000", 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3999999999999997e-07 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875595+0000", 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9680000000000003e-06 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875602+0000", 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875608+0000", 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0430000000000004e-06 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875887+0000", 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000279578 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875904+0000", 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6439e-05 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883027+0000", 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0071234480000000001 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883088+0000", 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0424000000000003e-05 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 32.483352283999999, 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01193195, 2024-09-10T10:27:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:27:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2399999999999999e-07 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941671+0000", 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8329999999999998e-06 2024-09-10T10:27:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2919999999999998e-06 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941686+0000", 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944874+0000", 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031881639999999998 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945305+0000", 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00043084599999999997 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945438+0000", 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013344600000000001 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950653+0000", 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005214733 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953496+0000", 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028433989999999999 2024-09-10T10:27:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953554+0000", 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7933000000000003e-05 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953599+0000", 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5130999999999999e-05 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 32.483326337999998, 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0093592050000000006, 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:27:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941694+0000", 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7899999999999999e-07 2024-09-10T10:27:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941711+0000", 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7530999999999999e-05 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941720+0000", 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4279999999999996e-06 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941725+0000", 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2390000000000002e-06 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942690+0000", 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00096541400000000005 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942807+0000", 2024-09-10T10:27:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000116777 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942967+0000", 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015997599999999999 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944708+0000", 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001740492 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950956+0000", 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062481580000000002 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951016+0000", 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0299e-05 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:27:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:27:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951052+0000", 2024-09-10T10:27:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6412000000000003e-05 2024-09-10T10:27:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:27:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:27:58.566 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:27:58.568 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:27:58.568 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:58.566+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:58.669 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:58.667+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:58.673 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:58.673 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:58.673 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:58.673 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:58.682 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:27:58.770 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:58.769+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:58.790 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:58.790 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:58.790 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:58.790 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:58.790 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:58.800 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:27:58.871 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:58.870+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:58.907 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:58.907 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:1 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.637559585999995, 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028809700000000001, 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9999999999999999e-07 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268119+0000", 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1810000000000003e-06 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268129+0000", 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9790000000000005e-06 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268137+0000", 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4800000000000001e-06 2024-09-10T10:27:58.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268198+0000", 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0918000000000002e-05 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268226+0000", 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7267000000000002e-05 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270943+0000", 2024-09-10T10:27:58.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027177529999999998 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270996+0000", 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2191999999999998e-05 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:119 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'81, mlcod=16'78)", 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:27:58.912 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.411708633000003, 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029329880000000001, 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:27:58.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493966+0000", 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1699999999999998e-07 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494040+0000", 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3883999999999999e-05 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494048+0000", 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4880000000000007e-06 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494057+0000", 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1310000000000006e-06 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494095+0000", 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8470000000000003e-05 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494108+0000", 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2588999999999999e-05 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496869+0000", 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027610400000000002 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496898+0000", 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9869000000000002e-05 2024-09-10T10:27:58.916 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.411324996999994, 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005336365, 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494350+0000", 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494416+0000", 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6413000000000003e-05 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494424+0000", 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.576e-06 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494430+0000", 2024-09-10T10:27:58.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6579999999999996e-06 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494466+0000", 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5648999999999997e-05 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494478+0000", 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.234e-05 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499654+0000", 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051755209999999998 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499685+0000", 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.167e-05 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-10T10:27:58.921 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.410930187000005, 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049664970000000003, 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494819+0000", 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4690000000000005e-05 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494825+0000", 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7080000000000002e-06 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494828+0000", 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1109999999999999e-06 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494869+0000", 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1146000000000002e-05 2024-09-10T10:27:58.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494881+0000", 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2295e-05 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499687+0000", 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048061459999999999 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499710+0000", 2024-09-10T10:27:58.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-05 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.125573983999999, 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054008370000000003, 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780104+0000", 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3039999999999998e-06 2024-09-10T10:27:58.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0738e-05 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780141+0000", 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6317999999999999e-05 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2199999999999992e-06 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780405+0000", 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025557599999999998 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780498+0000", 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2860000000000002e-05 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780591+0000", 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3094e-05 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024814469999999999 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785441+0000", 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023678280000000002 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785480+0000", 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9511999999999997e-05 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785501+0000", 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0939999999999999e-05 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:58.931 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.125499149999996, 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054780480000000001, 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780176+0000", 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6700000000000003e-07 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:27:58.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1289999999999998e-06 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780188+0000", 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.96e-06 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780194+0000", 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9989999999999997e-06 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780882+0000", 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068832400000000003 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780968+0000", 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5959999999999997e-05 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781098+0000", 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012994100000000001 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783078+0000", 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019803540000000001 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785553+0000", 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002474515 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785608+0000", 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5319999999999999e-05 2024-09-10T10:27:58.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785653+0000", 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4879000000000002e-05 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.125467607000004, 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042025200000000004, 2024-09-10T10:27:58.937 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780217+0000", 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780224+0000", 2024-09-10T10:27:58.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2529999999999998e-06 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780230+0000", 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6110000000000001e-06 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780282+0000", 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2148000000000003e-05 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780394+0000", 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111864 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780553+0000", 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015913799999999999 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783054+0000", 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025008809999999999 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784349+0000", 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012947480000000001 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784381+0000", 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2268999999999999e-05 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784409+0000", 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8007000000000001e-05 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.942 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.125425254999996, 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054782659999999999, 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780253+0000", 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4199999999999999e-06 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780260+0000", 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5100000000000001e-06 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780267+0000", 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.443e-06 2024-09-10T10:27:58.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781353+0000", 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010859769999999999 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781457+0000", 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104545 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781561+0000", 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010409200000000001 2024-09-10T10:27:58.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783532+0000", 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019702280000000001 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785685+0000", 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002153313 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785710+0000", 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5024999999999998e-05 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785727+0000", 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7175000000000001e-05 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:58.948 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.125399164000001, 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005104407, 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:58.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780278+0000", 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.959e-06 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780285+0000", 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8040000000000004e-06 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780292+0000", 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4099999999999996e-06 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781345+0000", 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010533579999999999 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781456+0000", 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110624 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781570+0000", 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011417999999999999 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783625+0000", 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020553780000000001 2024-09-10T10:27:58.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785303+0000", 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001677827 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785345+0000", 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2364999999999998e-05 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785379+0000", 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4054000000000003e-05 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.125375336000005, 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0060500800000000002, 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:58.954 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780306+0000", 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780313+0000", 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-06 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1739999999999999e-06 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782543+0000", 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022253419999999999 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010785699999999999 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.786349+0000", 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036978829999999999 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.033673296000003, 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058824769999999997, 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.958 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872002+0000", 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5149999999999999e-06 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872008+0000", 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.66e-06 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872022+0000", 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3932e-05 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872031+0000", 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5259999999999999e-06 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872111+0000", 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9412000000000005e-05 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872231+0000", 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119843 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872340+0000", 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109442 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.875579+0000", 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032383899999999998 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877781+0000", 2024-09-10T10:27:58.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022022299999999999 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877835+0000", 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4419000000000002e-05 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877883+0000", 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8108000000000001e-05 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.032584464999999, 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049968149999999999, 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7300000000000005e-07 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:27:58.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.245e-06 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3898e-05 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873116+0000", 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8050000000000007e-06 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873184+0000", 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7809000000000001e-05 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873257+0000", 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2602999999999996e-05 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873374+0000", 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011744900000000001 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.876282+0000", 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029079459999999998 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878003+0000", 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001720886 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878054+0000", 2024-09-10T10:27:58.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0596999999999997e-05 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878086+0000", 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2804e-05 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.958192093000001, 2024-09-10T10:27:58.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029964739999999998, 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947484+0000", 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.857e-06 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947490+0000", 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9789999999999998e-06 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947504+0000", 2024-09-10T10:27:58.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3942e-05 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947512+0000", 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5620000000000004e-06 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947599+0000", 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6525000000000003e-05 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947714+0000", 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114709 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947823+0000", 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109477 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950336+0000", 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025131509999999999 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950386+0000", 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9974e-05 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950440+0000", 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4011000000000002e-05 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950478+0000", 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8287000000000001e-05 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.974 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.956751695000001, 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043004360000000004, 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948923+0000", 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8999999999999996e-07 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948928+0000", 2024-09-10T10:27:58.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-10T10:27:58.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948940+0000", 2024-09-10T10:27:58.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1759e-05 2024-09-10T10:27:58.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948949+0000", 2024-09-10T10:27:58.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5149999999999995e-06 2024-09-10T10:27:58.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949028+0000", 2024-09-10T10:27:58.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8182999999999993e-05 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949137+0000", 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109065 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949245+0000", 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108047 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950193+0000", 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094856200000000002 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953173+0000", 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029801960000000001 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953200+0000", 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6707999999999999e-05 2024-09-10T10:27:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953223+0000", 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2758000000000001e-05 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.956696182999998, 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046333320000000004, 2024-09-10T10:27:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948979+0000", 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-07 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6490000000000001e-06 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948990+0000", 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2560000000000002e-06 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948996+0000", 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9739999999999999e-06 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949496+0000", 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049934299999999997 2024-09-10T10:27:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949588+0000", 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1762999999999995e-05 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949729+0000", 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014145299999999999 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950984+0000", 2024-09-10T10:27:58.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012548419999999999 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953505+0000", 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025208129999999998 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953568+0000", 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3695000000000006e-05 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953611+0000", 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2913000000000001e-05 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.984644568, 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048620219999999997, 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.987 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4899999999999995e-07 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921034+0000", 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7050000000000001e-06 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921043+0000", 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1840000000000005e-06 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921049+0000", 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4459999999999998e-06 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921575+0000", 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052602300000000003 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921646+0000", 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0475000000000005e-05 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921769+0000", 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123347 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923077+0000", 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013078759999999999 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925794+0000", 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027172310000000001 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925853+0000", 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8739999999999997e-05 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925892+0000", 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8445999999999997e-05 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.984607533999998, 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032737339999999999, 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:27:58.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9919999999999999e-06 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921080+0000", 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0063e-05 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921086+0000", 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:58.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921142+0000", 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6174999999999997e-05 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013480499999999999 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921405+0000", 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012819200000000001 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923001+0000", 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015962249999999999 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924243+0000", 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012425820000000001 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924300+0000", 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6901999999999998e-05 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924340+0000", 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9918e-05 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(2.4 epoch 26/18 rep_tid 19 entries 26'1 (0'0) error 2:218afd09:::203.00000003:head by mds.3.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 38.982469088999999, 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030589089999999998, 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:58.998 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:58.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923206+0000", 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7249999999999997e-06 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923229+0000", 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5905000000000001e-05 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923241+0000", 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1990999999999999e-05 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923317+0000", 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5622000000000002e-05 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926264+0000", 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029470170000000001 2024-09-10T10:27:59.001 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.962111088999997, 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050642550000000001, 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943564+0000", 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.13e-07 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943576+0000", 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2187e-05 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943585+0000", 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9279999999999999e-06 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943592+0000", 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0110000000000003e-06 2024-09-10T10:27:59.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943681+0000", 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9487999999999994e-05 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943767+0000", 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5866999999999996e-05 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943882+0000", 2024-09-10T10:27:59.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011443299999999999 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.947042+0000", 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031600629999999998 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948540+0000", 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001498451 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948581+0000", 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0839e-05 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948627+0000", 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6375000000000002e-05 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 32.962042766000003, 2024-09-10T10:27:59.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051386089999999997, 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943632+0000", 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1500000000000004e-07 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943643+0000", 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0954e-05 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943652+0000", 2024-09-10T10:27:59.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4929999999999998e-06 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943657+0000", 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6509999999999998e-06 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944230+0000", 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057322999999999999 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944340+0000", 2024-09-10T10:27:59.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010986 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948770+0000", 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044297060000000003 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.011 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:59.013 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:27:59.013 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:59.011+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:59.114 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:59.113+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:59.164 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:59.164 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:59.164 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:59.164 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:59.174 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:27:59.215 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:27:59.214+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:27:59.284 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:59.284 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:59.284 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:59.284 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:59.284 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:59.293 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:27:59.316 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:59.315+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:59.404 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:59.404 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:27:59.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.622633961999995, 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049304329999999997, 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780094+0000", 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780107+0000", 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3703000000000001e-05 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780116+0000", 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5210000000000002e-06 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6340000000000007e-06 2024-09-10T10:27:59.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001735838 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782000+0000", 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013984100000000001 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782131+0000", 2024-09-10T10:27:59.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000131025 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783737+0000", 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016054179999999999 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784937+0000", 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001200638 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784987+0000", 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9591e-05 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785024+0000", 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6751000000000001e-05 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:27:59.410 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.622594507000002, 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052929559999999997, 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780136+0000", 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9950000000000001e-06 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780144+0000", 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.174e-06 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4949999999999999e-06 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781163+0000", 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001013438 2024-09-10T10:27:59.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781279+0000", 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011579200000000001 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781388+0000", 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108834 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783243+0000", 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00185491 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785355+0000", 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021117990000000001 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785386+0000", 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1612000000000002e-05 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785426+0000", 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9474999999999997e-05 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:8 2.10 e24/18 2:09401a81:::601.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.620403276999994, 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005352224, 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4400000000000001e-07 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782467+0000", 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142452 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782598+0000", 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013078300000000001 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782604+0000", 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.533e-06 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7073999999999997e-05 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782668+0000", 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6861999999999999e-05 2024-09-10T10:27:59.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787632+0000", 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049636510000000003 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787676+0000", 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4425000000000002e-05 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.420 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:9 2.10 e24/18 2:09401a81:::601.00000000:head v 24'4, mlcod=0'0)", 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.619766261999999, 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047438960000000001, 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782962+0000", 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4499999999999997e-07 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782967+0000", 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2000000000000002e-06 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782980+0000", 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3685e-05 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782987+0000", 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4690000000000002e-06 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783112+0000", 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012481300000000001 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783130+0000", 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7753000000000001e-05 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787678+0000", 2024-09-10T10:27:59.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045487419999999997 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787705+0000", 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6588999999999999e-05 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:10 2.10 e24/18 2:0bd6d154:::602.00000000:head v 24'5, mlcod=0'0)", 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.619659327999997, 2024-09-10T10:27:59.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00466629, 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1300000000000001e-07 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6970000000000001e-06 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783309+0000", 2024-09-10T10:27:59.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000236209 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783319+0000", 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7180000000000008e-06 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783365+0000", 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6179e-05 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783383+0000", 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8382000000000001e-05 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787708+0000", 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043246059999999999 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787734+0000", 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6086000000000001e-05 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.429 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.530257134000003, 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054527359999999997, 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872472+0000", 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5409999999999999e-06 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872479+0000", 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9550000000000003e-06 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872494+0000", 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5452999999999999e-05 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872503+0000", 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1509999999999993e-06 2024-09-10T10:27:59.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872571+0000", 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7747e-05 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872704+0000", 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013269900000000001 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:59.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872804+0000", 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9934999999999998e-05 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877617+0000", 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048138069999999998 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877819+0000", 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020164799999999999 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877880+0000", 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1296000000000004e-05 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877923+0000", 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2503999999999998e-05 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.435 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.529235567000001, 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0098302110000000002, 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873493+0000", 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873498+0000", 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1619999999999999e-06 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873511+0000", 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3319000000000001e-05 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873520+0000", 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3409999999999992e-06 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873586+0000", 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6115e-05 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873660+0000", 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3103999999999997e-05 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873775+0000", 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115617 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878285+0000", 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045096049999999999 2024-09-10T10:27:59.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883229+0000", 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049443999999999998 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883283+0000", 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4262999999999999e-05 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883322+0000", 2024-09-10T10:27:59.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8417000000000002e-05 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.528746427000002, 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047062800000000002, 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873982+0000", 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2900000000000003e-07 2024-09-10T10:27:59.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873986+0000", 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3939999999999998e-06 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873995+0000", 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0750000000000004e-06 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874002+0000", 2024-09-10T10:27:59.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874061+0000", 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8770999999999997e-05 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874178+0000", 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117108 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:59.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874290+0000", 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112503 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878269+0000", 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039784870000000002 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878593+0000", 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032385299999999998 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878644+0000", 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1790999999999999e-05 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878687+0000", 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2738999999999999e-05 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.446 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.528146147999998, 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089185449999999999, 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874582+0000", 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1699999999999997e-07 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874592+0000", 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0604999999999999e-05 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874602+0000", 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874610+0000", 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1919999999999999e-06 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874696+0000", 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6033999999999997e-05 2024-09-10T10:27:59.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874809+0000", 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113373 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874923+0000", 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114077 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878289+0000", 2024-09-10T10:27:59.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033662380000000001 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883420+0000", 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051305760000000004 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883465+0000", 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5252e-05 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883500+0000", 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4415000000000002e-05 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.454282827, 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073932399999999997, 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.452 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948446+0000", 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3400000000000001e-06 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948451+0000", 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6219999999999997e-06 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948464+0000", 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2955e-05 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948472+0000", 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0730000000000005e-06 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948534+0000", 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1727e-05 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948649+0000", 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114991 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948757+0000", 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107935 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950179+0000", 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014219440000000001 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:27:59.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055817970000000003 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955794+0000", 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3012000000000002e-05 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955838+0000", 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3844000000000002e-05 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.453744008000001, 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069680200000000001, 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948984+0000", 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6100000000000001e-07 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948993+0000", 2024-09-10T10:27:59.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9809999999999992e-06 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949002+0000", 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8789999999999992e-06 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949008+0000", 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2759999999999997e-06 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949057+0000", 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9159999999999997e-05 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949128+0000", 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1292999999999997e-05 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949261+0000", 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132564 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950220+0000", 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00095953300000000004 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955890+0000", 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056692790000000002 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955928+0000", 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8278000000000002e-05 2024-09-10T10:27:59.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955951+0000", 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3215999999999999e-05 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 39.481767498000004, 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050489669999999997, 2024-09-10T10:27:59.463 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5799999999999999e-07 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920964+0000", 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6720000000000002e-06 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920971+0000", 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5140000000000004e-06 2024-09-10T10:27:59.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920977+0000", 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0100000000000001e-06 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921033+0000", 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5899000000000001e-05 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921172+0000", 2024-09-10T10:27:59.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139011 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921288+0000", 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011605200000000001 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922851+0000", 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001562499 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925915+0000", 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003063727 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925969+0000", 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4083999999999999e-05 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926009+0000", 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9941000000000003e-05 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.468 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 39.481160641000002, 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045836510000000002, 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4499999999999996e-07 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921572+0000", 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3239999999999996e-06 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921581+0000", 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9190000000000005e-06 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921589+0000", 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0649999999999999e-06 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921635+0000", 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6863999999999997e-05 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921705+0000", 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9046000000000003e-05 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921818+0000", 2024-09-10T10:27:59.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011313499999999999 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923033+0000", 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012153590000000001 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926055+0000", 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030223310000000001 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926104+0000", 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8421999999999998e-05 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926150+0000", 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6440999999999998e-05 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:59.474 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 39.480721697, 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051621749999999997, 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:59.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3600000000000003e-07 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922011+0000", 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4449999999999998e-06 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922019+0000", 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0930000000000003e-06 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922025+0000", 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5060000000000001e-06 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922076+0000", 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0671000000000003e-05 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922175+0000", 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9611000000000006e-05 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922278+0000", 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102565 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925217+0000", 2024-09-10T10:27:59.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029384670000000002 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927074+0000", 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018579499999999999 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927129+0000", 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4428000000000001e-05 2024-09-10T10:27:59.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927168+0000", 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8803000000000001e-05 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 39.480686026999997, 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059957930000000001, 2024-09-10T10:27:59.480 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922042+0000", 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922045+0000", 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4209999999999999e-06 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922052+0000", 2024-09-10T10:27:59.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1430000000000001e-06 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922059+0000", 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2139999999999999e-06 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922495+0000", 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043517499999999999 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922563+0000", 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8084999999999996e-05 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922679+0000", 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011609599999999999 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927208+0000", 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045292170000000003 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927952+0000", 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00074421700000000003 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928001+0000", 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9004e-05 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928037+0000", 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.578e-05 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.485 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 39.480659267999997, 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045268130000000002, 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922078+0000", 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922085+0000", 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3710000000000002e-06 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922091+0000", 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0340000000000002e-06 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922144+0000", 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3106999999999998e-05 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922248+0000", 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010373400000000001 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922399+0000", 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015127200000000001 2024-09-10T10:27:59.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925201+0000", 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028019189999999999 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926514+0000", 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001312279 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926561+0000", 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000002e-05 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926595+0000", 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3408000000000003e-05 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 39.480024419999999, 2024-09-10T10:27:59.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005322116, 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:59.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999996e-07 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922707+0000", 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.766e-06 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922717+0000", 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7229999999999999e-06 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922724+0000", 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8410000000000002e-06 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922792+0000", 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8557000000000001e-05 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922912+0000", 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119354 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928025+0000", 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051131919999999999 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:27:59.495 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 39.479548266999998, 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049271260000000004, 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:27:59.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923180+0000", 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923183+0000", 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4000000000000001e-06 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923196+0000", 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3127e-05 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923203+0000", 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.939e-06 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923270+0000", 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7087000000000003e-05 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923359+0000", 2024-09-10T10:27:59.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9054999999999994e-05 2024-09-10T10:27:59.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928106+0000", 2024-09-10T10:27:59.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047469239999999996 2024-09-10T10:27:59.499 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.500 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.500 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.500 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:59.500 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 39.479087626000002, 2024-09-10T10:27:59.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045860270000000003, 2024-09-10T10:27:59.500 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.500 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2500000000000005e-07 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923644+0000", 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8240000000000001e-06 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923659+0000", 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5358000000000001e-05 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923666+0000", 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9659999999999999e-06 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923725+0000", 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8674000000000001e-05 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923850+0000", 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000124797 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928226+0000", 2024-09-10T10:27:59.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043757830000000003 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.461565190999998, 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005230859, 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:27:59.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:27:59.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941164+0000", 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.863e-06 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941172+0000", 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2360000000000004e-06 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941190+0000", 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7812000000000002e-05 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941200+0000", 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7599999999999997e-06 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:27:59.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941350+0000", 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015070400000000001 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941501+0000", 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015044200000000001 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000167076 2024-09-10T10:27:59.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943487+0000", 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018190260000000001 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946297+0000", 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028102840000000001 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946349+0000", 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1649e-05 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946393+0000", 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4007000000000001e-05 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:27:59.511 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:27:59.512 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:59.513 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:27:59.514 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:59.512+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:59.614 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:27:59.613+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:27:59.620 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:59.620 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:59.621 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:27:59.621 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:59.629 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:27:59.715 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:27:59.714+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:27:59.726 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:27:59.733 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:27:59.733 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:27:59.733 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:27:59.733 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:27:59.734 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:27:59.743 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:27:59.815 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:27:59.814+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:27:59.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.826+0000 7f865ac8d640 1 Processor -- start 2024-09-10T10:27:59.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.826+0000 7f865ac8d640 1 -- start start 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f865ac8d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8654154c10 0x7f865415b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f865ac8d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86540cca70 0x7f8654073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f865ac8d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f86540cafa0 0x7f8654073880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f865ac8d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f865415de60 con 0x7f8654154c10 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f865ac8d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f865415e0b0 con 0x7f86540cafa0 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f865ac8d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f865415e390 con 0x7f86540cca70 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f8658a29640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8654154c10 0x7f865415b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f8658a29640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8654154c10 0x7f865415b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34924/0 (socket says 172.21.15.120:34924) 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f8658a29640 1 -- 172.21.15.120:0/128898127 learned_addr learned my addr 172.21.15.120:0/128898127 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f8653fff640 1 --2- 172.21.15.120:0/128898127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86540cca70 0x7f8654073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.827+0000 7f865922a640 1 --2- 172.21.15.120:0/128898127 >> v2:172.21.15.134:3300/0 conn(0x7f86540cafa0 0x7f8654073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.828+0000 7f8658a29640 1 -- 172.21.15.120:0/128898127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86540cca70 msgr2=0x7f8654073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:59.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.828+0000 7f8658a29640 1 --2- 172.21.15.120:0/128898127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86540cca70 0x7f8654073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.828+0000 7f8658a29640 1 -- 172.21.15.120:0/128898127 >> v2:172.21.15.134:3300/0 conn(0x7f86540cafa0 msgr2=0x7f8654073880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:27:59.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.828+0000 7f8658a29640 1 --2- 172.21.15.120:0/128898127 >> v2:172.21.15.134:3300/0 conn(0x7f86540cafa0 0x7f8654073880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.828+0000 7f8658a29640 1 -- 172.21.15.120:0/128898127 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865415e670 con 0x7f8654154c10 2024-09-10T10:27:59.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.828+0000 7f8658a29640 1 --2- 172.21.15.120:0/128898127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8654154c10 0x7f865415b370 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86540d1310 tx=0x7f864802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17380aeae3c9c266 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f86537fe640 1 -- 172.21.15.120:0/128898127 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648037070 con 0x7f8654154c10 2024-09-10T10:27:59.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f86537fe640 1 -- 172.21.15.120:0/128898127 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8648032440 con 0x7f8654154c10 2024-09-10T10:27:59.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f86537fe640 1 -- 172.21.15.120:0/128898127 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648032810 con 0x7f8654154c10 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f865ac8d640 1 -- 172.21.15.120:0/128898127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8654154c10 msgr2=0x7f865415b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f865ac8d640 1 --2- 172.21.15.120:0/128898127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8654154c10 0x7f865415b370 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86540d1310 tx=0x7f864802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f865ac8d640 1 -- 172.21.15.120:0/128898127 shutdown_connections 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f865ac8d640 1 --2- 172.21.15.120:0/128898127 >> v2:172.21.15.134:3300/0 conn(0x7f86540cafa0 0x7f8654073880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f865ac8d640 1 --2- 172.21.15.120:0/128898127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86540cca70 0x7f8654073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f865ac8d640 1 --2- 172.21.15.120:0/128898127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8654154c10 0x7f865415b370 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f865ac8d640 1 -- 172.21.15.120:0/128898127 >> 172.21.15.120:0/128898127 conn(0x7f865406ce80 msgr2=0x7f865406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f865ac8d640 1 -- 172.21.15.120:0/128898127 shutdown_connections 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.829+0000 7f865ac8d640 1 -- 172.21.15.120:0/128898127 wait complete. 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.830+0000 7f865ac8d640 1 Processor -- start 2024-09-10T10:27:59.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.831+0000 7f865ac8d640 1 -- start start 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.831+0000 7f865ac8d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86540cafa0 0x7f86540c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.831+0000 7f865ac8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86540cca70 0x7f865415cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.831+0000 7f865ac8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8654154c10 0x7f865415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.831+0000 7f865ac8d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8654154ff0 con 0x7f8654154c10 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.831+0000 7f865ac8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f865415ec30 con 0x7f86540cca70 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.831+0000 7f865ac8d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f86540cb6f0 con 0x7f86540cafa0 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f865922a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8654154c10 0x7f865415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f865922a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8654154c10 0x7f865415d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34932/0 (socket says 172.21.15.120:34932) 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f865922a640 1 -- 172.21.15.120:0/2060085636 learned_addr learned my addr 172.21.15.120:0/2060085636 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f8658a29640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86540cafa0 0x7f86540c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f8653fff640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86540cca70 0x7f865415cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f865922a640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86540cafa0 msgr2=0x7f86540c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f865922a640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86540cafa0 0x7f86540c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f865922a640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86540cca70 msgr2=0x7f865415cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f865922a640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86540cca70 0x7f865415cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f865922a640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86541e4f50 con 0x7f8654154c10 2024-09-10T10:27:59.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f865922a640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8654154c10 0x7f865415d4b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f864c00ef00 tx=0x7f864c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c018020 con 0x7f8654154c10 2024-09-10T10:27:59.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f864c014400 con 0x7f8654154c10 2024-09-10T10:27:59.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c0147f0 con 0x7f8654154c10 2024-09-10T10:27:59.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.832+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86541e5230 con 0x7f8654154c10 2024-09-10T10:27:59.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.833+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865415c490 con 0x7f8654154c10 2024-09-10T10:27:59.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.833+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f865415b370 con 0x7f8654154c10 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.834+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f864c01b020 con 0x7f8654154c10 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.835+0000 7f86517fa640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f863806be30 0x7f863806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.835+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6764+0+0 (secure 0 0 0) 0x7f864c08b900 con 0x7f8654154c10 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.835+0000 7f8658a29640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f863806be30 0x7f863806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.835+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v40) ==== 54+0+16 (secure 0 0 0) 0x7f864c022030 con 0x7f8654154c10 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.835+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f865415c660 0x7f865414e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.835+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86541e66b0 con 0x7f865415c660 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.835+0000 7f8653fff640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f865415c660 0x7f865414e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.835+0000 7f8658a29640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f863806be30 0x7f863806e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8648034000 tx=0x7f864802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.836+0000 7f8653fff640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f865415c660 0x7f865414e0d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.836+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86541e66b0 con 0x7f865415c660 2024-09-10T10:27:59.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.851+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f86541e63d0 con 0x7f865415c660 2024-09-10T10:27:59.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.853+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f86541e63d0 con 0x7f865415c660 2024-09-10T10:27:59.854 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:27:59.854 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:27:59.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.853+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f865415ca40 0x7f86541f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.853+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f865414eb20 con 0x7f865415ca40 2024-09-10T10:27:59.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.853+0000 7f8658a29640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f865415ca40 0x7f86541f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.853+0000 7f8658a29640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f865415ca40 0x7f86541f3f10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.854+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8648032540 con 0x7f865415ca40 2024-09-10T10:27:59.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.869+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f865415a5e0 con 0x7f865415ca40 2024-09-10T10:27:59.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.871+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f865415a5e0 con 0x7f865415ca40 2024-09-10T10:27:59.872 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:27:59.873 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:27:59.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.871+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f86541f4810 0x7f86541f0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.871+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f86541f1180 con 0x7f86541f4810 2024-09-10T10:27:59.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.871+0000 7f8653fff640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f86541f4810 0x7f86541f0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.871+0000 7f8653fff640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f86541f4810 0x7f86541f0c40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.872+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86541f1180 con 0x7f86541f4810 2024-09-10T10:27:59.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.887+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8654062630 con 0x7f86541f4810 2024-09-10T10:27:59.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.888+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8654062630 con 0x7f86541f4810 2024-09-10T10:27:59.890 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:27:59.890 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:27:59.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.889+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f8654063110 0x7f86540c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.889+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8654063590 con 0x7f8654063110 2024-09-10T10:27:59.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.889+0000 7f865922a640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f8654063110 0x7f86540c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.889+0000 7f865922a640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f8654063110 0x7f86540c5b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.890+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8654063590 con 0x7f8654063110 2024-09-10T10:27:59.895 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:27:59.895 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:27:59.904 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:27:59.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.905+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8654063590 con 0x7f8654063110 2024-09-10T10:27:59.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.907+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8654063590 con 0x7f8654063110 2024-09-10T10:27:59.908 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:27:59.909 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:27:59.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.907+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f86540c6900 0x7f865402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.907+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8654063590 con 0x7f86540c6900 2024-09-10T10:27:59.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.907+0000 7f8658a29640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f86540c6900 0x7f865402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.908+0000 7f8658a29640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f86540c6900 0x7f865402c640 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.909+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8654063590 con 0x7f86540c6900 2024-09-10T10:27:59.916 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:27:59.915+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:27:59.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.923+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8654063590 con 0x7f86540c6900 2024-09-10T10:27:59.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.926+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8654063590 con 0x7f86540c6900 2024-09-10T10:27:59.927 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:27:59.927 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:27:59.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.926+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f865402d090 0x7f865402d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.926+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8654209910 con 0x7f865402d090 2024-09-10T10:27:59.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.926+0000 7f8653fff640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f865402d090 0x7f865402d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.927+0000 7f8653fff640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f865402d090 0x7f865402d4d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.927+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8640028a50 con 0x7f865402d090 2024-09-10T10:27:59.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.942+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8654208790 con 0x7f865402d090 2024-09-10T10:27:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.943+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8654208790 con 0x7f865402d090 2024-09-10T10:27:59.945 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:27:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:27:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.944+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f8654208eb0 0x7f86542092d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.944+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f865420ccd0 con 0x7f8654208eb0 2024-09-10T10:27:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.944+0000 7f865922a640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f8654208eb0 0x7f86542092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.944+0000 7f865922a640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f8654208eb0 0x7f86542092d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.945+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f865420ccd0 con 0x7f8654208eb0 2024-09-10T10:27:59.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.959+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f865420bb50 con 0x7f8654208eb0 2024-09-10T10:27:59.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.961+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f865420bb50 con 0x7f8654208eb0 2024-09-10T10:27:59.962 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:27:59.963 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:27:59.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.961+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f865420c270 0x7f865420c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:27:59.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.961+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8654210070 con 0x7f865420c270 2024-09-10T10:27:59.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.961+0000 7f8658a29640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f865420c270 0x7f865420c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:27:59.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.962+0000 7f8658a29640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f865420c270 0x7f865420c6b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:27:59.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.963+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8654210070 con 0x7f865420c270 2024-09-10T10:27:59.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.976+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8654205020 con 0x7f865420c270 2024-09-10T10:27:59.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.978+0000 7f86517fa640 1 -- 172.21.15.120:0/2060085636 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8654205020 con 0x7f865420c270 2024-09-10T10:27:59.980 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:27:59.981 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:27:59.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f865415c660 msgr2=0x7f865414e0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f865415c660 0x7f865414e0d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f865415ca40 msgr2=0x7f86541f3f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f865415ca40 0x7f86541f3f10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f86541f4810 msgr2=0x7f86541f0c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f86541f4810 0x7f86541f0c40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f8654063110 msgr2=0x7f86540c5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f8654063110 0x7f86540c5b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865922a640 1 -- 172.21.15.120:0/2060085636 reap_dead start 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f86540c6900 msgr2=0x7f865402c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f86540c6900 0x7f865402c640 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865922a640 1 -- 172.21.15.120:0/2060085636 reap_dead start 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f865402d090 msgr2=0x7f865402d4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f865402d090 0x7f865402d4d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f8654208eb0 msgr2=0x7f86542092d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f8654208eb0 0x7f86542092d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f865420c270 msgr2=0x7f865420c6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f865420c270 0x7f865420c6b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:27:59.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f863806be30 msgr2=0x7f863806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f863806be30 0x7f863806e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8648034000 tx=0x7f864802e000 comp rx=0 tx=0).stop 2024-09-10T10:27:59.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8654154c10 msgr2=0x7f865415d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:27:59.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 --2- 172.21.15.120:0/2060085636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8654154c10 0x7f865415d4b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f864c00ef00 tx=0x7f864c00b480 comp rx=0 tx=0).stop 2024-09-10T10:27:59.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865922a640 1 -- 172.21.15.120:0/2060085636 reap_dead start 2024-09-10T10:27:59.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 shutdown_connections 2024-09-10T10:27:59.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 >> 172.21.15.120:0/2060085636 conn(0x7f865406ce80 msgr2=0x7f8654151bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:27:59.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 shutdown_connections 2024-09-10T10:27:59.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:27:59.979+0000 7f865ac8d640 1 -- 172.21.15.120:0/2060085636 wait complete. 2024-09-10T10:28:00.019 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:00.019 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:00.019 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:00.018+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:00.028 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:00.080 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-10T10:28:00.118 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:00.117+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:00.177 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:00.177 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:00.187 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:00.219 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:00.217+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:00.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.217+0000 7f06a4ee4640 1 Processor -- start 2024-09-10T10:28:00.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.217+0000 7f06a4ee4640 1 -- start start 2024-09-10T10:28:00.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.218+0000 7f06a4ee4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f06a0158130 0x7f06a006d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:00.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.218+0000 7f06a4ee4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a00c4b80 con 0x7f06a0151b30 2024-09-10T10:28:00.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.218+0000 7f06a4ee4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a006ce50 con 0x7f06a00c72e0 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.218+0000 7f06a4ee4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f06a00c45c0 con 0x7f06a0158130 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069e59c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f06a0151b30 0x7f06a00c60c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:53074/0 (socket says 172.21.15.120:53074) 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069e59c640 1 -- 172.21.15.120:0/931877965 learned_addr learned my addr 172.21.15.120:0/931877965 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069ed9d640 1 --2- 172.21.15.120:0/931877965 >> v2:172.21.15.134:3300/0 conn(0x7f06a0158130 0x7f06a006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069d59a640 1 -- 172.21.15.120:0/931877965 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 884063643 0 0) 0x7f0690001110 con 0x7f06a0151b30 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069d59a640 1 -- 172.21.15.120:0/931877965 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0688005730 con 0x7f06a0151b30 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069d59a640 1 -- 172.21.15.120:0/931877965 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2524561353 0 0) 0x7f06a006ce50 con 0x7f06a00c72e0 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069d59a640 1 -- 172.21.15.120:0/931877965 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0690001110 con 0x7f06a00c72e0 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069ed9d640 1 -- 172.21.15.120:0/931877965 >> v1:172.21.15.134:6789/0 conn(0x7f06a00c72e0 legacy=0x7f06a0157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069ed9d640 1 -- 172.21.15.120:0/931877965 >> v1:172.21.15.120:6789/0 conn(0x7f06a0151b30 legacy=0x7f06a00c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069d59a640 1 -- 172.21.15.120:0/931877965 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2943855179 0 0) 0x7f0688005730 con 0x7f06a0151b30 2024-09-10T10:28:00.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.219+0000 7f069ed9d640 1 -- 172.21.15.120:0/931877965 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a00c8bc0 con 0x7f06a0158130 2024-09-10T10:28:00.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.220+0000 7f069ed9d640 1 --2- 172.21.15.120:0/931877965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0158130 0x7f06a006d980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0698009d90 tx=0x7f069802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4961711e930ea02f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:00.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.220+0000 7f069d59a640 1 -- 172.21.15.120:0/931877965 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690001840 con 0x7f06a0158130 2024-09-10T10:28:00.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.220+0000 7f069d59a640 1 -- 172.21.15.120:0/931877965 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0698007540 con 0x7f06a0158130 2024-09-10T10:28:00.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.220+0000 7f069d59a640 1 -- 172.21.15.120:0/931877965 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698007910 con 0x7f06a0158130 2024-09-10T10:28:00.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.221+0000 7f06a4ee4640 1 -- 172.21.15.120:0/931877965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0158130 msgr2=0x7f06a006d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:00.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.221+0000 7f06a4ee4640 1 --2- 172.21.15.120:0/931877965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0158130 0x7f06a006d980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0698009d90 tx=0x7f069802a680 comp rx=0 tx=0).stop 2024-09-10T10:28:00.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.221+0000 7f06a4ee4640 1 -- 172.21.15.120:0/931877965 shutdown_connections 2024-09-10T10:28:00.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.221+0000 7f06a4ee4640 1 --2- 172.21.15.120:0/931877965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0158130 0x7f06a006d980 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:00.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.221+0000 7f06a4ee4640 1 -- 172.21.15.120:0/931877965 >> 172.21.15.120:0/931877965 conn(0x7f06a0067950 msgr2=0x7f06a0067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:00.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.221+0000 7f06a4ee4640 1 -- 172.21.15.120:0/931877965 shutdown_connections 2024-09-10T10:28:00.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.221+0000 7f06a4ee4640 1 -- 172.21.15.120:0/931877965 wait complete. 2024-09-10T10:28:00.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.222+0000 7f06a4ee4640 1 Processor -- start 2024-09-10T10:28:00.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.222+0000 7f06a4ee4640 1 -- start start 2024-09-10T10:28:00.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.223+0000 7f06a4ee4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a00c72e0 0x7f06a014be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:00.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.223+0000 7f06a4ee4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a0151b30 0x7f06a01f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:00.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.223+0000 7f06a4ee4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0158130 0x7f06a01f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:00.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.223+0000 7f06a4ee4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06a006d3c0 con 0x7f06a0151b30 2024-09-10T10:28:00.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.223+0000 7f06a4ee4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06a00c79c0 con 0x7f06a0158130 2024-09-10T10:28:00.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.223+0000 7f06a4ee4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f06a006d540 con 0x7f06a00c72e0 2024-09-10T10:28:00.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.223+0000 7f069dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a0151b30 0x7f06a01f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:00.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.223+0000 7f069dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a0151b30 0x7f06a01f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34944/0 (socket says 172.21.15.120:34944) 2024-09-10T10:28:00.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.223+0000 7f069dd9b640 1 -- 172.21.15.120:0/901991260 learned_addr learned my addr 172.21.15.120:0/901991260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:00.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.223+0000 7f069ed9d640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0158130 0x7f06a01f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f069e59c640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a00c72e0 0x7f06a014be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f069dd9b640 1 -- 172.21.15.120:0/901991260 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a00c72e0 msgr2=0x7f06a014be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f069dd9b640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a00c72e0 0x7f06a014be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f069dd9b640 1 -- 172.21.15.120:0/901991260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0158130 msgr2=0x7f06a01f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f069dd9b640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0158130 0x7f06a01f39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f069dd9b640 1 -- 172.21.15.120:0/901991260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a014c710 con 0x7f06a0151b30 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f069e59c640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a00c72e0 0x7f06a014be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f069dd9b640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a0151b30 0x7f06a01f3470 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f069400c430 tx=0x7f069400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f068effd640 1 -- 172.21.15.120:0/901991260 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694017070 con 0x7f06a0151b30 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f068effd640 1 -- 172.21.15.120:0/901991260 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06940052d0 con 0x7f06a0151b30 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f06a4ee4640 1 -- 172.21.15.120:0/901991260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a01f00b0 con 0x7f06a0151b30 2024-09-10T10:28:00.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f068effd640 1 -- 172.21.15.120:0/901991260 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694003810 con 0x7f06a0151b30 2024-09-10T10:28:00.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.224+0000 7f06a4ee4640 1 -- 172.21.15.120:0/901991260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a01f0670 con 0x7f06a0151b30 2024-09-10T10:28:00.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.225+0000 7f06a4ee4640 1 -- 172.21.15.120:0/901991260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a00ca2e0 con 0x7f06a0151b30 2024-09-10T10:28:00.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.226+0000 7f068effd640 1 -- 172.21.15.120:0/901991260 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f069401a070 con 0x7f06a0151b30 2024-09-10T10:28:00.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.226+0000 7f068effd640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f068006be60 0x7f068006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:00.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.227+0000 7f068effd640 1 -- 172.21.15.120:0/901991260 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6764+0+0 (secure 0 0 0) 0x7f069408a610 con 0x7f06a0151b30 2024-09-10T10:28:00.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.227+0000 7f069e59c640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f068006be60 0x7f068006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:00.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.227+0000 7f069e59c640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f068006be60 0x7f068006e320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0690000f60 tx=0x7f0690009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:00.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.231+0000 7f068effd640 1 -- 172.21.15.120:0/901991260 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0694058550 con 0x7f06a0151b30 2024-09-10T10:28:00.301 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:00.301 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:00.310 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:00.351 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:00.350+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:00.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:00.418+0000 7f06a4ee4640 1 -- 172.21.15.120:0/901991260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f06a00756f0 con 0x7f06a0151b30 2024-09-10T10:28:00.420 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:00.419+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:00.426 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:00.426 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:00.435 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:00.520 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:00.519+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:00.579 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:00.580 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:00.589 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:00.632 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:00.631+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:00.692 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:00.692 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:00.701 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:00.722 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:00.721+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:00.823 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:00.822+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:00.846 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:00.846 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:00.855 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:00.924 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:00.923+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:00.958 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:00.958 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:00.967 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:01.024 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:01.023+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:01.070 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:01.070 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:01.080 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.082+0000 7f068effd640 1 -- 172.21.15.120:0/901991260 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v41) ==== 87+0+0 (secure 0 0 0) 0x7f069405c850 con 0x7f06a0151b30 2024-09-10T10:28:01.083 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-10T10:28:01.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 -- 172.21.15.120:0/901991260 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f068006be60 msgr2=0x7f068006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:01.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f068006be60 0x7f068006e320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0690000f60 tx=0x7f0690009040 comp rx=0 tx=0).stop 2024-09-10T10:28:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 -- 172.21.15.120:0/901991260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a0151b30 msgr2=0x7f06a01f3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a0151b30 0x7f06a01f3470 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f069400c430 tx=0x7f069400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:28:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 -- 172.21.15.120:0/901991260 shutdown_connections 2024-09-10T10:28:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f068006be60 0x7f068006e320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06a0158130 0x7f06a01f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a0151b30 0x7f06a01f3470 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 --2- 172.21.15.120:0/901991260 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06a00c72e0 0x7f06a014be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 -- 172.21.15.120:0/901991260 >> 172.21.15.120:0/901991260 conn(0x7f06a0067950 msgr2=0x7f06a0157a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 -- 172.21.15.120:0/901991260 shutdown_connections 2024-09-10T10:28:01.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:01.084+0000 7f06a4ee4640 1 -- 172.21.15.120:0/901991260 wait complete. 2024-09-10T10:28:01.140 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:01.139+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:01.184 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:01.185 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:01.193 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:01.237 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:01.236+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:01.297 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:01.297 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:01.305 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:01.347 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:01.346+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:01.427 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:01.426+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:01.449 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:01.449 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:01.458 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:01.542 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:01.541+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:01.601 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:01.602 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:01.610 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:01.651 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:01.650+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:01.730 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:01.729+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:01.753 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:01.753 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:01.762 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:01.831 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:01.830+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:01.868 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:01.868 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:01.877 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:01.931 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:01.931+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:01.986 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:01.987 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:01.995 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:28:01.997 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:02.032 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:02.031+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:02.078 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:28:02.078 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:28:02.079 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-10T10:28:02.079 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-10T10:28:02.133 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:02.132+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:02.149 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:02.150 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:02.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.158+0000 7f762c986640 1 Processor -- start 2024-09-10T10:28:02.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.159+0000 7f762c986640 1 -- start start 2024-09-10T10:28:02.161 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:02.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f762c986640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f76281589b0 0x7f76280c7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f762c986640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f76280731e0 0x7f7628158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f762c986640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f762806bdb0 con 0x7f7628072e00 2024-09-10T10:28:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f762c986640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f762815c3c0 con 0x7f76280731e0 2024-09-10T10:28:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f762c986640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f762806c090 con 0x7f76281589b0 2024-09-10T10:28:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f7626d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f76280731e0 0x7f7628158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f7626d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f76280731e0 0x7f7628158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34952/0 (socket says 172.21.15.120:34952) 2024-09-10T10:28:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f762659c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f7628072e00 0x7f762815c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:53076/0 (socket says 172.21.15.120:53076) 2024-09-10T10:28:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f7626d9d640 1 -- 172.21.15.120:0/1266230620 learned_addr learned my addr 172.21.15.120:0/1266230620 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f7625d9b640 1 --2- 172.21.15.120:0/1266230620 >> v2:172.21.15.134:3300/0 conn(0x7f76281589b0 0x7f76280c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f762559a640 1 -- 172.21.15.120:0/1266230620 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414214458 0 0) 0x7f762806bdb0 con 0x7f7628072e00 2024-09-10T10:28:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f762559a640 1 -- 172.21.15.120:0/1266230620 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7608003520 con 0x7f7628072e00 2024-09-10T10:28:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f7625d9b640 1 -- 172.21.15.120:0/1266230620 >> v2:172.21.15.120:3300/0 conn(0x7f76280731e0 msgr2=0x7f7628158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f7625d9b640 1 --2- 172.21.15.120:0/1266230620 >> v2:172.21.15.120:3300/0 conn(0x7f76280731e0 0x7f7628158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f7625d9b640 1 -- 172.21.15.120:0/1266230620 >> v1:172.21.15.120:6789/0 conn(0x7f7628072e00 legacy=0x7f762815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.160+0000 7f7625d9b640 1 -- 172.21.15.120:0/1266230620 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762815c050 con 0x7f76281589b0 2024-09-10T10:28:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.161+0000 7f7625d9b640 1 --2- 172.21.15.120:0/1266230620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76281589b0 0x7f76280c7940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f762806fff0 tx=0x7f761002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee113b1da23ef55c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.161+0000 7f762559a640 1 -- 172.21.15.120:0/1266230620 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610037070 con 0x7f76281589b0 2024-09-10T10:28:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.161+0000 7f762559a640 1 -- 172.21.15.120:0/1266230620 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7610032440 con 0x7f76281589b0 2024-09-10T10:28:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.161+0000 7f762559a640 1 -- 172.21.15.120:0/1266230620 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610032810 con 0x7f76281589b0 2024-09-10T10:28:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.162+0000 7f762c986640 1 -- 172.21.15.120:0/1266230620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76281589b0 msgr2=0x7f76280c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.162+0000 7f762c986640 1 --2- 172.21.15.120:0/1266230620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76281589b0 0x7f76280c7940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f762806fff0 tx=0x7f761002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.162+0000 7f762c986640 1 -- 172.21.15.120:0/1266230620 shutdown_connections 2024-09-10T10:28:02.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.162+0000 7f762c986640 1 --2- 172.21.15.120:0/1266230620 >> v2:172.21.15.120:3300/0 conn(0x7f76280731e0 0x7f7628158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.162+0000 7f762c986640 1 --2- 172.21.15.120:0/1266230620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76281589b0 0x7f76280c7940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.162+0000 7f762c986640 1 -- 172.21.15.120:0/1266230620 >> 172.21.15.120:0/1266230620 conn(0x7f7628067950 msgr2=0x7f7628067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.162+0000 7f762c986640 1 -- 172.21.15.120:0/1266230620 shutdown_connections 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.162+0000 7f762c986640 1 -- 172.21.15.120:0/1266230620 wait complete. 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.164+0000 7f762c986640 1 Processor -- start 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.164+0000 7f762c986640 1 -- start start 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f762c986640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7628072e00 0x7f7628159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f762c986640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76280731e0 0x7f76281f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f762c986640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76281589b0 0x7f762815a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f762c986640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76280ca940 con 0x7f7628072e00 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f762c986640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f76280ca7c0 con 0x7f76281589b0 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f762c986640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f762806d970 con 0x7f76280731e0 2024-09-10T10:28:02.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f762659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7628072e00 0x7f7628159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f7625d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76280731e0 0x7f76281f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f762659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7628072e00 0x7f7628159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34966/0 (socket says 172.21.15.120:34966) 2024-09-10T10:28:02.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f762659c640 1 -- 172.21.15.120:0/2286393107 learned_addr learned my addr 172.21.15.120:0/2286393107 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:02.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f7626d9d640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76281589b0 0x7f762815a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f7625d9b640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76281589b0 msgr2=0x7f762815a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f7625d9b640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76281589b0 0x7f762815a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f7625d9b640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7628072e00 msgr2=0x7f7628159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f7625d9b640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7628072e00 0x7f7628159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.165+0000 7f7625d9b640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76281f0b50 con 0x7f76280731e0 2024-09-10T10:28:02.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.166+0000 7f7625d9b640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76280731e0 0x7f76281f4070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7610031000 tx=0x7f76100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.166+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610041030 con 0x7f76280731e0 2024-09-10T10:28:02.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.166+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f761003d4e0 con 0x7f76280731e0 2024-09-10T10:28:02.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.166+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76281f0e30 con 0x7f76280731e0 2024-09-10T10:28:02.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.166+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610037040 con 0x7f76280731e0 2024-09-10T10:28:02.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.166+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f76281f3870 con 0x7f76280731e0 2024-09-10T10:28:02.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.167+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f762806dc10 con 0x7f76280731e0 2024-09-10T10:28:02.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.168+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f761003d8b0 con 0x7f76280731e0 2024-09-10T10:28:02.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.169+0000 7f761effd640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f760c06be30 0x7f760c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.169+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(41..41 src has 1..41) ==== 6764+0+0 (secure 0 0 0) 0x7f76100ab440 con 0x7f76280731e0 2024-09-10T10:28:02.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.169+0000 7f762659c640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f760c06be30 0x7f760c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.169+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v41) ==== 54+0+16 (secure 0 0 0) 0x7f761004b020 con 0x7f76280731e0 2024-09-10T10:28:02.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.169+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f76281f3ab0 0x7f76280c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.169+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76280c63e0 con 0x7f76281f3ab0 2024-09-10T10:28:02.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.169+0000 7f7626d9d640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f76281f3ab0 0x7f76280c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.170+0000 7f762659c640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f760c06be30 0x7f760c06e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7620001620 tx=0x7f7620009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.170+0000 7f7626d9d640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f76281f3ab0 0x7f76280c1ed0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.171+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76280c63e0 con 0x7f76281f3ab0 2024-09-10T10:28:02.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.188+0000 7f4056fb7640 1 Processor -- start 2024-09-10T10:28:02.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.188+0000 7f4056fb7640 1 -- start start 2024-09-10T10:28:02.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4056fb7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f40500d0570 0x7f405015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4056fb7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500d0950 0x7f40500cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4056fb7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f405015b290 con 0x7f40500c7770 2024-09-10T10:28:02.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4056fb7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f40500ccb20 con 0x7f40500d0570 2024-09-10T10:28:02.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4056fb7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f405015b570 con 0x7f40500d0950 2024-09-10T10:28:02.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4054d53640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f40500d0570 0x7f405015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f404ffff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f40500c7770 0x7f405015ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:53086/0 (socket says 172.21.15.120:53086) 2024-09-10T10:28:02.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f404ffff640 1 -- 172.21.15.120:0/413257899 learned_addr learned my addr 172.21.15.120:0/413257899 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:02.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4055554640 1 --2- 172.21.15.120:0/413257899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500d0950 0x7f40500cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4054d53640 1 -- 172.21.15.120:0/413257899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500d0950 msgr2=0x7f40500cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:02.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4054d53640 1 --2- 172.21.15.120:0/413257899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500d0950 0x7f40500cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4054d53640 1 -- 172.21.15.120:0/413257899 >> v1:172.21.15.120:6789/0 conn(0x7f40500c7770 legacy=0x7f405015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.189+0000 7f4054d53640 1 -- 172.21.15.120:0/413257899 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405015b850 con 0x7f40500d0570 2024-09-10T10:28:02.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.190+0000 7f4054d53640 1 --2- 172.21.15.120:0/413257899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40500d0570 0x7f405015ab70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f40440097a0 tx=0x7f404402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40282bdc9a0fcba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.190+0000 7f404f7fe640 1 -- 172.21.15.120:0/413257899 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044037070 con 0x7f40500d0570 2024-09-10T10:28:02.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.190+0000 7f404f7fe640 1 -- 172.21.15.120:0/413257899 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4044032440 con 0x7f40500d0570 2024-09-10T10:28:02.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.190+0000 7f404f7fe640 1 -- 172.21.15.120:0/413257899 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044032810 con 0x7f40500d0570 2024-09-10T10:28:02.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.191+0000 7f4056fb7640 1 -- 172.21.15.120:0/413257899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40500d0570 msgr2=0x7f405015ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.191+0000 7f4056fb7640 1 --2- 172.21.15.120:0/413257899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40500d0570 0x7f405015ab70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f40440097a0 tx=0x7f404402a180 comp rx=0 tx=0).stop 2024-09-10T10:28:02.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.191+0000 7f4056fb7640 1 -- 172.21.15.120:0/413257899 shutdown_connections 2024-09-10T10:28:02.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.191+0000 7f4056fb7640 1 --2- 172.21.15.120:0/413257899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500d0950 0x7f40500cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.191+0000 7f4056fb7640 1 --2- 172.21.15.120:0/413257899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40500d0570 0x7f405015ab70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.191+0000 7f4056fb7640 1 -- 172.21.15.120:0/413257899 >> 172.21.15.120:0/413257899 conn(0x7f405006ca90 msgr2=0x7f405006e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:02.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.191+0000 7f4056fb7640 1 -- 172.21.15.120:0/413257899 shutdown_connections 2024-09-10T10:28:02.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.191+0000 7f4056fb7640 1 -- 172.21.15.120:0/413257899 wait complete. 2024-09-10T10:28:02.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.192+0000 7f4056fb7640 1 Processor -- start 2024-09-10T10:28:02.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.192+0000 7f4056fb7640 1 -- start start 2024-09-10T10:28:02.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f4056fb7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500c7770 0x7f40500c2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f4056fb7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40500d0570 0x7f405015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f4056fb7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40500d0950 0x7f405014f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f4056fb7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f40500cbbe0 con 0x7f40500d0570 2024-09-10T10:28:02.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f4056fb7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f40500cba60 con 0x7f40500d0950 2024-09-10T10:28:02.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f4056fb7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f40501543e0 con 0x7f40500c7770 2024-09-10T10:28:02.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f404ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40500d0570 0x7f405015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f404ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40500d0570 0x7f405015d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34990/0 (socket says 172.21.15.120:34990) 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f404ffff640 1 -- 172.21.15.120:0/3032948837 learned_addr learned my addr 172.21.15.120:0/3032948837 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f4054d53640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500c7770 0x7f40500c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.193+0000 7f4055554640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40500d0950 0x7f405014f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f4054d53640 1 -- 172.21.15.120:0/3032948837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40500d0950 msgr2=0x7f405014f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f4054d53640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40500d0950 0x7f405014f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f4054d53640 1 -- 172.21.15.120:0/3032948837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40500d0570 msgr2=0x7f405015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f4054d53640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40500d0570 0x7f405015d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f4054d53640 1 -- 172.21.15.120:0/3032948837 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40501594a0 con 0x7f40500c7770 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f4054d53640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500c7770 0x7f40500c2170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f40500cada0 tx=0x7f40440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f404d7fa640 1 -- 172.21.15.120:0/3032948837 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044041030 con 0x7f40500c7770 2024-09-10T10:28:02.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f404d7fa640 1 -- 172.21.15.120:0/3032948837 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4044030750 con 0x7f40500c7770 2024-09-10T10:28:02.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f4056fb7640 1 -- 172.21.15.120:0/3032948837 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4050159780 con 0x7f40500c7770 2024-09-10T10:28:02.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f404d7fa640 1 -- 172.21.15.120:0/3032948837 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044037040 con 0x7f40500c7770 2024-09-10T10:28:02.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.194+0000 7f4056fb7640 1 -- 172.21.15.120:0/3032948837 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f40500c2e50 con 0x7f40500c7770 2024-09-10T10:28:02.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.195+0000 7f404d7fa640 1 -- 172.21.15.120:0/3032948837 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4044044070 con 0x7f40500c7770 2024-09-10T10:28:02.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.196+0000 7f4056fb7640 1 -- 172.21.15.120:0/3032948837 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40500c5980 con 0x7f40500c7770 2024-09-10T10:28:02.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.196+0000 7f404d7fa640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f403406be30 0x7f403406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.196+0000 7f404d7fa640 1 -- 172.21.15.120:0/3032948837 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(41..41 src has 1..41) ==== 6764+0+0 (secure 0 0 0) 0x7f40440abae0 con 0x7f40500c7770 2024-09-10T10:28:02.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.196+0000 7f404ffff640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f403406be30 0x7f403406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.197+0000 7f404ffff640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f403406be30 0x7f403406e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40500c2a50 tx=0x7f403c007ee0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.199+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f762815aee0 con 0x7f76281f3ab0 2024-09-10T10:28:02.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.201+0000 7f404d7fa640 1 -- 172.21.15.120:0/3032948837 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4044079970 con 0x7f40500c7770 2024-09-10T10:28:02.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.201+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f762815aee0 con 0x7f76281f3ab0 2024-09-10T10:28:02.202 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:02.203 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:02.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.201+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f7628159360 0x7f76280c2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.201+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f7628067f80 con 0x7f7628159360 2024-09-10T10:28:02.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.202+0000 7f762659c640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f7628159360 0x7f76280c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.202+0000 7f762659c640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f7628159360 0x7f76280c2920 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.203+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7628067f80 con 0x7f7628159360 2024-09-10T10:28:02.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.217+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f762815bbd0 con 0x7f7628159360 2024-09-10T10:28:02.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.219+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f762815bbd0 con 0x7f7628159360 2024-09-10T10:28:02.220 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:02.221 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:02.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.219+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f7628062b40 0x7f7628062f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.219+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f76280634a0 con 0x7f7628062b40 2024-09-10T10:28:02.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.219+0000 7f7625d9b640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f7628062b40 0x7f7628062f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.220+0000 7f7625d9b640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f7628062b40 0x7f7628062f60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.220+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76280634a0 con 0x7f7628062b40 2024-09-10T10:28:02.234 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:02.233+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:02.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.234+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f762802c890 con 0x7f7628062b40 2024-09-10T10:28:02.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.235+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f762802c890 con 0x7f7628062b40 2024-09-10T10:28:02.237 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:02.238 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:02.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.236+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f762802d370 0x7f762802d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.236+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f76280cc5b0 con 0x7f762802d370 2024-09-10T10:28:02.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.236+0000 7f7626d9d640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f762802d370 0x7f762802d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.236+0000 7f7626d9d640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f762802d370 0x7f762802d7b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.237+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76280cc5b0 con 0x7f762802d370 2024-09-10T10:28:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.251+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76280caea0 con 0x7f762802d370 2024-09-10T10:28:02.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.253+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76280caea0 con 0x7f762802d370 2024-09-10T10:28:02.255 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:02.255 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:02.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.254+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f76280cb5c0 0x7f76280cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.254+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f76280cbf20 con 0x7f76280cb5c0 2024-09-10T10:28:02.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.254+0000 7f762659c640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f76280cb5c0 0x7f76280cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.254+0000 7f762659c640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f76280cb5c0 0x7f76280cb9e0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.255+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76280cbf20 con 0x7f76280cb5c0 2024-09-10T10:28:02.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.282+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76280cbf20 con 0x7f76280cb5c0 2024-09-10T10:28:02.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.284+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76280cbf20 con 0x7f76280cb5c0 2024-09-10T10:28:02.285 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.284+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f76280cc130 0x7f7628206e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.284+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f7628209c60 con 0x7f76280cc130 2024-09-10T10:28:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.285+0000 7f7625d9b640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f76280cc130 0x7f7628206e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.285+0000 7f7625d9b640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f76280cc130 0x7f7628206e90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.286+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7628209c60 con 0x7f76280cc130 2024-09-10T10:28:02.300 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:02.301 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:02.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:02.301 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:02.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.301 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:1 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-09-10T10:28:02.302 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:02.302 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.030347375999995, 2024-09-10T10:28:02.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028809700000000001, 2024-09-10T10:28:02.302 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.302 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.303 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.303 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:02.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:02.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9999999999999999e-07 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268119+0000", 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1810000000000003e-06 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268129+0000", 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9790000000000005e-06 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268137+0000", 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4800000000000001e-06 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268198+0000", 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0918000000000002e-05 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268226+0000", 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7267000000000002e-05 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270943+0000", 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027177529999999998 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270996+0000", 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2191999999999998e-05 2024-09-10T10:28:02.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:119 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'81, mlcod=16'78)", 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 89.804496423000003, 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029329880000000001, 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493966+0000", 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1699999999999998e-07 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494040+0000", 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3883999999999999e-05 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494048+0000", 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4880000000000007e-06 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494057+0000", 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1310000000000006e-06 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494095+0000", 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8470000000000003e-05 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494108+0000", 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2588999999999999e-05 2024-09-10T10:28:02.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496869+0000", 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027610400000000002 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496898+0000", 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9869000000000002e-05 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 89.804112786999994, 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005336365, 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:02.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494350+0000", 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494416+0000", 2024-09-10T10:28:02.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6413000000000003e-05 2024-09-10T10:28:02.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494424+0000", 2024-09-10T10:28:02.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.576e-06 2024-09-10T10:28:02.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494430+0000", 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6579999999999996e-06 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494466+0000", 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5648999999999997e-05 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494478+0000", 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.234e-05 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499654+0000", 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051755209999999998 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499685+0000", 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.167e-05 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.317 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 89.803717977000005, 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049664970000000003, 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:02.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494819+0000", 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4690000000000005e-05 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494825+0000", 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7080000000000002e-06 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494828+0000", 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1109999999999999e-06 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494869+0000", 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1146000000000002e-05 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494881+0000", 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2295e-05 2024-09-10T10:28:02.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499687+0000", 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048061459999999999 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499710+0000", 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-05 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.518361773999999, 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054008370000000003, 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780104+0000", 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3039999999999998e-06 2024-09-10T10:28:02.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:28:02.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0738e-05 2024-09-10T10:28:02.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780141+0000", 2024-09-10T10:28:02.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6317999999999999e-05 2024-09-10T10:28:02.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2199999999999992e-06 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780405+0000", 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025557599999999998 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780498+0000", 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2860000000000002e-05 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780591+0000", 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3094e-05 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024814469999999999 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785441+0000", 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023678280000000002 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785480+0000", 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9511999999999997e-05 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785501+0000", 2024-09-10T10:28:02.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0939999999999999e-05 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.518286939999996, 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054780480000000001, 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.329 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780176+0000", 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6700000000000003e-07 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1289999999999998e-06 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780188+0000", 2024-09-10T10:28:02.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.96e-06 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780194+0000", 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9989999999999997e-06 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780882+0000", 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068832400000000003 2024-09-10T10:28:02.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780968+0000", 2024-09-10T10:28:02.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5959999999999997e-05 2024-09-10T10:28:02.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:02.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781098+0000", 2024-09-10T10:28:02.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012994100000000001 2024-09-10T10:28:02.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783078+0000", 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019803540000000001 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785553+0000", 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002474515 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785608+0000", 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5319999999999999e-05 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785653+0000", 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4879000000000002e-05 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:02.335 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.518255397000004, 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042025200000000004, 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780217+0000", 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780224+0000", 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2529999999999998e-06 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780230+0000", 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6110000000000001e-06 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780282+0000", 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2148000000000003e-05 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780394+0000", 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111864 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780553+0000", 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015913799999999999 2024-09-10T10:28:02.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783054+0000", 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025008809999999999 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784349+0000", 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012947480000000001 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784381+0000", 2024-09-10T10:28:02.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2268999999999999e-05 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784409+0000", 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8007000000000001e-05 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.518213044999996, 2024-09-10T10:28:02.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054782659999999999, 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:02.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780253+0000", 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4199999999999999e-06 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780260+0000", 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5100000000000001e-06 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780267+0000", 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.443e-06 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781353+0000", 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010859769999999999 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781457+0000", 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104545 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781561+0000", 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010409200000000001 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783532+0000", 2024-09-10T10:28:02.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019702280000000001 2024-09-10T10:28:02.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785685+0000", 2024-09-10T10:28:02.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002153313 2024-09-10T10:28:02.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785710+0000", 2024-09-10T10:28:02.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5024999999999998e-05 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785727+0000", 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7175000000000001e-05 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.518186954000001, 2024-09-10T10:28:02.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005104407, 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:02.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780278+0000", 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.959e-06 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780285+0000", 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8040000000000004e-06 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780292+0000", 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4099999999999996e-06 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781345+0000", 2024-09-10T10:28:02.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010533579999999999 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781456+0000", 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110624 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781570+0000", 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011417999999999999 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783625+0000", 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020553780000000001 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785303+0000", 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001677827 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785345+0000", 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2364999999999998e-05 2024-09-10T10:28:02.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785379+0000", 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4054000000000003e-05 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.518163126000005, 2024-09-10T10:28:02.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0060500800000000002, 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:02.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:28:02.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780306+0000", 2024-09-10T10:28:02.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-10T10:28:02.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780313+0000", 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-06 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1739999999999999e-06 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782543+0000", 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022253419999999999 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010785699999999999 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.786349+0000", 2024-09-10T10:28:02.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036978829999999999 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.426461086000003, 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058824769999999997, 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872002+0000", 2024-09-10T10:28:02.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5149999999999999e-06 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872008+0000", 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.66e-06 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872022+0000", 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3932e-05 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872031+0000", 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5259999999999999e-06 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872111+0000", 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9412000000000005e-05 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872231+0000", 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119843 2024-09-10T10:28:02.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872340+0000", 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109442 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.875579+0000", 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032383899999999998 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877781+0000", 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022022299999999999 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877835+0000", 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4419000000000002e-05 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877883+0000", 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8108000000000001e-05 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.363 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.425372254999999, 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049968149999999999, 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7300000000000005e-07 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.245e-06 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3898e-05 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873116+0000", 2024-09-10T10:28:02.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8050000000000007e-06 2024-09-10T10:28:02.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873184+0000", 2024-09-10T10:28:02.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7809000000000001e-05 2024-09-10T10:28:02.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873257+0000", 2024-09-10T10:28:02.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2602999999999996e-05 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873374+0000", 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011744900000000001 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.876282+0000", 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029079459999999998 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878003+0000", 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001720886 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878054+0000", 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0596999999999997e-05 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878086+0000", 2024-09-10T10:28:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2804e-05 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.350979883000001, 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029964739999999998, 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947484+0000", 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.857e-06 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947490+0000", 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9789999999999998e-06 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947504+0000", 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3942e-05 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947512+0000", 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5620000000000004e-06 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947599+0000", 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6525000000000003e-05 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947714+0000", 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114709 2024-09-10T10:28:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947823+0000", 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109477 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950336+0000", 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025131509999999999 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950386+0000", 2024-09-10T10:28:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9974e-05 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950440+0000", 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4011000000000002e-05 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950478+0000", 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8287000000000001e-05 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.349539485000001, 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043004360000000004, 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:02.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948923+0000", 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8999999999999996e-07 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948928+0000", 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948940+0000", 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1759e-05 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948949+0000", 2024-09-10T10:28:02.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5149999999999995e-06 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949028+0000", 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8182999999999993e-05 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949137+0000", 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109065 2024-09-10T10:28:02.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949245+0000", 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108047 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950193+0000", 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094856200000000002 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953173+0000", 2024-09-10T10:28:02.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029801960000000001 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953200+0000", 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6707999999999999e-05 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953223+0000", 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2758000000000001e-05 2024-09-10T10:28:02.381 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.349483972999998, 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046333320000000004, 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948979+0000", 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-07 2024-09-10T10:28:02.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6490000000000001e-06 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948990+0000", 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2560000000000002e-06 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948996+0000", 2024-09-10T10:28:02.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9739999999999999e-06 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949496+0000", 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049934299999999997 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949588+0000", 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1762999999999995e-05 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949729+0000", 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014145299999999999 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950984+0000", 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012548419999999999 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953505+0000", 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025208129999999998 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953568+0000", 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3695000000000006e-05 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953611+0000", 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2913000000000001e-05 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.387 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.377432358, 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048620219999999997, 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4899999999999995e-07 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921034+0000", 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7050000000000001e-06 2024-09-10T10:28:02.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921043+0000", 2024-09-10T10:28:02.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1840000000000005e-06 2024-09-10T10:28:02.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921049+0000", 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4459999999999998e-06 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921575+0000", 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052602300000000003 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921646+0000", 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0475000000000005e-05 2024-09-10T10:28:02.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921769+0000", 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123347 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923077+0000", 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013078759999999999 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925794+0000", 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027172310000000001 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925853+0000", 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8739999999999997e-05 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925892+0000", 2024-09-10T10:28:02.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8445999999999997e-05 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.377395323999998, 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032737339999999999, 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9919999999999999e-06 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921080+0000", 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0063e-05 2024-09-10T10:28:02.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921086+0000", 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921142+0000", 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6174999999999997e-05 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013480499999999999 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921405+0000", 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012819200000000001 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923001+0000", 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015962249999999999 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924243+0000", 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012425820000000001 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924300+0000", 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6901999999999998e-05 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924340+0000", 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9918e-05 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.399 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(2.4 epoch 26/18 rep_tid 19 entries 26'1 (0'0) error 2:218afd09:::203.00000003:head by mds.3.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.375256878999998, 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030589089999999998, 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923206+0000", 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-09-10T10:28:02.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7249999999999997e-06 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923229+0000", 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5905000000000001e-05 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923241+0000", 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1990999999999999e-05 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923317+0000", 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5622000000000002e-05 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926264+0000", 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029470170000000001 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.403 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.354898878999997, 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050642550000000001, 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943564+0000", 2024-09-10T10:28:02.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.13e-07 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943576+0000", 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2187e-05 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943585+0000", 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9279999999999999e-06 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943592+0000", 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0110000000000003e-06 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943681+0000", 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9487999999999994e-05 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943767+0000", 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5866999999999996e-05 2024-09-10T10:28:02.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943882+0000", 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011443299999999999 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.947042+0000", 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031600629999999998 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948540+0000", 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001498451 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948581+0000", 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0839e-05 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948627+0000", 2024-09-10T10:28:02.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6375000000000002e-05 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.354830556000003, 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051386089999999997, 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943632+0000", 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1500000000000004e-07 2024-09-10T10:28:02.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943643+0000", 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0954e-05 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943652+0000", 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4929999999999998e-06 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943657+0000", 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6509999999999998e-06 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944230+0000", 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057322999999999999 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944340+0000", 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010986 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948770+0000", 2024-09-10T10:28:02.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044297060000000003 2024-09-10T10:28:02.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.414 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.414 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.414 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.414 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:02.416 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:02.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.313+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f76282073d0 con 0x7f76280cc130 2024-09-10T10:28:02.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.315+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f76282073d0 con 0x7f76280cc130 2024-09-10T10:28:02.416 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:02.417 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:02.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.315+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f7628207a50 0x7f7628209e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.315+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f762820cc10 con 0x7f7628207a50 2024-09-10T10:28:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.315+0000 7f7626d9d640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f7628207a50 0x7f7628209e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.316+0000 7f7626d9d640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f7628207a50 0x7f7628209e40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.317+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f762820cc10 con 0x7f7628207a50 2024-09-10T10:28:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.344+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f762820a380 con 0x7f7628207a50 2024-09-10T10:28:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.346+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f762820a380 con 0x7f7628207a50 2024-09-10T10:28:02.418 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:02.419 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:02.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.346+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f762820aa00 0x7f762820cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.346+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f762820fda0 con 0x7f762820aa00 2024-09-10T10:28:02.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.346+0000 7f762659c640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f762820aa00 0x7f762820cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.346+0000 7f762659c640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f762820aa00 0x7f762820cdf0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.347+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f762820fda0 con 0x7f762820aa00 2024-09-10T10:28:02.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.367+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f762820eac0 con 0x7f762820aa00 2024-09-10T10:28:02.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f761effd640 1 -- 172.21.15.120:0/2286393107 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f762820eac0 con 0x7f762820aa00 2024-09-10T10:28:02.420 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:02.421 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f76281f3ab0 msgr2=0x7f76280c1ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f76281f3ab0 0x7f76280c1ed0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f7628159360 msgr2=0x7f76280c2920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f7628159360 0x7f76280c2920 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f7628062b40 msgr2=0x7f7628062f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f7628062b40 0x7f7628062f60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f762802d370 msgr2=0x7f762802d7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f762802d370 0x7f762802d7b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f76280cb5c0 msgr2=0x7f76280cb9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f7626d9d640 1 -- 172.21.15.120:0/2286393107 reap_dead start 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f76280cb5c0 0x7f76280cb9e0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f7626d9d640 1 -- 172.21.15.120:0/2286393107 reap_dead start 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f76280cc130 msgr2=0x7f7628206e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f76280cc130 0x7f7628206e90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f7628207a50 msgr2=0x7f7628209e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f7628207a50 0x7f7628209e40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f762820aa00 msgr2=0x7f762820cdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f762820aa00 0x7f762820cdf0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f760c06be30 msgr2=0x7f760c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f760c06be30 0x7f760c06e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7620001620 tx=0x7f7620009040 comp rx=0 tx=0).stop 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76280731e0 msgr2=0x7f76281f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 --2- 172.21.15.120:0/2286393107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76280731e0 0x7f76281f4070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7610031000 tx=0x7f76100046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f7626d9d640 1 -- 172.21.15.120:0/2286393107 reap_dead start 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 shutdown_connections 2024-09-10T10:28:02.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.369+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 >> 172.21.15.120:0/2286393107 conn(0x7f7628067950 msgr2=0x7f76281565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:02.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.370+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 shutdown_connections 2024-09-10T10:28:02.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.370+0000 7f762c986640 1 -- 172.21.15.120:0/2286393107 wait complete. 2024-09-10T10:28:02.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.419+0000 7f4056fb7640 1 -- 172.21.15.120:0/3032948837 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f40500d14c0 con 0x7f40500c7770 2024-09-10T10:28:02.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.420+0000 7f404d7fa640 1 -- 172.21.15.120:0/3032948837 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v41) ==== 81+0+38916 (secure 0 0 0) 0x7f404407dc70 con 0x7f40500c7770 2024-09-10T10:28:02.425 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:02.414+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:02.425 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 41, 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-10T10:26:18.018818+0000", 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-10T10:28:01.074548+0000", 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-10T10:26:29.078139+0000", 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-10T10:26:20.091201+0000", 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5801984, 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "noscrub", 2024-09-10T10:28:02.426 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 4, 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-10T10:28:02.427 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "luminous", 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:30.118649+0000", 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:28:02.428 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:02.429 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:02.430 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:02.431 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:02.432 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-10T10:28:02.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.434 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_metadata", 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:38.936606+0000", 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:02.435 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:02.436 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "20", 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:02.437 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:02.438 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:02.438 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:02.438 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:02.438 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:02.438 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:02.438 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:02.439 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:02.439 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:02.439 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:02.439 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:02.439 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:02.439 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:02.439 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:02.440 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:28:02.440 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:02.440 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:02.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.440 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:02.440 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:02.440 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:02.440 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:02.440 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_bias": 4, 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 16, 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_priority": 5 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:02.441 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:28:02.442 INFO:teuthology.orchestra.run.smithi120.stdout: "metadata": "cephfs" 2024-09-10T10:28:02.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.442 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:28:02.442 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:28:02.442 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-10T10:28:02.442 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-10T10:28:02.442 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:28:02.443 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-10T10:28:02.443 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-10T10:28:02.443 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:28:02.443 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:28:02.443 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:28:02.443 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.443 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data", 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:40.094780+0000", 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 64, 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 64, 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 64, 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 64, 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 64, 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:02.445 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:02.446 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:02.447 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:28:02.448 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.05000000074505806 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:28:02.449 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 4, 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data_ec", 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:43.814005+0000", 2024-09-10T10:28:02.450 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 5, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,ec_overwrites", 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 3, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 4, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 3, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 1, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:02.451 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "26", 2024-09-10T10:28:02.452 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:02.453 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "cephfs_data_ec", 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.454 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 8192, 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:02.455 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.456 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e3b9c3c3-cf7d-4b33-913e-60e928439d8e", 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 23, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:02.457 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.460 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:02.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/3664783490", 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/3664783490", 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/3664783490", 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/3664783490", 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:28:02.462 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "62c20a9d-e6ab-446d-998c-9ee46e46aa90", 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:02.464 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.465 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:02.466 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.466 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-10T10:28:02.466 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:02.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:02.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/3972761964", 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/3972761964", 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/3972761964", 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/3972761964", 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:02.468 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1528f8e0-5e13-4ec7-9d98-0d94b7a505c3", 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:02.469 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.470 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.471 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:02.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2916410859", 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2916410859", 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2916410859", 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2916410859", 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:02.474 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "74942f78-c1fc-4886-abf0-950722c5edc6", 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:02.475 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 39, 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.476 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-10T10:28:02.477 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:02.477 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.477 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.477 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:02.477 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.477 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.478 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.479 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:02.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1224278950", 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1224278950", 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1224278950", 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1224278950", 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e71319d4-5318-4441-84a8-7665ae037596", 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:02.482 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6816", 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6817", 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6818", 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6819", 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.484 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6822", 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6823", 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:02.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6820", 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6821", 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:02.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6817/1874635282", 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6819/1874635282", 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6823/1874635282", 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6821/1874635282", 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.488 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:28:02.488 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "29e193cd-37af-43e5-bd0c-2f85ce0631e8", 2024-09-10T10:28:02.488 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:02.488 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:02.488 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:02.488 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:02.488 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6808", 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.490 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.490 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6809", 2024-09-10T10:28:02.490 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:02.490 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.490 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.490 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:02.490 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6810", 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6811", 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.491 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6814", 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6815", 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.492 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6812", 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6813", 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:02.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6809/457724087", 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6811/457724087", 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6815/457724087", 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6813/457724087", 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.494 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "55523ec2-cf12-44b3-aeea-f1cedabc996a", 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:02.495 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6824", 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6825", 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.496 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6826", 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6827", 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:02.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.498 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.498 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:02.498 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.498 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6830", 2024-09-10T10:28:02.498 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:02.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.499 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.499 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6831", 2024-09-10T10:28:02.499 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:02.499 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.499 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.499 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:02.499 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.499 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6828", 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6829", 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6825/50133702", 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6827/50133702", 2024-09-10T10:28:02.500 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6831/50133702", 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6829/50133702", 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "097b1149-3e3c-4e5a-890b-d4cc4361d51b", 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:02.501 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 35, 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6800", 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6801", 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6802", 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:02.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.504 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.504 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6803", 2024-09-10T10:28:02.504 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:02.504 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.504 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.504 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:02.504 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6806", 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6807", 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6804", 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6805", 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6801/251960875", 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6803/251960875", 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6807/251960875", 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6805/251960875", 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-10T10:28:02.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.143056+0000", 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:28:02.508 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:02.509 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:02.509 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:02.509 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:02.509 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:02.509 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.423968+0000", 2024-09-10T10:28:02.509 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:02.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.509 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:28:02.510 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:02.510 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:02.510 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:02.510 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:02.510 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:02.510 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.640076+0000", 2024-09-10T10:28:02.510 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.536160+0000", 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:02.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.146033+0000", 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:28:02.512 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.426011+0000", 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:02.513 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.598992+0000", 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.182929+0000", 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.514 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [ 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.34", 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: 3, 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: 1 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [ 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "2.1d", 2024-09-10T10:28:02.515 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 6, 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 4 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.16", 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 1 2024-09-10T10:28:02.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.1d", 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.28", 2024-09-10T10:28:02.517 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.36", 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:02.518 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.d", 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 5 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.1b", 2024-09-10T10:28:02.519 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:02.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.520 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:28:02.520 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:02.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.520 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.520 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.520 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:02.520 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-10T10:28:02.520 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-10T10:28:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-10T10:28:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-10T10:28:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-10T10:28:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-10T10:28:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_data_ec": { 2024-09-10T10:28:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-device-class": "", 2024-09-10T10:28:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:28:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-num-failure-domains": "0", 2024-09-10T10:28:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-osds-per-failure-domain": "0", 2024-09-10T10:28:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-root": "default", 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "jerasure-per-chunk-alignment": "false", 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "2", 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van", 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "w": "8" 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-10T10:28:02.522 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [], 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [], 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-10T10:28:02.523 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.524 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:02.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.422+0000 7f4056fb7640 1 -- 172.21.15.120:0/3032948837 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f403406be30 msgr2=0x7f403406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.422+0000 7f4056fb7640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f403406be30 0x7f403406e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40500c2a50 tx=0x7f403c007ee0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.422+0000 7f4056fb7640 1 -- 172.21.15.120:0/3032948837 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500c7770 msgr2=0x7f40500c2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.422+0000 7f4056fb7640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500c7770 0x7f40500c2170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f40500cada0 tx=0x7f40440046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.423+0000 7f4056fb7640 1 -- 172.21.15.120:0/3032948837 shutdown_connections 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.423+0000 7f4056fb7640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f403406be30 0x7f403406e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.423+0000 7f4056fb7640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40500d0950 0x7f405014f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.423+0000 7f4056fb7640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40500d0570 0x7f405015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.423+0000 7f4056fb7640 1 --2- 172.21.15.120:0/3032948837 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40500c7770 0x7f40500c2170 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.423+0000 7f4056fb7640 1 -- 172.21.15.120:0/3032948837 >> 172.21.15.120:0/3032948837 conn(0x7f405006ca90 msgr2=0x7f40500cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.423+0000 7f4056fb7640 1 -- 172.21.15.120:0/3032948837 shutdown_connections 2024-09-10T10:28:02.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.423+0000 7f4056fb7640 1 -- 172.21.15.120:0/3032948837 wait complete. 2024-09-10T10:28:02.527 INFO:tasks.thrashosds.thrasher:j is {'epoch': 41, 'fsid': '7d0d0830-4370-41c3-9acc-b67b07024f68', 'created': '2024-09-10T10:26:18.018818+0000', 'modified': '2024-09-10T10:28:01.074548+0000', 'last_up_change': '2024-09-10T10:26:29.078139+0000', 'last_in_change': '2024-09-10T10:26:20.091201+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:26:30.118649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:26:38.936606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:26:40.094780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T10:26:43.814005+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'e3b9c3c3-cf7d-4b33-913e-60e928439d8e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 3664783490}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 3664783490}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 3664783490}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 3664783490}]}, 'public_addr': '172.21.15.120:6809/3664783490', 'cluster_addr': '172.21.15.120:6811/3664783490', 'heartbeat_back_addr': '172.21.15.120:6815/3664783490', 'heartbeat_front_addr': '172.21.15.120:6813/3664783490', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '62c20a9d-e6ab-446d-998c-9ee46e46aa90', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 3972761964}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 3972761964}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 3972761964}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 3972761964}]}, 'public_addr': '172.21.15.120:6801/3972761964', 'cluster_addr': '172.21.15.120:6803/3972761964', 'heartbeat_back_addr': '172.21.15.120:6807/3972761964', 'heartbeat_front_addr': '172.21.15.120:6805/3972761964', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1528f8e0-5e13-4ec7-9d98-0d94b7a505c3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2916410859}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2916410859}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2916410859}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2916410859}]}, 'public_addr': '172.21.15.120:6817/2916410859', 'cluster_addr': '172.21.15.120:6819/2916410859', 'heartbeat_back_addr': '172.21.15.120:6823/2916410859', 'heartbeat_front_addr': '172.21.15.120:6821/2916410859', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '74942f78-c1fc-4886-abf0-950722c5edc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1224278950}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1224278950}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1224278950}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1224278950}]}, 'public_addr': '172.21.15.120:6825/1224278950', 'cluster_addr': '172.21.15.120:6827/1224278950', 'heartbeat_back_addr': '172.21.15.120:6831/1224278950', 'heartbeat_front_addr': '172.21.15.120:6829/1224278950', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e71319d4-5318-4441-84a8-7665ae037596', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6816', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6817', 'nonce': 1874635282}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6818', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6819', 'nonce': 1874635282}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6822', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6823', 'nonce': 1874635282}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6820', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6821', 'nonce': 1874635282}]}, 'public_addr': '172.21.15.134:6817/1874635282', 'cluster_addr': '172.21.15.134:6819/1874635282', 'heartbeat_back_addr': '172.21.15.134:6823/1874635282', 'heartbeat_front_addr': '172.21.15.134:6821/1874635282', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '29e193cd-37af-43e5-bd0c-2f85ce0631e8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6808', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6809', 'nonce': 457724087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6810', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6811', 'nonce': 457724087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6814', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6815', 'nonce': 457724087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6812', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6813', 'nonce': 457724087}]}, 'public_addr': '172.21.15.134:6809/457724087', 'cluster_addr': '172.21.15.134:6811/457724087', 'heartbeat_back_addr': '172.21.15.134:6815/457724087', 'heartbeat_front_addr': '172.21.15.134:6813/457724087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '55523ec2-cf12-44b3-aeea-f1cedabc996a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6824', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6825', 'nonce': 50133702}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6826', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6827', 'nonce': 50133702}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6830', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6831', 'nonce': 50133702}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6828', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6829', 'nonce': 50133702}]}, 'public_addr': '172.21.15.134:6825/50133702', 'cluster_addr': '172.21.15.134:6827/50133702', 'heartbeat_back_addr': '172.21.15.134:6831/50133702', 'heartbeat_front_addr': '172.21.15.134:6829/50133702', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '097b1149-3e3c-4e5a-890b-d4cc4361d51b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6800', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6801', 'nonce': 251960875}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6802', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6803', 'nonce': 251960875}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6806', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6807', 'nonce': 251960875}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6804', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6805', 'nonce': 251960875}]}, 'public_addr': '172.21.15.134:6801/251960875', 'cluster_addr': '172.21.15.134:6803/251960875', 'heartbeat_back_addr': '172.21.15.134:6807/251960875', 'heartbeat_front_addr': '172.21.15.134:6805/251960875', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.143056+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.423968+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.640076+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.536160+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.146033+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.426011+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.598992+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.182929+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.34', 'osds': [3, 1]}], 'pg_upmap_items': [{'pgid': '2.1d', 'mappings': [{'from': 6, 'to': 4}]}, {'pgid': '3.16', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.1d', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.28', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.36', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'cephfs_data_ec': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '2', 'plugin': 'jerasure', 'technique': 'reed_sol_van', 'w': '8'}, 'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-10T10:28:02.528 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:28:02.529 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:02.528+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:02.531 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:02.531 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:02.531 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:02.532 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:02.541 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:02.630 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:02.629+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:02.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.634+0000 7fe016f31640 1 Processor -- start 2024-09-10T10:28:02.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.635+0000 7fe016f31640 1 -- start start 2024-09-10T10:28:02.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.635+0000 7fe016f31640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 0x7fe0100720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.635+0000 7fe016f31640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01006d170 con 0x7fe0100c33f0 2024-09-10T10:28:02.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.635+0000 7fe016f31640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01006d450 con 0x7fe0100c37d0 2024-09-10T10:28:02.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.635+0000 7fe016f31640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe0100c3e00 con 0x7fe0100c9b10 2024-09-10T10:28:02.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.636+0000 7fe014ccd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fe0100c33f0 0x7fe010073870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:53102/0 (socket says 172.21.15.120:53102) 2024-09-10T10:28:02.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.636+0000 7fe014ccd640 1 -- 172.21.15.120:0/1452843847 learned_addr learned my addr 172.21.15.120:0/1452843847 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:02.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.636+0000 7fe0154ce640 1 --2- 172.21.15.120:0/1452843847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 0x7fe0100720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.636+0000 7fe00f7fe640 1 -- 172.21.15.120:0/1452843847 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3241915582 0 0) 0x7fe01006d170 con 0x7fe0100c33f0 2024-09-10T10:28:02.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.636+0000 7fe00f7fe640 1 -- 172.21.15.120:0/1452843847 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe000003520 con 0x7fe0100c33f0 2024-09-10T10:28:02.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.636+0000 7fe00f7fe640 1 -- 172.21.15.120:0/1452843847 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2181720480 0 0) 0x7fe01006d450 con 0x7fe0100c37d0 2024-09-10T10:28:02.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.636+0000 7fe00f7fe640 1 -- 172.21.15.120:0/1452843847 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe01006d170 con 0x7fe0100c37d0 2024-09-10T10:28:02.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.636+0000 7fe0154ce640 1 -- 172.21.15.120:0/1452843847 >> v1:172.21.15.134:6789/0 conn(0x7fe0100c37d0 legacy=0x7fe0101516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.636+0000 7fe0154ce640 1 -- 172.21.15.120:0/1452843847 >> v1:172.21.15.120:6789/0 conn(0x7fe0100c33f0 legacy=0x7fe010073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.636+0000 7fe0154ce640 1 -- 172.21.15.120:0/1452843847 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0100c4050 con 0x7fe0100c9b10 2024-09-10T10:28:02.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.637+0000 7fe0154ce640 1 --2- 172.21.15.120:0/1452843847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 0x7fe0100720c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe008009dc0 tx=0x7fe00802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0d229fce84000d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.637+0000 7fe00f7fe640 1 -- 172.21.15.120:0/1452843847 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00802ae60 con 0x7fe0100c9b10 2024-09-10T10:28:02.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.637+0000 7fe00f7fe640 1 -- 172.21.15.120:0/1452843847 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe008007740 con 0x7fe0100c9b10 2024-09-10T10:28:02.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.637+0000 7fe00f7fe640 1 -- 172.21.15.120:0/1452843847 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008007b10 con 0x7fe0100c9b10 2024-09-10T10:28:02.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.638+0000 7fe016f31640 1 -- 172.21.15.120:0/1452843847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 msgr2=0x7fe0100720c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.638+0000 7fe016f31640 1 --2- 172.21.15.120:0/1452843847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 0x7fe0100720c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe008009dc0 tx=0x7fe00802a640 comp rx=0 tx=0).stop 2024-09-10T10:28:02.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.638+0000 7fe016f31640 1 -- 172.21.15.120:0/1452843847 shutdown_connections 2024-09-10T10:28:02.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.638+0000 7fe016f31640 1 --2- 172.21.15.120:0/1452843847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 0x7fe0100720c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.638+0000 7fe016f31640 1 -- 172.21.15.120:0/1452843847 >> 172.21.15.120:0/1452843847 conn(0x7fe010070290 msgr2=0x7fe0100c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:02.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.638+0000 7fe016f31640 1 -- 172.21.15.120:0/1452843847 shutdown_connections 2024-09-10T10:28:02.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.638+0000 7fe016f31640 1 -- 172.21.15.120:0/1452843847 wait complete. 2024-09-10T10:28:02.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.639+0000 7fe016f31640 1 Processor -- start 2024-09-10T10:28:02.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.639+0000 7fe016f31640 1 -- start start 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe016f31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0100c33f0 0x7fe0101e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe016f31640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0100c37d0 0x7fe0100cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe016f31640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 0x7fe0100cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe016f31640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0100c5aa0 con 0x7fe0100c33f0 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe016f31640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe010073f80 con 0x7fe0100c37d0 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe016f31640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe010151db0 con 0x7fe0100c9b10 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe014ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0100c33f0 0x7fe0101e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe014ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0100c33f0 0x7fe0101e2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34998/0 (socket says 172.21.15.120:34998) 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe014ccd640 1 -- 172.21.15.120:0/3818850963 learned_addr learned my addr 172.21.15.120:0/3818850963 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe00ffff640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0100c37d0 0x7fe0100cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.640+0000 7fe0154ce640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 0x7fe0100cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe00ffff640 1 -- 172.21.15.120:0/3818850963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 msgr2=0x7fe0100cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe00ffff640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 0x7fe0100cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe00ffff640 1 -- 172.21.15.120:0/3818850963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0100c33f0 msgr2=0x7fe0101e2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe00ffff640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0100c33f0 0x7fe0101e2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe00ffff640 1 -- 172.21.15.120:0/3818850963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0100c2540 con 0x7fe0100c37d0 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe0154ce640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 0x7fe0100cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe014ccd640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0100c33f0 0x7fe0101e2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe00ffff640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0100c37d0 0x7fe0100cad00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00c400 tx=0x7fdffc00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe00d7fa640 1 -- 172.21.15.120:0/3818850963 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc017070 con 0x7fe0100c37d0 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe00d7fa640 1 -- 172.21.15.120:0/3818850963 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdffc0052d0 con 0x7fe0100c37d0 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe00d7fa640 1 -- 172.21.15.120:0/3818850963 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc003830 con 0x7fe0100c37d0 2024-09-10T10:28:02.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe016f31640 1 -- 172.21.15.120:0/3818850963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0100cc140 con 0x7fe0100c37d0 2024-09-10T10:28:02.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.641+0000 7fe016f31640 1 -- 172.21.15.120:0/3818850963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0100c2800 con 0x7fe0100c37d0 2024-09-10T10:28:02.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.642+0000 7fe016f31640 1 -- 172.21.15.120:0/3818850963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0100ca0f0 con 0x7fe0100c37d0 2024-09-10T10:28:02.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.643+0000 7fe00d7fa640 1 -- 172.21.15.120:0/3818850963 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdffc01a070 con 0x7fe0100c37d0 2024-09-10T10:28:02.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.643+0000 7fe00d7fa640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdff406be30 0x7fdff406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:02.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.644+0000 7fe00d7fa640 1 -- 172.21.15.120:0/3818850963 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6764+0+0 (secure 0 0 0) 0x7fdffc08a580 con 0x7fe0100c37d0 2024-09-10T10:28:02.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.644+0000 7fe014ccd640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdff406be30 0x7fdff406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:02.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.644+0000 7fe014ccd640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdff406be30 0x7fdff406e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe004000f60 tx=0x7fe004009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:02.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.648+0000 7fe00d7fa640 1 -- 172.21.15.120:0/3818850963 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdffc0584c0 con 0x7fe0100c37d0 2024-09-10T10:28:02.695 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:02.695 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:02.695 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:02.695 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:02.695 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:02.704 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:02.748 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:02.747+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:02.813 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:02.813 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:02.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:02.813 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:02.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.813 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:02.813 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:02.813 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.030652726, 2024-09-10T10:28:02.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049304329999999997, 2024-09-10T10:28:02.813 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780094+0000", 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780107+0000", 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3703000000000001e-05 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780116+0000", 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5210000000000002e-06 2024-09-10T10:28:02.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:28:02.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6340000000000007e-06 2024-09-10T10:28:02.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:02.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001735838 2024-09-10T10:28:02.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782000+0000", 2024-09-10T10:28:02.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013984100000000001 2024-09-10T10:28:02.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:02.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782131+0000", 2024-09-10T10:28:02.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000131025 2024-09-10T10:28:02.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783737+0000", 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016054179999999999 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784937+0000", 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001200638 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784987+0000", 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9591e-05 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785024+0000", 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6751000000000001e-05 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.030613271000007, 2024-09-10T10:28:02.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052929559999999997, 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:02.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780136+0000", 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9950000000000001e-06 2024-09-10T10:28:02.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780144+0000", 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.174e-06 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4949999999999999e-06 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781163+0000", 2024-09-10T10:28:02.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001013438 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781279+0000", 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011579200000000001 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781388+0000", 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108834 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783243+0000", 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00185491 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785355+0000", 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021117990000000001 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785386+0000", 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1612000000000002e-05 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785426+0000", 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9474999999999997e-05 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:8 2.10 e24/18 2:09401a81:::601.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.028422040999999, 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005352224, 2024-09-10T10:28:02.826 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:02.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4400000000000001e-07 2024-09-10T10:28:02.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782467+0000", 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142452 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782598+0000", 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013078300000000001 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782604+0000", 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.533e-06 2024-09-10T10:28:02.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7073999999999997e-05 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782668+0000", 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6861999999999999e-05 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787632+0000", 2024-09-10T10:28:02.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049636510000000003 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787676+0000", 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4425000000000002e-05 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:9 2.10 e24/18 2:09401a81:::601.00000000:head v 24'4, mlcod=0'0)", 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.027785026000004, 2024-09-10T10:28:02.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047438960000000001, 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:02.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782962+0000", 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4499999999999997e-07 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782967+0000", 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2000000000000002e-06 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782980+0000", 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3685e-05 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782987+0000", 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4690000000000002e-06 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783112+0000", 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012481300000000001 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783130+0000", 2024-09-10T10:28:02.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7753000000000001e-05 2024-09-10T10:28:02.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787678+0000", 2024-09-10T10:28:02.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045487419999999997 2024-09-10T10:28:02.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787705+0000", 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6588999999999999e-05 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:10 2.10 e24/18 2:0bd6d154:::602.00000000:head v 24'5, mlcod=0'0)", 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.027678092000002, 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00466629, 2024-09-10T10:28:02.836 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1300000000000001e-07 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6970000000000001e-06 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783309+0000", 2024-09-10T10:28:02.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000236209 2024-09-10T10:28:02.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783319+0000", 2024-09-10T10:28:02.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7180000000000008e-06 2024-09-10T10:28:02.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783365+0000", 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6179e-05 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783383+0000", 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8382000000000001e-05 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787708+0000", 2024-09-10T10:28:02.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043246059999999999 2024-09-10T10:28:02.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787734+0000", 2024-09-10T10:28:02.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6086000000000001e-05 2024-09-10T10:28:02.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.841 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.938275898000001, 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054527359999999997, 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872472+0000", 2024-09-10T10:28:02.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5409999999999999e-06 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872479+0000", 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9550000000000003e-06 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872494+0000", 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5452999999999999e-05 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872503+0000", 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1509999999999993e-06 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872571+0000", 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7747e-05 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872704+0000", 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013269900000000001 2024-09-10T10:28:02.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872804+0000", 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9934999999999998e-05 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877617+0000", 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048138069999999998 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877819+0000", 2024-09-10T10:28:02.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020164799999999999 2024-09-10T10:28:02.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877880+0000", 2024-09-10T10:28:02.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1296000000000004e-05 2024-09-10T10:28:02.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877923+0000", 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2503999999999998e-05 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.848 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:02.849 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.937254330999998, 2024-09-10T10:28:02.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0098302110000000002, 2024-09-10T10:28:02.849 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.849 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.849 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.849 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:02.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:02.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873493+0000", 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873498+0000", 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1619999999999999e-06 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873511+0000", 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3319000000000001e-05 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873520+0000", 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3409999999999992e-06 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873586+0000", 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6115e-05 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873660+0000", 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3103999999999997e-05 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873775+0000", 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115617 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878285+0000", 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045096049999999999 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883229+0000", 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049443999999999998 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883283+0000", 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4262999999999999e-05 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883322+0000", 2024-09-10T10:28:02.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8417000000000002e-05 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.936765190999999, 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047062800000000002, 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873982+0000", 2024-09-10T10:28:02.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2900000000000003e-07 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873986+0000", 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3939999999999998e-06 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873995+0000", 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0750000000000004e-06 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874002+0000", 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874061+0000", 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8770999999999997e-05 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874178+0000", 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117108 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874290+0000", 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112503 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878269+0000", 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039784870000000002 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878593+0000", 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032385299999999998 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878644+0000", 2024-09-10T10:28:02.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1790999999999999e-05 2024-09-10T10:28:02.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878687+0000", 2024-09-10T10:28:02.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2738999999999999e-05 2024-09-10T10:28:02.860 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.936164912000002, 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089185449999999999, 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.861 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874582+0000", 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1699999999999997e-07 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874592+0000", 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0604999999999999e-05 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874602+0000", 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874610+0000", 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1919999999999999e-06 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874696+0000", 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6033999999999997e-05 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874809+0000", 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113373 2024-09-10T10:28:02.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874923+0000", 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114077 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878289+0000", 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033662380000000001 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883420+0000", 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051305760000000004 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883465+0000", 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5252e-05 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883500+0000", 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4415000000000002e-05 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.866 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.862301590999998, 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073932399999999997, 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:02.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948446+0000", 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3400000000000001e-06 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948451+0000", 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6219999999999997e-06 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948464+0000", 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2955e-05 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948472+0000", 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0730000000000005e-06 2024-09-10T10:28:02.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948534+0000", 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1727e-05 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948649+0000", 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114991 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:02.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948757+0000", 2024-09-10T10:28:02.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107935 2024-09-10T10:28:02.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950179+0000", 2024-09-10T10:28:02.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014219440000000001 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055817970000000003 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955794+0000", 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3012000000000002e-05 2024-09-10T10:28:02.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955838+0000", 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3844000000000002e-05 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 48.861762771999999, 2024-09-10T10:28:02.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069680200000000001, 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948984+0000", 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6100000000000001e-07 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948993+0000", 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9809999999999992e-06 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949002+0000", 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8789999999999992e-06 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949008+0000", 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2759999999999997e-06 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949057+0000", 2024-09-10T10:28:02.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9159999999999997e-05 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949128+0000", 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1292999999999997e-05 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949261+0000", 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132564 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950220+0000", 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00095953300000000004 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955890+0000", 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056692790000000002 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955928+0000", 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8278000000000002e-05 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955951+0000", 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3215999999999999e-05 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.889786262000001, 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050489669999999997, 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5799999999999999e-07 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920964+0000", 2024-09-10T10:28:02.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6720000000000002e-06 2024-09-10T10:28:02.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920971+0000", 2024-09-10T10:28:02.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5140000000000004e-06 2024-09-10T10:28:02.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920977+0000", 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0100000000000001e-06 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921033+0000", 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5899000000000001e-05 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921172+0000", 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139011 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921288+0000", 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011605200000000001 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922851+0000", 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001562499 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925915+0000", 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003063727 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925969+0000", 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4083999999999999e-05 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926009+0000", 2024-09-10T10:28:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9941000000000003e-05 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.889179405, 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045836510000000002, 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4499999999999996e-07 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921572+0000", 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3239999999999996e-06 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921581+0000", 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9190000000000005e-06 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921589+0000", 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0649999999999999e-06 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921635+0000", 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6863999999999997e-05 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921705+0000", 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9046000000000003e-05 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921818+0000", 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011313499999999999 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923033+0000", 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012153590000000001 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926055+0000", 2024-09-10T10:28:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030223310000000001 2024-09-10T10:28:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926104+0000", 2024-09-10T10:28:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8421999999999998e-05 2024-09-10T10:28:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926150+0000", 2024-09-10T10:28:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6440999999999998e-05 2024-09-10T10:28:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.888740460999998, 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051621749999999997, 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3600000000000003e-07 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922011+0000", 2024-09-10T10:28:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4449999999999998e-06 2024-09-10T10:28:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922019+0000", 2024-09-10T10:28:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0930000000000003e-06 2024-09-10T10:28:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922025+0000", 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5060000000000001e-06 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922076+0000", 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0671000000000003e-05 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922175+0000", 2024-09-10T10:28:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9611000000000006e-05 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922278+0000", 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102565 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925217+0000", 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029384670000000002 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927074+0000", 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018579499999999999 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927129+0000", 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4428000000000001e-05 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927168+0000", 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8803000000000001e-05 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.898 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.888704791000002, 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059957930000000001, 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922042+0000", 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922045+0000", 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4209999999999999e-06 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922052+0000", 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1430000000000001e-06 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922059+0000", 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2139999999999999e-06 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922495+0000", 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043517499999999999 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922563+0000", 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8084999999999996e-05 2024-09-10T10:28:02.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922679+0000", 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011609599999999999 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927208+0000", 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045292170000000003 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927952+0000", 2024-09-10T10:28:02.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00074421700000000003 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928001+0000", 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9004e-05 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928037+0000", 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.578e-05 2024-09-10T10:28:02.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.888678032000001, 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045268130000000002, 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.905 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:02.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922078+0000", 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922085+0000", 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3710000000000002e-06 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922091+0000", 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0340000000000002e-06 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922144+0000", 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3106999999999998e-05 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922248+0000", 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010373400000000001 2024-09-10T10:28:02.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922399+0000", 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015127200000000001 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925201+0000", 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028019189999999999 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926514+0000", 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001312279 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926561+0000", 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000002e-05 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926595+0000", 2024-09-10T10:28:02.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3408000000000003e-05 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.888043183999997, 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005322116, 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:02.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999996e-07 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922707+0000", 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.766e-06 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922717+0000", 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7229999999999999e-06 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922724+0000", 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8410000000000002e-06 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922792+0000", 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8557000000000001e-05 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922912+0000", 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119354 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928025+0000", 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051131919999999999 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:02.915 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.887567031000003, 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049271260000000004, 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:02.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923180+0000", 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923183+0000", 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4000000000000001e-06 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923196+0000", 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3127e-05 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923203+0000", 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.939e-06 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923270+0000", 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7087000000000003e-05 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923359+0000", 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9054999999999994e-05 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928106+0000", 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047469239999999996 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.919 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.88710639, 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045860270000000003, 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2500000000000005e-07 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923644+0000", 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8240000000000001e-06 2024-09-10T10:28:02.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923659+0000", 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5358000000000001e-05 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923666+0000", 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9659999999999999e-06 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923725+0000", 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8674000000000001e-05 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923850+0000", 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000124797 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928226+0000", 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043757830000000003 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.869583955000003, 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005230859, 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:02.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941164+0000", 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.863e-06 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941172+0000", 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2360000000000004e-06 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941190+0000", 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7812000000000002e-05 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941200+0000", 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7599999999999997e-06 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941350+0000", 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015070400000000001 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941501+0000", 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015044200000000001 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:28:02.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000167076 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943487+0000", 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018190260000000001 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946297+0000", 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028102840000000001 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946349+0000", 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1649e-05 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946393+0000", 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4007000000000001e-05 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:02.929 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:02.931 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:02.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.824+0000 7fe016f31640 1 -- 172.21.15.120:0/3818850963 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0100caa20 con 0x7fdff406be30 2024-09-10T10:28:02.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.832+0000 7fe00d7fa640 1 -- 172.21.15.120:0/3818850963 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332739 (secure 0 0 0) 0x7fe004001900 con 0x7fdff406be30 2024-09-10T10:28:02.935 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:28:02.937 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:28:02.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.835+0000 7fe016f31640 1 -- 172.21.15.120:0/3818850963 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdff406be30 msgr2=0x7fdff406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.835+0000 7fe016f31640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdff406be30 0x7fdff406e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe004000f60 tx=0x7fe004009040 comp rx=0 tx=0).stop 2024-09-10T10:28:02.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.835+0000 7fe016f31640 1 -- 172.21.15.120:0/3818850963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0100c37d0 msgr2=0x7fe0100cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:02.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.835+0000 7fe016f31640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0100c37d0 0x7fe0100cad00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00c400 tx=0x7fdffc00c8c0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.835+0000 7fe016f31640 1 -- 172.21.15.120:0/3818850963 shutdown_connections 2024-09-10T10:28:02.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.836+0000 7fe016f31640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdff406be30 0x7fdff406e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.836+0000 7fe016f31640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0100c9b10 0x7fe0100cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.836+0000 7fe016f31640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0100c37d0 0x7fe0100cad00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.836+0000 7fe016f31640 1 --2- 172.21.15.120:0/3818850963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0100c33f0 0x7fe0101e2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:02.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.836+0000 7fe016f31640 1 -- 172.21.15.120:0/3818850963 >> 172.21.15.120:0/3818850963 conn(0x7fe010070290 msgr2=0x7fe0100c5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:02.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.836+0000 7fe016f31640 1 -- 172.21.15.120:0/3818850963 shutdown_connections 2024-09-10T10:28:02.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:02.836+0000 7fe016f31640 1 -- 172.21.15.120:0/3818850963 wait complete. 2024-09-10T10:28:02.939 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-10T10:28:02.106567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603223,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":376,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":321,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220132,"kb_used_data":3900,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754588,"statfs":{"total":773094113280,"available":772868698112,"internally_reserved":0,"allocated":3993600,"data_stored":2875042,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001037"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":64,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.389081+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:27:58.389081+0000","last_peered":"2024-09-10T10:27:58.389081+0000","last_clean":"2024-09-10T10:27:58.389081+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:27:58.389081+0000","last_undegraded":"2024-09-10T10:27:58.389081+0000","last_fullsized":"2024-09-10T10:27:58.389081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068152+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:27:58.068152+0000","last_peered":"2024-09-10T10:27:58.068152+0000","last_clean":"2024-09-10T10:27:58.068152+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:27:58.068152+0000","last_undegraded":"2024-09-10T10:27:58.068152+0000","last_fullsized":"2024-09-10T10:27:58.068152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067142+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:27:58.067142+0000","last_peered":"2024-09-10T10:27:58.067142+0000","last_clean":"2024-09-10T10:27:58.067142+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:27:58.067142+0000","last_undegraded":"2024-09-10T10:27:58.067142+0000","last_fullsized":"2024-09-10T10:27:58.067142+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":54,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068243+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:27:58.068243+0000","last_peered":"2024-09-10T10:27:58.068243+0000","last_clean":"2024-09-10T10:27:58.068243+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:27:58.068243+0000","last_undegraded":"2024-09-10T10:27:58.068243+0000","last_fullsized":"2024-09-10T10:27:58.068243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388869+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:27:58.388869+0000","last_peered":"2024-09-10T10:27:58.388869+0000","last_clean":"2024-09-10T10:27:58.388869+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:27:58.388869+0000","last_undegraded":"2024-09-10T10:27:58.388869+0000","last_fullsized":"2024-09-10T10:27:58.388869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067165+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:27:58.067165+0000","last_peered":"2024-09-10T10:27:58.067165+0000","last_clean":"2024-09-10T10:27:58.067165+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:27:58.067165+0000","last_undegraded":"2024-09-10T10:27:58.067165+0000","last_fullsized":"2024-09-10T10:27:58.067165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"28'12","reported_seq":84,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369484+0000","last_change":"2024-09-10T10:27:51.060242+0000","last_active":"2024-09-10T10:27:58.369484+0000","last_peered":"2024-09-10T10:27:58.369484+0000","last_clean":"2024-09-10T10:27:58.369484+0000","last_became_active":"2024-09-10T10:27:51.049148+0000","last_became_peered":"2024-09-10T10:27:51.049148+0000","last_unstale":"2024-09-10T10:27:58.369484+0000","last_undegraded":"2024-09-10T10:27:58.369484+0000","last_fullsized":"2024-09-10T10:27:58.369484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.604144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067899+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:27:58.067899+0000","last_peered":"2024-09-10T10:27:58.067899+0000","last_clean":"2024-09-10T10:27:58.067899+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:27:58.067899+0000","last_undegraded":"2024-09-10T10:27:58.067899+0000","last_fullsized":"2024-09-10T10:27:58.067899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":52,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068090+0000","last_change":"2024-09-10T10:27:23.955254+0000","last_active":"2024-09-10T10:27:58.068090+0000","last_peered":"2024-09-10T10:27:58.068090+0000","last_clean":"2024-09-10T10:27:58.068090+0000","last_became_active":"2024-09-10T10:27:23.954737+0000","last_became_peered":"2024-09-10T10:27:23.954737+0000","last_unstale":"2024-09-10T10:27:58.068090+0000","last_undegraded":"2024-09-10T10:27:58.068090+0000","last_fullsized":"2024-09-10T10:27:58.068090+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369268+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:27:58.369268+0000","last_peered":"2024-09-10T10:27:58.369268+0000","last_clean":"2024-09-10T10:27:58.369268+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:27:58.369268+0000","last_undegraded":"2024-09-10T10:27:58.369268+0000","last_fullsized":"2024-09-10T10:27:58.369268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388625+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:27:58.388625+0000","last_peered":"2024-09-10T10:27:58.388625+0000","last_clean":"2024-09-10T10:27:58.388625+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:27:58.388625+0000","last_undegraded":"2024-09-10T10:27:58.388625+0000","last_fullsized":"2024-09-10T10:27:58.388625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067428+0000","last_change":"2024-09-10T10:26:45.816587+0000","last_active":"2024-09-10T10:27:58.067428+0000","last_peered":"2024-09-10T10:27:58.067428+0000","last_clean":"2024-09-10T10:27:58.067428+0000","last_became_active":"2024-09-10T10:26:45.816138+0000","last_became_peered":"2024-09-10T10:26:45.816138+0000","last_unstale":"2024-09-10T10:27:58.067428+0000","last_undegraded":"2024-09-10T10:27:58.067428+0000","last_fullsized":"2024-09-10T10:27:58.067428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387787+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:27:58.387787+0000","last_peered":"2024-09-10T10:27:58.387787+0000","last_clean":"2024-09-10T10:27:58.387787+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:27:58.387787+0000","last_undegraded":"2024-09-10T10:27:58.387787+0000","last_fullsized":"2024-09-10T10:27:58.387787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'4","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369419+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:27:58.369419+0000","last_peered":"2024-09-10T10:27:58.369419+0000","last_clean":"2024-09-10T10:27:58.369419+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:27:58.369419+0000","last_undegraded":"2024-09-10T10:27:58.369419+0000","last_fullsized":"2024-09-10T10:27:58.369419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":52,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067983+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:27:58.067983+0000","last_peered":"2024-09-10T10:27:58.067983+0000","last_clean":"2024-09-10T10:27:58.067983+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:27:58.067983+0000","last_undegraded":"2024-09-10T10:27:58.067983+0000","last_fullsized":"2024-09-10T10:27:58.067983+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067571+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:27:58.067571+0000","last_peered":"2024-09-10T10:27:58.067571+0000","last_clean":"2024-09-10T10:27:58.067571+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:27:58.067571+0000","last_undegraded":"2024-09-10T10:27:58.067571+0000","last_fullsized":"2024-09-10T10:27:58.067571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'4","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.327206+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:27:58.327206+0000","last_peered":"2024-09-10T10:27:58.327206+0000","last_clean":"2024-09-10T10:27:58.327206+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:27:58.327206+0000","last_undegraded":"2024-09-10T10:27:58.327206+0000","last_fullsized":"2024-09-10T10:27:58.327206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067616+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:27:58.067616+0000","last_peered":"2024-09-10T10:27:58.067616+0000","last_clean":"2024-09-10T10:27:58.067616+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:27:58.067616+0000","last_undegraded":"2024-09-10T10:27:58.067616+0000","last_fullsized":"2024-09-10T10:27:58.067616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067295+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:27:58.067295+0000","last_peered":"2024-09-10T10:27:58.067295+0000","last_clean":"2024-09-10T10:27:58.067295+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:27:58.067295+0000","last_undegraded":"2024-09-10T10:27:58.067295+0000","last_fullsized":"2024-09-10T10:27:58.067295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":58,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326305+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:27:58.326305+0000","last_peered":"2024-09-10T10:27:58.326305+0000","last_clean":"2024-09-10T10:27:58.326305+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:27:58.326305+0000","last_undegraded":"2024-09-10T10:27:58.326305+0000","last_fullsized":"2024-09-10T10:27:58.326305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326060+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:27:58.326060+0000","last_peered":"2024-09-10T10:27:58.326060+0000","last_clean":"2024-09-10T10:27:58.326060+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:27:58.326060+0000","last_undegraded":"2024-09-10T10:27:58.326060+0000","last_fullsized":"2024-09-10T10:27:58.326060+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068523+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:27:58.068523+0000","last_peered":"2024-09-10T10:27:58.068523+0000","last_clean":"2024-09-10T10:27:58.068523+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:27:58.068523+0000","last_undegraded":"2024-09-10T10:27:58.068523+0000","last_fullsized":"2024-09-10T10:27:58.068523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.368747+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:27:58.368747+0000","last_peered":"2024-09-10T10:27:58.368747+0000","last_clean":"2024-09-10T10:27:58.368747+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:27:58.368747+0000","last_undegraded":"2024-09-10T10:27:58.368747+0000","last_fullsized":"2024-09-10T10:27:58.368747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067467+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:27:58.067467+0000","last_peered":"2024-09-10T10:27:58.067467+0000","last_clean":"2024-09-10T10:27:58.067467+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:27:58.067467+0000","last_undegraded":"2024-09-10T10:27:58.067467+0000","last_fullsized":"2024-09-10T10:27:58.067467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.389255+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:27:58.389255+0000","last_peered":"2024-09-10T10:27:58.389255+0000","last_clean":"2024-09-10T10:27:58.389255+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:27:58.389255+0000","last_undegraded":"2024-09-10T10:27:58.389255+0000","last_fullsized":"2024-09-10T10:27:58.389255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'7","reported_seq":59,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388267+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:27:58.388267+0000","last_peered":"2024-09-10T10:27:58.388267+0000","last_clean":"2024-09-10T10:27:58.388267+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:27:58.388267+0000","last_undegraded":"2024-09-10T10:27:58.388267+0000","last_fullsized":"2024-09-10T10:27:58.388267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068172+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:27:58.068172+0000","last_peered":"2024-09-10T10:27:58.068172+0000","last_clean":"2024-09-10T10:27:58.068172+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:27:58.068172+0000","last_undegraded":"2024-09-10T10:27:58.068172+0000","last_fullsized":"2024-09-10T10:27:58.068172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068442+0000","last_change":"2024-09-10T10:26:41.795085+0000","last_active":"2024-09-10T10:27:58.068442+0000","last_peered":"2024-09-10T10:27:58.068442+0000","last_clean":"2024-09-10T10:27:58.068442+0000","last_became_active":"2024-09-10T10:26:41.794819+0000","last_became_peered":"2024-09-10T10:26:41.794819+0000","last_unstale":"2024-09-10T10:27:58.068442+0000","last_undegraded":"2024-09-10T10:27:58.068442+0000","last_fullsized":"2024-09-10T10:27:58.068442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"28'8","reported_seq":59,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326974+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:27:58.326974+0000","last_peered":"2024-09-10T10:27:58.326974+0000","last_clean":"2024-09-10T10:27:58.326974+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:27:58.326974+0000","last_undegraded":"2024-09-10T10:27:58.326974+0000","last_fullsized":"2024-09-10T10:27:58.326974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067084+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:27:58.067084+0000","last_peered":"2024-09-10T10:27:58.067084+0000","last_clean":"2024-09-10T10:27:58.067084+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:27:58.067084+0000","last_undegraded":"2024-09-10T10:27:58.067084+0000","last_fullsized":"2024-09-10T10:27:58.067084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326709+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:27:58.326709+0000","last_peered":"2024-09-10T10:27:58.326709+0000","last_clean":"2024-09-10T10:27:58.326709+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:27:58.326709+0000","last_undegraded":"2024-09-10T10:27:58.326709+0000","last_fullsized":"2024-09-10T10:27:58.326709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369575+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:27:58.369575+0000","last_peered":"2024-09-10T10:27:58.369575+0000","last_clean":"2024-09-10T10:27:58.369575+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:27:58.369575+0000","last_undegraded":"2024-09-10T10:27:58.369575+0000","last_fullsized":"2024-09-10T10:27:58.369575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388135+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:27:58.388135+0000","last_peered":"2024-09-10T10:27:58.388135+0000","last_clean":"2024-09-10T10:27:58.388135+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:27:58.388135+0000","last_undegraded":"2024-09-10T10:27:58.388135+0000","last_fullsized":"2024-09-10T10:27:58.388135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068710+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:27:58.068710+0000","last_peered":"2024-09-10T10:27:58.068710+0000","last_clean":"2024-09-10T10:27:58.068710+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:27:58.068710+0000","last_undegraded":"2024-09-10T10:27:58.068710+0000","last_fullsized":"2024-09-10T10:27:58.068710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":52,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326788+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:27:58.326788+0000","last_peered":"2024-09-10T10:27:58.326788+0000","last_clean":"2024-09-10T10:27:58.326788+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:27:58.326788+0000","last_undegraded":"2024-09-10T10:27:58.326788+0000","last_fullsized":"2024-09-10T10:27:58.326788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.327026+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:27:58.327026+0000","last_peered":"2024-09-10T10:27:58.327026+0000","last_clean":"2024-09-10T10:27:58.327026+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:27:58.327026+0000","last_undegraded":"2024-09-10T10:27:58.327026+0000","last_fullsized":"2024-09-10T10:27:58.327026+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068428+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:27:58.068428+0000","last_peered":"2024-09-10T10:27:58.068428+0000","last_clean":"2024-09-10T10:27:58.068428+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:27:58.068428+0000","last_undegraded":"2024-09-10T10:27:58.068428+0000","last_fullsized":"2024-09-10T10:27:58.068428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068044+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:27:58.068044+0000","last_peered":"2024-09-10T10:27:58.068044+0000","last_clean":"2024-09-10T10:27:58.068044+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:27:58.068044+0000","last_undegraded":"2024-09-10T10:27:58.068044+0000","last_fullsized":"2024-09-10T10:27:58.068044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326685+0000","last_change":"2024-09-10T10:26:45.814008+0000","last_active":"2024-09-10T10:27:58.326685+0000","last_peered":"2024-09-10T10:27:58.326685+0000","last_clean":"2024-09-10T10:27:58.326685+0000","last_became_active":"2024-09-10T10:26:45.813627+0000","last_became_peered":"2024-09-10T10:26:45.813627+0000","last_unstale":"2024-09-10T10:27:58.326685+0000","last_undegraded":"2024-09-10T10:27:58.326685+0000","last_fullsized":"2024-09-10T10:27:58.326685+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326659+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:27:58.326659+0000","last_peered":"2024-09-10T10:27:58.326659+0000","last_clean":"2024-09-10T10:27:58.326659+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:27:58.326659+0000","last_undegraded":"2024-09-10T10:27:58.326659+0000","last_fullsized":"2024-09-10T10:27:58.326659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"28'5","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388294+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:27:58.388294+0000","last_peered":"2024-09-10T10:27:58.388294+0000","last_clean":"2024-09-10T10:27:58.388294+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:27:58.388294+0000","last_undegraded":"2024-09-10T10:27:58.388294+0000","last_fullsized":"2024-09-10T10:27:58.388294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326927+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:27:58.326927+0000","last_peered":"2024-09-10T10:27:58.326927+0000","last_clean":"2024-09-10T10:27:58.326927+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:27:58.326927+0000","last_undegraded":"2024-09-10T10:27:58.326927+0000","last_fullsized":"2024-09-10T10:27:58.326927+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387942+0000","last_change":"2024-09-10T10:26:41.799224+0000","last_active":"2024-09-10T10:27:58.387942+0000","last_peered":"2024-09-10T10:27:58.387942+0000","last_clean":"2024-09-10T10:27:58.387942+0000","last_became_active":"2024-09-10T10:26:41.798948+0000","last_became_peered":"2024-09-10T10:26:41.798948+0000","last_unstale":"2024-09-10T10:27:58.387942+0000","last_undegraded":"2024-09-10T10:27:58.387942+0000","last_fullsized":"2024-09-10T10:27:58.387942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.368872+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:27:58.368872+0000","last_peered":"2024-09-10T10:27:58.368872+0000","last_clean":"2024-09-10T10:27:58.368872+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:27:58.368872+0000","last_undegraded":"2024-09-10T10:27:58.368872+0000","last_fullsized":"2024-09-10T10:27:58.368872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.389183+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:27:58.389183+0000","last_peered":"2024-09-10T10:27:58.389183+0000","last_clean":"2024-09-10T10:27:58.389183+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:27:58.389183+0000","last_undegraded":"2024-09-10T10:27:58.389183+0000","last_fullsized":"2024-09-10T10:27:58.389183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067478+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:27:58.067478+0000","last_peered":"2024-09-10T10:27:58.067478+0000","last_clean":"2024-09-10T10:27:58.067478+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:27:58.067478+0000","last_undegraded":"2024-09-10T10:27:58.067478+0000","last_fullsized":"2024-09-10T10:27:58.067478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":58,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068326+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:27:58.068326+0000","last_peered":"2024-09-10T10:27:58.068326+0000","last_clean":"2024-09-10T10:27:58.068326+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:27:58.068326+0000","last_undegraded":"2024-09-10T10:27:58.068326+0000","last_fullsized":"2024-09-10T10:27:58.068326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":207,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.368878+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:27:58.368878+0000","last_peered":"2024-09-10T10:27:58.368878+0000","last_clean":"2024-09-10T10:27:58.368878+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:27:58.368878+0000","last_undegraded":"2024-09-10T10:27:58.368878+0000","last_fullsized":"2024-09-10T10:27:58.368878+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387413+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:27:58.387413+0000","last_peered":"2024-09-10T10:27:58.387413+0000","last_clean":"2024-09-10T10:27:58.387413+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:27:58.387413+0000","last_undegraded":"2024-09-10T10:27:58.387413+0000","last_fullsized":"2024-09-10T10:27:58.387413+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369370+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:27:58.369370+0000","last_peered":"2024-09-10T10:27:58.369370+0000","last_clean":"2024-09-10T10:27:58.369370+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:27:58.369370+0000","last_undegraded":"2024-09-10T10:27:58.369370+0000","last_fullsized":"2024-09-10T10:27:58.369370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388562+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:27:58.388562+0000","last_peered":"2024-09-10T10:27:58.388562+0000","last_clean":"2024-09-10T10:27:58.388562+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:27:58.388562+0000","last_undegraded":"2024-09-10T10:27:58.388562+0000","last_fullsized":"2024-09-10T10:27:58.388562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387720+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:27:58.387720+0000","last_peered":"2024-09-10T10:27:58.387720+0000","last_clean":"2024-09-10T10:27:58.387720+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:27:58.387720+0000","last_undegraded":"2024-09-10T10:27:58.387720+0000","last_fullsized":"2024-09-10T10:27:58.387720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067982+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:27:58.067982+0000","last_peered":"2024-09-10T10:27:58.067982+0000","last_clean":"2024-09-10T10:27:58.067982+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:27:58.067982+0000","last_undegraded":"2024-09-10T10:27:58.067982+0000","last_fullsized":"2024-09-10T10:27:58.067982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":54,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326718+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:27:58.326718+0000","last_peered":"2024-09-10T10:27:58.326718+0000","last_clean":"2024-09-10T10:27:58.326718+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:27:58.326718+0000","last_undegraded":"2024-09-10T10:27:58.326718+0000","last_fullsized":"2024-09-10T10:27:58.326718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388403+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:27:58.388403+0000","last_peered":"2024-09-10T10:27:58.388403+0000","last_clean":"2024-09-10T10:27:58.388403+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:27:58.388403+0000","last_undegraded":"2024-09-10T10:27:58.388403+0000","last_fullsized":"2024-09-10T10:27:58.388403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369670+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:27:58.369670+0000","last_peered":"2024-09-10T10:27:58.369670+0000","last_clean":"2024-09-10T10:27:58.369670+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:27:58.369670+0000","last_undegraded":"2024-09-10T10:27:58.369670+0000","last_fullsized":"2024-09-10T10:27:58.369670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.389061+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:27:58.389061+0000","last_peered":"2024-09-10T10:27:58.389061+0000","last_clean":"2024-09-10T10:27:58.389061+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:27:58.389061+0000","last_undegraded":"2024-09-10T10:27:58.389061+0000","last_fullsized":"2024-09-10T10:27:58.389061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067329+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:27:58.067329+0000","last_peered":"2024-09-10T10:27:58.067329+0000","last_clean":"2024-09-10T10:27:58.067329+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:27:58.067329+0000","last_undegraded":"2024-09-10T10:27:58.067329+0000","last_fullsized":"2024-09-10T10:27:58.067329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369467+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:27:58.369467+0000","last_peered":"2024-09-10T10:27:58.369467+0000","last_clean":"2024-09-10T10:27:58.369467+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:27:58.369467+0000","last_undegraded":"2024-09-10T10:27:58.369467+0000","last_fullsized":"2024-09-10T10:27:58.369467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":54,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067371+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:27:58.067371+0000","last_peered":"2024-09-10T10:27:58.067371+0000","last_clean":"2024-09-10T10:27:58.067371+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:27:58.067371+0000","last_undegraded":"2024-09-10T10:27:58.067371+0000","last_fullsized":"2024-09-10T10:27:58.067371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068075+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:27:58.068075+0000","last_peered":"2024-09-10T10:27:58.068075+0000","last_clean":"2024-09-10T10:27:58.068075+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:27:58.068075+0000","last_undegraded":"2024-09-10T10:27:58.068075+0000","last_fullsized":"2024-09-10T10:27:58.068075+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.368999+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:27:58.368999+0000","last_peered":"2024-09-10T10:27:58.368999+0000","last_clean":"2024-09-10T10:27:58.368999+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:27:58.368999+0000","last_undegraded":"2024-09-10T10:27:58.368999+0000","last_fullsized":"2024-09-10T10:27:58.368999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"28'17","reported_seq":69,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387870+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:27:58.387870+0000","last_peered":"2024-09-10T10:27:58.387870+0000","last_clean":"2024-09-10T10:27:58.387870+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:27:58.387870+0000","last_undegraded":"2024-09-10T10:27:58.387870+0000","last_fullsized":"2024-09-10T10:27:58.387870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.389051+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:27:58.389051+0000","last_peered":"2024-09-10T10:27:58.389051+0000","last_clean":"2024-09-10T10:27:58.389051+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:27:58.389051+0000","last_undegraded":"2024-09-10T10:27:58.389051+0000","last_fullsized":"2024-09-10T10:27:58.389051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387625+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:27:58.387625+0000","last_peered":"2024-09-10T10:27:58.387625+0000","last_clean":"2024-09-10T10:27:58.387625+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:27:58.387625+0000","last_undegraded":"2024-09-10T10:27:58.387625+0000","last_fullsized":"2024-09-10T10:27:58.387625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387776+0000","last_change":"2024-09-10T10:26:40.794739+0000","last_active":"2024-09-10T10:27:58.387776+0000","last_peered":"2024-09-10T10:27:58.387776+0000","last_clean":"2024-09-10T10:27:58.387776+0000","last_became_active":"2024-09-10T10:26:40.794430+0000","last_became_peered":"2024-09-10T10:26:40.794430+0000","last_unstale":"2024-09-10T10:27:58.387776+0000","last_undegraded":"2024-09-10T10:27:58.387776+0000","last_fullsized":"2024-09-10T10:27:58.387776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068322+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:27:58.068322+0000","last_peered":"2024-09-10T10:27:58.068322+0000","last_clean":"2024-09-10T10:27:58.068322+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:27:58.068322+0000","last_undegraded":"2024-09-10T10:27:58.068322+0000","last_fullsized":"2024-09-10T10:27:58.068322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369320+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:27:58.369320+0000","last_peered":"2024-09-10T10:27:58.369320+0000","last_clean":"2024-09-10T10:27:58.369320+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:27:58.369320+0000","last_undegraded":"2024-09-10T10:27:58.369320+0000","last_fullsized":"2024-09-10T10:27:58.369320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"28'5","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067956+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:27:58.067956+0000","last_peered":"2024-09-10T10:27:58.067956+0000","last_clean":"2024-09-10T10:27:58.067956+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:27:58.067956+0000","last_undegraded":"2024-09-10T10:27:58.067956+0000","last_fullsized":"2024-09-10T10:27:58.067956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067837+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:27:58.067837+0000","last_peered":"2024-09-10T10:27:58.067837+0000","last_clean":"2024-09-10T10:27:58.067837+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:27:58.067837+0000","last_undegraded":"2024-09-10T10:27:58.067837+0000","last_fullsized":"2024-09-10T10:27:58.067837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388453+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:27:58.388453+0000","last_peered":"2024-09-10T10:27:58.388453+0000","last_clean":"2024-09-10T10:27:58.388453+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:27:58.388453+0000","last_undegraded":"2024-09-10T10:27:58.388453+0000","last_fullsized":"2024-09-10T10:27:58.388453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388712+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:27:58.388712+0000","last_peered":"2024-09-10T10:27:58.388712+0000","last_clean":"2024-09-10T10:27:58.388712+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:27:58.388712+0000","last_undegraded":"2024-09-10T10:27:58.388712+0000","last_fullsized":"2024-09-10T10:27:58.388712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068955+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:27:58.068955+0000","last_peered":"2024-09-10T10:27:58.068955+0000","last_clean":"2024-09-10T10:27:58.068955+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:27:58.068955+0000","last_undegraded":"2024-09-10T10:27:58.068955+0000","last_fullsized":"2024-09-10T10:27:58.068955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369367+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:27:58.369367+0000","last_peered":"2024-09-10T10:27:58.369367+0000","last_clean":"2024-09-10T10:27:58.369367+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:27:58.369367+0000","last_undegraded":"2024-09-10T10:27:58.369367+0000","last_fullsized":"2024-09-10T10:27:58.369367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"28'16","reported_seq":68,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388116+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:27:58.388116+0000","last_peered":"2024-09-10T10:27:58.388116+0000","last_clean":"2024-09-10T10:27:58.388116+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:27:58.388116+0000","last_undegraded":"2024-09-10T10:27:58.388116+0000","last_fullsized":"2024-09-10T10:27:58.388116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6177,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.327115+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:27:58.327115+0000","last_peered":"2024-09-10T10:27:58.327115+0000","last_clean":"2024-09-10T10:27:58.327115+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:27:58.327115+0000","last_undegraded":"2024-09-10T10:27:58.327115+0000","last_fullsized":"2024-09-10T10:27:58.327115+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068107+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:27:58.068107+0000","last_peered":"2024-09-10T10:27:58.068107+0000","last_clean":"2024-09-10T10:27:58.068107+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:27:58.068107+0000","last_undegraded":"2024-09-10T10:27:58.068107+0000","last_fullsized":"2024-09-10T10:27:58.068107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'5","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326652+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:27:58.326652+0000","last_peered":"2024-09-10T10:27:58.326652+0000","last_clean":"2024-09-10T10:27:58.326652+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:27:58.326652+0000","last_undegraded":"2024-09-10T10:27:58.326652+0000","last_fullsized":"2024-09-10T10:27:58.326652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068227+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:27:58.068227+0000","last_peered":"2024-09-10T10:27:58.068227+0000","last_clean":"2024-09-10T10:27:58.068227+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:27:58.068227+0000","last_undegraded":"2024-09-10T10:27:58.068227+0000","last_fullsized":"2024-09-10T10:27:58.068227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.389042+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:27:58.389042+0000","last_peered":"2024-09-10T10:27:58.389042+0000","last_clean":"2024-09-10T10:27:58.389042+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:27:58.389042+0000","last_undegraded":"2024-09-10T10:27:58.389042+0000","last_fullsized":"2024-09-10T10:27:58.389042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068453+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:27:58.068453+0000","last_peered":"2024-09-10T10:27:58.068453+0000","last_clean":"2024-09-10T10:27:58.068453+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:27:58.068453+0000","last_undegraded":"2024-09-10T10:27:58.068453+0000","last_fullsized":"2024-09-10T10:27:58.068453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":59,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067194+0000","last_change":"2024-09-10T10:27:44.443362+0000","last_active":"2024-09-10T10:27:58.067194+0000","last_peered":"2024-09-10T10:27:58.067194+0000","last_clean":"2024-09-10T10:27:58.067194+0000","last_became_active":"2024-09-10T10:27:44.442834+0000","last_became_peered":"2024-09-10T10:27:44.442834+0000","last_unstale":"2024-09-10T10:27:58.067194+0000","last_undegraded":"2024-09-10T10:27:58.067194+0000","last_fullsized":"2024-09-10T10:27:58.067194+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:53.600117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326579+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:27:58.326579+0000","last_peered":"2024-09-10T10:27:58.326579+0000","last_clean":"2024-09-10T10:27:58.326579+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:27:58.326579+0000","last_undegraded":"2024-09-10T10:27:58.326579+0000","last_fullsized":"2024-09-10T10:27:58.326579+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"28'5","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388241+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:27:58.388241+0000","last_peered":"2024-09-10T10:27:58.388241+0000","last_clean":"2024-09-10T10:27:58.388241+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:27:58.388241+0000","last_undegraded":"2024-09-10T10:27:58.388241+0000","last_fullsized":"2024-09-10T10:27:58.388241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067265+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:27:58.067265+0000","last_peered":"2024-09-10T10:27:58.067265+0000","last_clean":"2024-09-10T10:27:58.067265+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:27:58.067265+0000","last_undegraded":"2024-09-10T10:27:58.067265+0000","last_fullsized":"2024-09-10T10:27:58.067265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":44,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387667+0000","last_change":"2024-09-10T10:26:46.193656+0000","last_active":"2024-09-10T10:27:58.387667+0000","last_peered":"2024-09-10T10:27:58.387667+0000","last_clean":"2024-09-10T10:27:58.387667+0000","last_became_active":"2024-09-10T10:26:46.193109+0000","last_became_peered":"2024-09-10T10:26:46.193109+0000","last_unstale":"2024-09-10T10:27:58.387667+0000","last_undegraded":"2024-09-10T10:27:58.387667+0000","last_fullsized":"2024-09-10T10:27:58.387667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326204+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:27:58.326204+0000","last_peered":"2024-09-10T10:27:58.326204+0000","last_clean":"2024-09-10T10:27:58.326204+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:27:58.326204+0000","last_undegraded":"2024-09-10T10:27:58.326204+0000","last_fullsized":"2024-09-10T10:27:58.326204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067741+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:27:58.067741+0000","last_peered":"2024-09-10T10:27:58.067741+0000","last_clean":"2024-09-10T10:27:58.067741+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:27:58.067741+0000","last_undegraded":"2024-09-10T10:27:58.067741+0000","last_fullsized":"2024-09-10T10:27:58.067741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388074+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:27:58.388074+0000","last_peered":"2024-09-10T10:27:58.388074+0000","last_clean":"2024-09-10T10:27:58.388074+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:27:58.388074+0000","last_undegraded":"2024-09-10T10:27:58.388074+0000","last_fullsized":"2024-09-10T10:27:58.388074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067249+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:27:58.067249+0000","last_peered":"2024-09-10T10:27:58.067249+0000","last_clean":"2024-09-10T10:27:58.067249+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:27:58.067249+0000","last_undegraded":"2024-09-10T10:27:58.067249+0000","last_fullsized":"2024-09-10T10:27:58.067249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067517+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:27:58.067517+0000","last_peered":"2024-09-10T10:27:58.067517+0000","last_clean":"2024-09-10T10:27:58.067517+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:27:58.067517+0000","last_undegraded":"2024-09-10T10:27:58.067517+0000","last_fullsized":"2024-09-10T10:27:58.067517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":59,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387875+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:27:58.387875+0000","last_peered":"2024-09-10T10:27:58.387875+0000","last_clean":"2024-09-10T10:27:58.387875+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:27:58.387875+0000","last_undegraded":"2024-09-10T10:27:58.387875+0000","last_fullsized":"2024-09-10T10:27:58.387875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388573+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:27:58.388573+0000","last_peered":"2024-09-10T10:27:58.388573+0000","last_clean":"2024-09-10T10:27:58.388573+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:27:58.388573+0000","last_undegraded":"2024-09-10T10:27:58.388573+0000","last_fullsized":"2024-09-10T10:27:58.388573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388471+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:27:58.388471+0000","last_peered":"2024-09-10T10:27:58.388471+0000","last_clean":"2024-09-10T10:27:58.388471+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:27:58.388471+0000","last_undegraded":"2024-09-10T10:27:58.388471+0000","last_fullsized":"2024-09-10T10:27:58.388471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'6","reported_seq":58,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067686+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:27:58.067686+0000","last_peered":"2024-09-10T10:27:58.067686+0000","last_clean":"2024-09-10T10:27:58.067686+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:27:58.067686+0000","last_undegraded":"2024-09-10T10:27:58.067686+0000","last_fullsized":"2024-09-10T10:27:58.067686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067644+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:27:58.067644+0000","last_peered":"2024-09-10T10:27:58.067644+0000","last_clean":"2024-09-10T10:27:58.067644+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:27:58.067644+0000","last_undegraded":"2024-09-10T10:27:58.067644+0000","last_fullsized":"2024-09-10T10:27:58.067644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067993+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:27:58.067993+0000","last_peered":"2024-09-10T10:27:58.067993+0000","last_clean":"2024-09-10T10:27:58.067993+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:27:58.067993+0000","last_undegraded":"2024-09-10T10:27:58.067993+0000","last_fullsized":"2024-09-10T10:27:58.067993+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387533+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:27:58.387533+0000","last_peered":"2024-09-10T10:27:58.387533+0000","last_clean":"2024-09-10T10:27:58.387533+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:27:58.387533+0000","last_undegraded":"2024-09-10T10:27:58.387533+0000","last_fullsized":"2024-09-10T10:27:58.387533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388210+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:27:58.388210+0000","last_peered":"2024-09-10T10:27:58.388210+0000","last_clean":"2024-09-10T10:27:58.388210+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:27:58.388210+0000","last_undegraded":"2024-09-10T10:27:58.388210+0000","last_fullsized":"2024-09-10T10:27:58.388210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067877+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:27:58.067877+0000","last_peered":"2024-09-10T10:27:58.067877+0000","last_clean":"2024-09-10T10:27:58.067877+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:27:58.067877+0000","last_undegraded":"2024-09-10T10:27:58.067877+0000","last_fullsized":"2024-09-10T10:27:58.067877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388493+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:27:58.388493+0000","last_peered":"2024-09-10T10:27:58.388493+0000","last_clean":"2024-09-10T10:27:58.388493+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:27:58.388493+0000","last_undegraded":"2024-09-10T10:27:58.388493+0000","last_fullsized":"2024-09-10T10:27:58.388493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068511+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:27:58.068511+0000","last_peered":"2024-09-10T10:27:58.068511+0000","last_clean":"2024-09-10T10:27:58.068511+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:27:58.068511+0000","last_undegraded":"2024-09-10T10:27:58.068511+0000","last_fullsized":"2024-09-10T10:27:58.068511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326539+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:27:58.326539+0000","last_peered":"2024-09-10T10:27:58.326539+0000","last_clean":"2024-09-10T10:27:58.326539+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:27:58.326539+0000","last_undegraded":"2024-09-10T10:27:58.326539+0000","last_fullsized":"2024-09-10T10:27:58.326539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388230+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:27:58.388230+0000","last_peered":"2024-09-10T10:27:58.388230+0000","last_clean":"2024-09-10T10:27:58.388230+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:27:58.388230+0000","last_undegraded":"2024-09-10T10:27:58.388230+0000","last_fullsized":"2024-09-10T10:27:58.388230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068408+0000","last_change":"2024-09-10T10:26:41.802903+0000","last_active":"2024-09-10T10:27:58.068408+0000","last_peered":"2024-09-10T10:27:58.068408+0000","last_clean":"2024-09-10T10:27:58.068408+0000","last_became_active":"2024-09-10T10:26:41.802586+0000","last_became_peered":"2024-09-10T10:26:41.802586+0000","last_unstale":"2024-09-10T10:27:58.068408+0000","last_undegraded":"2024-09-10T10:27:58.068408+0000","last_fullsized":"2024-09-10T10:27:58.068408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067653+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:27:58.067653+0000","last_peered":"2024-09-10T10:27:58.067653+0000","last_clean":"2024-09-10T10:27:58.067653+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:27:58.067653+0000","last_undegraded":"2024-09-10T10:27:58.067653+0000","last_fullsized":"2024-09-10T10:27:58.067653+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067479+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:27:58.067479+0000","last_peered":"2024-09-10T10:27:58.067479+0000","last_clean":"2024-09-10T10:27:58.067479+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:27:58.067479+0000","last_undegraded":"2024-09-10T10:27:58.067479+0000","last_fullsized":"2024-09-10T10:27:58.067479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067793+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:27:58.067793+0000","last_peered":"2024-09-10T10:27:58.067793+0000","last_clean":"2024-09-10T10:27:58.067793+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:27:58.067793+0000","last_undegraded":"2024-09-10T10:27:58.067793+0000","last_fullsized":"2024-09-10T10:27:58.067793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388315+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:27:58.388315+0000","last_peered":"2024-09-10T10:27:58.388315+0000","last_clean":"2024-09-10T10:27:58.388315+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:27:58.388315+0000","last_undegraded":"2024-09-10T10:27:58.388315+0000","last_fullsized":"2024-09-10T10:27:58.388315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388106+0000","last_change":"2024-09-10T10:26:41.787435+0000","last_active":"2024-09-10T10:27:58.388106+0000","last_peered":"2024-09-10T10:27:58.388106+0000","last_clean":"2024-09-10T10:27:58.388106+0000","last_became_active":"2024-09-10T10:26:41.787083+0000","last_became_peered":"2024-09-10T10:26:41.787083+0000","last_unstale":"2024-09-10T10:27:58.388106+0000","last_undegraded":"2024-09-10T10:27:58.388106+0000","last_fullsized":"2024-09-10T10:27:58.388106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369490+0000","last_change":"2024-09-10T10:26:41.792587+0000","last_active":"2024-09-10T10:27:58.369490+0000","last_peered":"2024-09-10T10:27:58.369490+0000","last_clean":"2024-09-10T10:27:58.369490+0000","last_became_active":"2024-09-10T10:26:41.792320+0000","last_became_peered":"2024-09-10T10:26:41.792320+0000","last_unstale":"2024-09-10T10:27:58.369490+0000","last_undegraded":"2024-09-10T10:27:58.369490+0000","last_fullsized":"2024-09-10T10:27:58.369490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388971+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:27:58.388971+0000","last_peered":"2024-09-10T10:27:58.388971+0000","last_clean":"2024-09-10T10:27:58.388971+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:27:58.388971+0000","last_undegraded":"2024-09-10T10:27:58.388971+0000","last_fullsized":"2024-09-10T10:27:58.388971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.068824+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:27:58.068824+0000","last_peered":"2024-09-10T10:27:58.068824+0000","last_clean":"2024-09-10T10:27:58.068824+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:27:58.068824+0000","last_undegraded":"2024-09-10T10:27:58.068824+0000","last_fullsized":"2024-09-10T10:27:58.068824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067486+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:27:58.067486+0000","last_peered":"2024-09-10T10:27:58.067486+0000","last_clean":"2024-09-10T10:27:58.067486+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:27:58.067486+0000","last_undegraded":"2024-09-10T10:27:58.067486+0000","last_fullsized":"2024-09-10T10:27:58.067486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326210+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:27:58.326210+0000","last_peered":"2024-09-10T10:27:58.326210+0000","last_clean":"2024-09-10T10:27:58.326210+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:27:58.326210+0000","last_undegraded":"2024-09-10T10:27:58.326210+0000","last_fullsized":"2024-09-10T10:27:58.326210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388356+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:27:58.388356+0000","last_peered":"2024-09-10T10:27:58.388356+0000","last_clean":"2024-09-10T10:27:58.388356+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:27:58.388356+0000","last_undegraded":"2024-09-10T10:27:58.388356+0000","last_fullsized":"2024-09-10T10:27:58.388356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067530+0000","last_change":"2024-09-10T10:26:41.802117+0000","last_active":"2024-09-10T10:27:58.067530+0000","last_peered":"2024-09-10T10:27:58.067530+0000","last_clean":"2024-09-10T10:27:58.067530+0000","last_became_active":"2024-09-10T10:26:41.801798+0000","last_became_peered":"2024-09-10T10:26:41.801798+0000","last_unstale":"2024-09-10T10:27:58.067530+0000","last_undegraded":"2024-09-10T10:27:58.067530+0000","last_fullsized":"2024-09-10T10:27:58.067530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.330721+0000","last_change":"2024-09-10T10:27:58.327792+0000","last_active":"2024-09-10T10:27:58.330721+0000","last_peered":"2024-09-10T10:27:58.330721+0000","last_clean":"2024-09-10T10:27:58.330721+0000","last_became_active":"2024-09-10T10:27:58.327502+0000","last_became_peered":"2024-09-10T10:27:58.327502+0000","last_unstale":"2024-09-10T10:27:58.330721+0000","last_undegraded":"2024-09-10T10:27:58.330721+0000","last_fullsized":"2024-09-10T10:27:58.330721+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.387977+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:27:58.387977+0000","last_peered":"2024-09-10T10:27:58.387977+0000","last_clean":"2024-09-10T10:27:58.387977+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:27:58.387977+0000","last_undegraded":"2024-09-10T10:27:58.387977+0000","last_fullsized":"2024-09-10T10:27:58.387977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388122+0000","last_change":"2024-09-10T10:27:23.958305+0000","last_active":"2024-09-10T10:27:58.388122+0000","last_peered":"2024-09-10T10:27:58.388122+0000","last_clean":"2024-09-10T10:27:58.388122+0000","last_became_active":"2024-09-10T10:27:23.957954+0000","last_became_peered":"2024-09-10T10:27:23.957954+0000","last_unstale":"2024-09-10T10:27:58.388122+0000","last_undegraded":"2024-09-10T10:27:58.388122+0000","last_fullsized":"2024-09-10T10:27:58.388122+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:32.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369641+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:27:58.369641+0000","last_peered":"2024-09-10T10:27:58.369641+0000","last_clean":"2024-09-10T10:27:58.369641+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:27:58.369641+0000","last_undegraded":"2024-09-10T10:27:58.369641+0000","last_fullsized":"2024-09-10T10:27:58.369641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067154+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:27:58.067154+0000","last_peered":"2024-09-10T10:27:58.067154+0000","last_clean":"2024-09-10T10:27:58.067154+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:27:58.067154+0000","last_undegraded":"2024-09-10T10:27:58.067154+0000","last_fullsized":"2024-09-10T10:27:58.067154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.368983+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:27:58.368983+0000","last_peered":"2024-09-10T10:27:58.368983+0000","last_clean":"2024-09-10T10:27:58.368983+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:27:58.368983+0000","last_undegraded":"2024-09-10T10:27:58.368983+0000","last_fullsized":"2024-09-10T10:27:58.368983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067719+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:27:58.067719+0000","last_peered":"2024-09-10T10:27:58.067719+0000","last_clean":"2024-09-10T10:27:58.067719+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:27:58.067719+0000","last_undegraded":"2024-09-10T10:27:58.067719+0000","last_fullsized":"2024-09-10T10:27:58.067719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369169+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:27:58.369169+0000","last_peered":"2024-09-10T10:27:58.369169+0000","last_clean":"2024-09-10T10:27:58.369169+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:27:58.369169+0000","last_undegraded":"2024-09-10T10:27:58.369169+0000","last_fullsized":"2024-09-10T10:27:58.369169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.067534+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:27:58.067534+0000","last_peered":"2024-09-10T10:27:58.067534+0000","last_clean":"2024-09-10T10:27:58.067534+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:27:58.067534+0000","last_undegraded":"2024-09-10T10:27:58.067534+0000","last_fullsized":"2024-09-10T10:27:58.067534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.388673+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:27:58.388673+0000","last_peered":"2024-09-10T10:27:58.388673+0000","last_clean":"2024-09-10T10:27:58.388673+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:27:58.388673+0000","last_undegraded":"2024-09-10T10:27:58.388673+0000","last_fullsized":"2024-09-10T10:27:58.388673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.326906+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:27:58.326906+0000","last_peered":"2024-09-10T10:27:58.326906+0000","last_clean":"2024-09-10T10:27:58.326906+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:27:58.326906+0000","last_undegraded":"2024-09-10T10:27:58.326906+0000","last_fullsized":"2024-09-10T10:27:58.326906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:27:58.369579+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:27:58.369579+0000","last_peered":"2024-09-10T10:27:58.369579+0000","last_clean":"2024-09-10T10:27:58.369579+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:27:58.369579+0000","last_undegraded":"2024-09-10T10:27:58.369579+0000","last_fullsized":"2024-09-10T10:27:58.369579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":12855,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":217,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":25710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":189,"ondisk_log_size":189,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":552960,"data_stored":802170,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":216413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":213378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":475136,"data_stored":210776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":216703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":210908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":204006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":800688,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":7257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:28:02.944 INFO:tasks.thrashosds.thrasher:Setting 4.4 to [3, 6, 5, 7, 0, 4, 2, 1] 2024-09-10T10:28:02.944 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.4', '3', '6', '5', '7', '0', '4', '2', '1'] 2024-09-10T10:28:02.944 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 4.4 3 6 5 7 0 4 2 1 2024-09-10T10:28:02.945 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:02.930+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:03.031 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:03.030+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:03.081 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:03.081 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:03.081 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:03.081 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:03.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.085+0000 7f7df872a640 1 Processor -- start 2024-09-10T10:28:03.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.085+0000 7f7df872a640 1 -- start start 2024-09-10T10:28:03.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.086+0000 7f7df872a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7df00ca4c0 0x7f7df0158d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.086+0000 7f7df872a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7df0152cf0 0x7f7df0159270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.086+0000 7f7df872a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df0158830 con 0x7f7df00ca8a0 2024-09-10T10:28:03.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.086+0000 7f7df872a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7df00c8570 con 0x7f7df00ca4c0 2024-09-10T10:28:03.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.086+0000 7f7df872a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7df00ce480 con 0x7f7df0152cf0 2024-09-10T10:28:03.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.087+0000 7f7df5cc5640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f7df00ca8a0 0x7f7df01525e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:53104/0 (socket says 172.21.15.120:53104) 2024-09-10T10:28:03.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.087+0000 7f7df5cc5640 1 -- 172.21.15.120:0/1076898958 learned_addr learned my addr 172.21.15.120:0/1076898958 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:03.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.087+0000 7f7df64c6640 1 --2- 172.21.15.120:0/1076898958 >> v2:172.21.15.120:3300/0 conn(0x7f7df00ca4c0 0x7f7df0158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.087+0000 7f7df6cc7640 1 --2- 172.21.15.120:0/1076898958 >> v2:172.21.15.134:3300/0 conn(0x7f7df0152cf0 0x7f7df0159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.087+0000 7f7df64c6640 1 -- 172.21.15.120:0/1076898958 >> v2:172.21.15.134:3300/0 conn(0x7f7df0152cf0 msgr2=0x7f7df0159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:03.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.087+0000 7f7df64c6640 1 --2- 172.21.15.120:0/1076898958 >> v2:172.21.15.134:3300/0 conn(0x7f7df0152cf0 0x7f7df0159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:03.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.087+0000 7f7df64c6640 1 -- 172.21.15.120:0/1076898958 >> v1:172.21.15.120:6789/0 conn(0x7f7df00ca8a0 legacy=0x7f7df01525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:03.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.087+0000 7f7df64c6640 1 -- 172.21.15.120:0/1076898958 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df00ce760 con 0x7f7df00ca4c0 2024-09-10T10:28:03.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.088+0000 7f7df64c6640 1 --2- 172.21.15.120:0/1076898958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df00ca4c0 0x7f7df0158d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7df00b47d0 tx=0x7f7dec02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc022ea976b26b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:03.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.088+0000 7f7df54c4640 1 -- 172.21.15.120:0/1076898958 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec037070 con 0x7f7df00ca4c0 2024-09-10T10:28:03.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.088+0000 7f7df54c4640 1 -- 172.21.15.120:0/1076898958 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dec02ad30 con 0x7f7df00ca4c0 2024-09-10T10:28:03.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.088+0000 7f7df54c4640 1 -- 172.21.15.120:0/1076898958 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec032660 con 0x7f7df00ca4c0 2024-09-10T10:28:03.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.089+0000 7f7df872a640 1 -- 172.21.15.120:0/1076898958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df00ca4c0 msgr2=0x7f7df0158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:03.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.089+0000 7f7df872a640 1 --2- 172.21.15.120:0/1076898958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df00ca4c0 0x7f7df0158d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7df00b47d0 tx=0x7f7dec02a0a0 comp rx=0 tx=0).stop 2024-09-10T10:28:03.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.089+0000 7f7df872a640 1 -- 172.21.15.120:0/1076898958 shutdown_connections 2024-09-10T10:28:03.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.089+0000 7f7df872a640 1 --2- 172.21.15.120:0/1076898958 >> v2:172.21.15.134:3300/0 conn(0x7f7df0152cf0 0x7f7df0159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:03.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.089+0000 7f7df872a640 1 --2- 172.21.15.120:0/1076898958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df00ca4c0 0x7f7df0158d30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:03.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.089+0000 7f7df872a640 1 -- 172.21.15.120:0/1076898958 >> 172.21.15.120:0/1076898958 conn(0x7f7df006c240 msgr2=0x7f7df0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:03.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.089+0000 7f7df872a640 1 -- 172.21.15.120:0/1076898958 shutdown_connections 2024-09-10T10:28:03.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.089+0000 7f7df872a640 1 -- 172.21.15.120:0/1076898958 wait complete. 2024-09-10T10:28:03.092 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.090+0000 7f7df872a640 1 Processor -- start 2024-09-10T10:28:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.090+0000 7f7df872a640 1 -- start start 2024-09-10T10:28:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df872a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df00ca4c0 0x7f7df0155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df872a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df00ca8a0 0x7f7df014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df872a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df0152cf0 0x7f7df00c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df872a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00c9e80 con 0x7f7df0152cf0 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df872a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00c9d00 con 0x7f7df00ca8a0 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df872a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7df00ca000 con 0x7f7df00ca4c0 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df6cc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df0152cf0 0x7f7df00c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df6cc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df0152cf0 0x7f7df00c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35012/0 (socket says 172.21.15.120:35012) 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df6cc7640 1 -- 172.21.15.120:0/2953821309 learned_addr learned my addr 172.21.15.120:0/2953821309 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df5cc5640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df00ca8a0 0x7f7df014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df64c6640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df00ca4c0 0x7f7df0155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df6cc7640 1 -- 172.21.15.120:0/2953821309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df00ca4c0 msgr2=0x7f7df0155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df6cc7640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df00ca4c0 0x7f7df0155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df6cc7640 1 -- 172.21.15.120:0/2953821309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df00ca8a0 msgr2=0x7f7df014cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df6cc7640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df00ca8a0 0x7f7df014cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df6cc7640 1 -- 172.21.15.120:0/2953821309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df0151410 con 0x7f7df0152cf0 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.091+0000 7f7df6cc7640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df0152cf0 0x7f7df00c6ec0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7de800efc0 tx=0x7f7de800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.092+0000 7f7de6ffd640 1 -- 172.21.15.120:0/2953821309 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8018020 con 0x7f7df0152cf0 2024-09-10T10:28:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.092+0000 7f7de6ffd640 1 -- 172.21.15.120:0/2953821309 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de8014480 con 0x7f7df0152cf0 2024-09-10T10:28:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.092+0000 7f7df872a640 1 -- 172.21.15.120:0/2953821309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df01f1790 con 0x7f7df0152cf0 2024-09-10T10:28:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.092+0000 7f7de6ffd640 1 -- 172.21.15.120:0/2953821309 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8014870 con 0x7f7df0152cf0 2024-09-10T10:28:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.092+0000 7f7df872a640 1 -- 172.21.15.120:0/2953821309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df0151a70 con 0x7f7df0152cf0 2024-09-10T10:28:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.093+0000 7f7df872a640 1 -- 172.21.15.120:0/2953821309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df01525c0 con 0x7f7df0152cf0 2024-09-10T10:28:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.093+0000 7f7de6ffd640 1 -- 172.21.15.120:0/2953821309 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7de8005d10 con 0x7f7df0152cf0 2024-09-10T10:28:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.094+0000 7f7de6ffd640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7dd406be30 0x7f7dd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.094+0000 7f7df64c6640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7dd406be30 0x7f7dd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.094+0000 7f7de6ffd640 1 -- 172.21.15.120:0/2953821309 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6764+0+0 (secure 0 0 0) 0x7f7de808bba0 con 0x7f7df0152cf0 2024-09-10T10:28:03.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.094+0000 7f7df64c6640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7dd406be30 0x7f7dd406e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7dec034000 tx=0x7f7dec02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:03.099 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-10T10:28:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.099+0000 7f7de6ffd640 1 -- 172.21.15.120:0/2953821309 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7df01525c0 con 0x7f7df0152cf0 2024-09-10T10:28:03.132 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:03.131+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:03.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.206+0000 7f544bb8d640 1 Processor -- start 2024-09-10T10:28:03.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.206+0000 7f544bb8d640 1 -- start start 2024-09-10T10:28:03.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f544bb8d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5444152d60 0x7f54440c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f544bb8d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f544415c230 con 0x7f54440c8df0 2024-09-10T10:28:03.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f544bb8d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f544415c510 con 0x7f54440c3df0 2024-09-10T10:28:03.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f544bb8d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f544415cdb0 con 0x7f5444152d60 2024-09-10T10:28:03.208 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:03.209 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:03.209 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:03.209 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:03.209 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:03.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f5449128640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f54440c8df0 0x7f5444159020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:53116/0 (socket says 172.21.15.120:53116) 2024-09-10T10:28:03.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f5449929640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5444152d60 0x7f54440c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f5449128640 1 -- 172.21.15.120:0/3696494502 learned_addr learned my addr 172.21.15.120:0/3696494502 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:03.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f5449929640 1 -- 172.21.15.120:0/3696494502 >> v1:172.21.15.134:6789/0 conn(0x7f54440c3df0 legacy=0x7f5444163190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:03.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f5449929640 1 -- 172.21.15.120:0/3696494502 >> v1:172.21.15.120:6789/0 conn(0x7f54440c8df0 legacy=0x7f5444159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:03.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f5449929640 1 -- 172.21.15.120:0/3696494502 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544415c7f0 con 0x7f5444152d60 2024-09-10T10:28:03.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.207+0000 7f5448927640 1 -- 172.21.15.120:0/3696494502 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3717968282 0 0) 0x7f544415c230 con 0x7f54440c8df0 2024-09-10T10:28:03.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.208+0000 7f5449929640 1 --2- 172.21.15.120:0/3696494502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5444152d60 0x7f54440c3410 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f54400097a0 tx=0x7f544002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f9f893867702e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:03.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.209+0000 7f5448927640 1 -- 172.21.15.120:0/3696494502 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440037070 con 0x7f5444152d60 2024-09-10T10:28:03.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.209+0000 7f5448927640 1 -- 172.21.15.120:0/3696494502 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5440032640 con 0x7f5444152d60 2024-09-10T10:28:03.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.209+0000 7f5448927640 1 -- 172.21.15.120:0/3696494502 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440032a10 con 0x7f5444152d60 2024-09-10T10:28:03.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.209+0000 7f544bb8d640 1 -- 172.21.15.120:0/3696494502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5444152d60 msgr2=0x7f54440c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:03.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.209+0000 7f544bb8d640 1 --2- 172.21.15.120:0/3696494502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5444152d60 0x7f54440c3410 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f54400097a0 tx=0x7f544002a2c0 comp rx=0 tx=0).stop 2024-09-10T10:28:03.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.209+0000 7f544bb8d640 1 -- 172.21.15.120:0/3696494502 shutdown_connections 2024-09-10T10:28:03.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.209+0000 7f544bb8d640 1 --2- 172.21.15.120:0/3696494502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5444152d60 0x7f54440c3410 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:03.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.209+0000 7f544bb8d640 1 -- 172.21.15.120:0/3696494502 >> 172.21.15.120:0/3696494502 conn(0x7f54440716e0 msgr2=0x7f5444071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:03.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.210+0000 7f544bb8d640 1 -- 172.21.15.120:0/3696494502 shutdown_connections 2024-09-10T10:28:03.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.210+0000 7f544bb8d640 1 -- 172.21.15.120:0/3696494502 wait complete. 2024-09-10T10:28:03.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.211+0000 7f544bb8d640 1 Processor -- start 2024-09-10T10:28:03.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.211+0000 7f544bb8d640 1 -- start start 2024-09-10T10:28:03.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.211+0000 7f544bb8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54440c3df0 0x7f54441f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.211+0000 7f544bb8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54440c8df0 0x7f54441f0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f544bb8d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5444152d60 0x7f54441f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f544bb8d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5444158b90 con 0x7f54440c3df0 2024-09-10T10:28:03.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f544bb8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54440cf3c0 con 0x7f54440c8df0 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f544bb8d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5444159730 con 0x7f5444152d60 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449929640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54440c3df0 0x7f54441f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449929640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54440c3df0 0x7f54441f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35026/0 (socket says 172.21.15.120:35026) 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449929640 1 -- 172.21.15.120:0/4003382731 learned_addr learned my addr 172.21.15.120:0/4003382731 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449128640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54440c8df0 0x7f54441f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f544a12a640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5444152d60 0x7f54441f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449929640 1 -- 172.21.15.120:0/4003382731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5444152d60 msgr2=0x7f54441f0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449929640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5444152d60 0x7f54441f0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449929640 1 -- 172.21.15.120:0/4003382731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54440c8df0 msgr2=0x7f54441f0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449929640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54440c8df0 0x7f54441f0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449929640 1 -- 172.21.15.120:0/4003382731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544406d8a0 con 0x7f54440c3df0 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449128640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54440c8df0 0x7f54441f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:03.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.212+0000 7f5449929640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54440c3df0 0x7f54441f4100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5440000c00 tx=0x7f54400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:03.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.213+0000 7f543a7fc640 1 -- 172.21.15.120:0/4003382731 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440041020 con 0x7f54440c3df0 2024-09-10T10:28:03.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.213+0000 7f543a7fc640 1 -- 172.21.15.120:0/4003382731 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5440030970 con 0x7f54440c3df0 2024-09-10T10:28:03.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.213+0000 7f543a7fc640 1 -- 172.21.15.120:0/4003382731 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440037040 con 0x7f54440c3df0 2024-09-10T10:28:03.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.213+0000 7f544bb8d640 1 -- 172.21.15.120:0/4003382731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544406db80 con 0x7f54440c3df0 2024-09-10T10:28:03.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.213+0000 7f544bb8d640 1 -- 172.21.15.120:0/4003382731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f544406e140 con 0x7f54440c3df0 2024-09-10T10:28:03.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.214+0000 7f544bb8d640 1 -- 172.21.15.120:0/4003382731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f544415be60 con 0x7f54440c3df0 2024-09-10T10:28:03.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.215+0000 7f543a7fc640 1 -- 172.21.15.120:0/4003382731 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5440044020 con 0x7f54440c3df0 2024-09-10T10:28:03.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.215+0000 7f543a7fc640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f542806be30 0x7f542806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:03.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.216+0000 7f543a7fc640 1 -- 172.21.15.120:0/4003382731 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6764+0+0 (secure 0 0 0) 0x7f54400aba90 con 0x7f54440c3df0 2024-09-10T10:28:03.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.216+0000 7f5449128640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f542806be30 0x7f542806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:03.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.216+0000 7f5449128640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f542806be30 0x7f542806e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5434007f70 tx=0x7f5434007f00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:03.219 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:03.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.224+0000 7f543a7fc640 1 -- 172.21.15.120:0/4003382731 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f544004b030 con 0x7f54440c3df0 2024-09-10T10:28:03.232 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:03.231+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:03.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.310+0000 7f7df872a640 1 -- 172.21.15.120:0/2953821309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pg-upmap-items", "pgid": "4.4", "id": [3, 6, 5, 7, 0, 4, 2, 1]} v 0) -- 0x7f7df00c97b0 con 0x7f7df0152cf0 2024-09-10T10:28:03.329 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:03.329 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:03.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:03.329 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:03.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.546642272, 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078690670000000004, 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2200000000000004e-07 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780048+0000", 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2427999999999999e-05 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780057+0000", 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1090000000000004e-06 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780063+0000", 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9780000000000002e-06 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011213480000000001 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781281+0000", 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6728e-05 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781382+0000", 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101455 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784820+0000", 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034376200000000002 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787779+0000", 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002959373 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787852+0000", 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3135999999999999e-05 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787894+0000", 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1369999999999999e-05 2024-09-10T10:28:03.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.546594730999999, 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079289789999999992, 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780073+0000", 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780075+0000", 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5129999999999999e-06 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780082+0000", 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9410000000000001e-06 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780087+0000", 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1880000000000001e-06 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781620+0000", 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001532346 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781698+0000", 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8232999999999994e-05 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781816+0000", 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011825800000000001 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784824+0000", 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030076349999999998 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787937+0000", 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031134829999999998 2024-09-10T10:28:03.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787969+0000", 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1745999999999997e-05 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788001+0000", 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2132000000000003e-05 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.546564516000004, 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079888619999999994, 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780103+0000", 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780106+0000", 2024-09-10T10:28:03.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1379999999999999e-06 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780115+0000", 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4299999999999995e-06 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780122+0000", 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3380000000000001e-06 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782041+0000", 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019193700000000001 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782149+0000", 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107493 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782246+0000", 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999996e-05 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785205+0000", 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002958535 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788041+0000", 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028359650000000002 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788069+0000", 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8123000000000002e-05 2024-09-10T10:28:03.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788091+0000", 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2506000000000001e-05 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.546535974999998, 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080531070000000003, 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.346 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780134+0000", 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0570000000000001e-06 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780143+0000", 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-06 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780156+0000", 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3179e-05 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023278280000000001 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782555+0000", 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0494000000000006e-05 2024-09-10T10:28:03.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782675+0000", 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120463 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785209+0000", 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002533455 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788127+0000", 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029181260000000001 2024-09-10T10:28:03.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788155+0000", 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8495000000000001e-05 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788184+0000", 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8736999999999999e-05 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.351 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.546503458000004, 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008097524, 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780164+0000", 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.13e-07 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9780000000000001e-06 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780174+0000", 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7510000000000002e-06 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780179+0000", 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0869999999999997e-06 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782890+0000", 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027113509999999999 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782985+0000", 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5152e-05 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783090+0000", 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104947 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785222+0000", 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021320269999999999 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788216+0000", 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029940629999999999 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788242+0000", 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5259999999999999e-05 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788261+0000", 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9395e-05 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.356 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.546481087000004, 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081619529999999996, 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780189+0000", 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.926e-06 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780197+0000", 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2039999999999997e-06 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780202+0000", 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783326+0000", 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003123855 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783402+0000", 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6548e-05 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783558+0000", 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015539199999999999 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785225+0000", 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001666989 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788290+0000", 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030647890000000001 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788316+0000", 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6662e-05 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788348+0000", 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1417000000000001e-05 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:3 2.10 2:0f65c9b8:::60a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.454887575000001, 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066829990000000002, 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871781+0000", 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2129999999999998e-06 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871790+0000", 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3380000000000005e-06 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871805+0000", 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5353999999999999e-05 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871817+0000", 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1399999999999999e-05 2024-09-10T10:28:03.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871932+0000", 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114982 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872069+0000", 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013713199999999999 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872208+0000", 2024-09-10T10:28:03.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013976900000000001 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874946+0000", 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027378789999999999 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878355+0000", 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034089139999999999 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878413+0000", 2024-09-10T10:28:03.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7540000000000001e-05 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878462+0000", 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9478000000000003e-05 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379444442999997, 2024-09-10T10:28:03.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067332750000000004, 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947223+0000", 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7299999999999995e-07 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947227+0000", 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.895e-06 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947235+0000", 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-09-10T10:28:03.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947246+0000", 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0738e-05 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948505+0000", 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001259402 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948578+0000", 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2788999999999997e-05 2024-09-10T10:28:03.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948718+0000", 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013953699999999999 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953391+0000", 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046736670000000003 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953881+0000", 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048956900000000003 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953923+0000", 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2153000000000001e-05 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953956+0000", 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2526e-05 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.372 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379385655, 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066654160000000004, 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947282+0000", 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947285+0000", 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7689999999999998e-06 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947292+0000", 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6200000000000001e-06 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947304+0000", 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1702000000000001e-05 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947363+0000", 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9426000000000003e-05 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947485+0000", 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012227599999999999 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947588+0000", 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102641 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950690+0000", 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003102264 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953856+0000", 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031656200000000001 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953907+0000", 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1430000000000001e-05 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953947+0000", 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9164e-05 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379348481999997, 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071148679999999999, 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:03.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947319+0000", 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947322+0000", 2024-09-10T10:28:03.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.523e-06 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947331+0000", 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3240000000000003e-06 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947336+0000", 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7189999999999998e-06 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947994+0000", 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065770599999999998 2024-09-10T10:28:03.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948080+0000", 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5649000000000006e-05 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111783 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953965+0000", 2024-09-10T10:28:03.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057732089999999996 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954343+0000", 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037846199999999998 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954395+0000", 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.126e-05 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954433+0000", 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8693999999999999e-05 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.379218285999997, 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067554449999999997, 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.383 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6899999999999996e-07 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947454+0000", 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947463+0000", 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0310000000000004e-06 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947477+0000", 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4328000000000001e-05 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3869999999999998e-05 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947631+0000", 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9874e-05 2024-09-10T10:28:03.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947750+0000", 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118483 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953644+0000", 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058946889999999998 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954111+0000", 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00046712099999999998 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954162+0000", 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000003e-05 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954204+0000", 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1718000000000003e-05 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.388 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.378831247999997, 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064551169999999998, 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3e-07 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947841+0000", 2024-09-10T10:28:03.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3460000000000004e-06 2024-09-10T10:28:03.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947856+0000", 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947864+0000", 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4939999999999997e-06 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947997+0000", 2024-09-10T10:28:03.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013305399999999999 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948076+0000", 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9146000000000004e-05 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115586 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953707+0000", 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055153470000000003 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954215+0000", 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050850400000000001 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954258+0000", 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2932000000000002e-05 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954291+0000", 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2322e-05 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.378791927999998, 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088919089999999999, 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7199999999999999e-07 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947879+0000", 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8380000000000004e-06 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947887+0000", 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5410000000000003e-06 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947893+0000", 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6930000000000004e-06 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948405+0000", 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051236000000000001 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948502+0000", 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6916999999999994e-05 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948647+0000", 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000145265 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954479+0000", 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058316970000000003 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956680+0000", 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002201494 2024-09-10T10:28:03.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956728+0000", 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8161999999999997e-05 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956767+0000", 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.837e-05 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.400 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.378229267000002, 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083279530000000008, 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8999999999999995e-07 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948443+0000", 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2189999999999998e-06 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948450+0000", 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8879999999999997e-06 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948457+0000", 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0750000000000003e-06 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948501+0000", 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948594+0000", 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3009000000000003e-05 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948696+0000", 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101461 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953756+0000", 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050607129999999997 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956690+0000", 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029334109999999999 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956737+0000", 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7080999999999999e-05 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956766+0000", 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8778000000000001e-05 2024-09-10T10:28:03.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.377689592999999, 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0077788589999999999, 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948986+0000", 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948998+0000", 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.19e-05 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949007+0000", 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.081e-06 2024-09-10T10:28:03.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949062+0000", 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754999999999997e-05 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949172+0000", 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110128 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949282+0000", 2024-09-10T10:28:03.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109289 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955941+0000", 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066594990000000001 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956654+0000", 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071279399999999999 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956706+0000", 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2027000000000002e-05 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956756+0000", 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0012999999999997e-05 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.377151499, 2024-09-10T10:28:03.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073200349999999999, 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949516+0000", 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999995e-07 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949522+0000", 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3980000000000004e-06 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949534+0000", 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1562e-05 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949541+0000", 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1380000000000001e-06 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950700+0000", 2024-09-10T10:28:03.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011593459999999999 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950786+0000", 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5556999999999994e-05 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950898+0000", 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111743 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954326+0000", 2024-09-10T10:28:03.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034280510000000001 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956774+0000", 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024477430000000001 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956815+0000", 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1752999999999998e-05 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956835+0000", 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0032999999999999e-05 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.377109197999999, 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073303700000000001, 2024-09-10T10:28:03.417 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949571+0000", 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2298e-05 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949579+0000", 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5909999999999996e-06 2024-09-10T10:28:03.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949585+0000", 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7370000000000003e-06 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953802+0000", 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042170929999999999 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953915+0000", 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011256699999999999 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956888+0000", 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029734639999999999 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:24 2.3 e26/18 2:c216c0d1:::mds2_inotable:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:03.421 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.376291197, 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0063254399999999999, 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950380+0000", 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2590000000000002e-06 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950387+0000", 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950392+0000", 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950438+0000", 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5592e-05 2024-09-10T10:28:03.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950457+0000", 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8569000000000001e-05 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956657+0000", 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062003409999999998 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956701+0000", 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4140000000000001e-05 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:29 2.10 2:0e4da237:::203.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 43.404630394000002, 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064228799999999997, 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922037+0000", 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922040+0000", 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.694e-06 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922054+0000", 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906e-05 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922065+0000", 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922125+0000", 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.982e-05 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922233+0000", 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108175 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928459+0000", 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062264570000000003 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.385366763999997, 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081214100000000008, 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941301+0000", 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.062e-06 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941306+0000", 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0919999999999997e-06 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941316+0000", 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5640000000000002e-06 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941323+0000", 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941405+0000", 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1650000000000006e-05 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941549+0000", 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014447599999999999 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012929 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943549+0000", 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018704049999999999 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949343+0000", 2024-09-10T10:28:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057943300000000003 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949378+0000", 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4474999999999998e-05 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949421+0000", 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3955999999999997e-05 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.436 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:03.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:03.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:03.415+0000 7f544bb8d640 1 -- 172.21.15.120:0/4003382731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f54440c9470 con 0x7f54440c3df0 2024-09-10T10:28:03.438 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:03.436+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:03.537 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:03.537+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:03.585 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:03.585 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:03.594 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:03.638 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:03.637+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:03.700 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:03.701 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:03.701 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:03.701 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:03.701 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:03.709 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:03.751 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:03.750+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:03.841 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:03.840+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:03.864 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.082021823999995, 2024-09-10T10:28:03.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054521709999999996, 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780407+0000", 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.517e-06 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780414+0000", 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3910000000000003e-06 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780429+0000", 2024-09-10T10:28:03.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5379999999999998e-05 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780438+0000", 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9230000000000008e-06 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780885+0000", 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044711300000000001 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780979+0000", 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4312000000000001e-05 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785856+0000", 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048765350000000004 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.081947096999997, 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028177609999999998, 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780479+0000", 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9800000000000003e-07 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780488+0000", 2024-09-10T10:28:03.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3349999999999996e-06 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780512+0000", 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3156999999999999e-05 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780518+0000", 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.764e-06 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780576+0000", 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7813999999999999e-05 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780700+0000", 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123733 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783296+0000", 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025963599999999998 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.872 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:19 2.14 e24/18 2:2f55791f:::606.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.080827936999995, 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003505083, 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9400000000000005e-07 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781606+0000", 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6559999999999992e-06 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781624+0000", 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7575e-05 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781630+0000", 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8630000000000002e-06 2024-09-10T10:28:03.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781683+0000", 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475000000000002e-05 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781700+0000", 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7393000000000001e-05 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785058+0000", 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033575660000000002 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785103+0000", 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:1 2.14 e24/18 2:292cf221:::200.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.080659431000001, 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033666960000000002, 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781767+0000", 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781772+0000", 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5539999999999997e-06 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781781+0000", 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6149999999999997e-06 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781787+0000", 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1589999999999998e-06 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2859000000000002e-05 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781876+0000", 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5916000000000001e-05 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785108+0000", 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003232146 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785133+0000", 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5015999999999999e-05 2024-09-10T10:28:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:23 2.e e24/18 2:717a0223:::608.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.080044895, 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027543580000000002, 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1699999999999998e-07 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782387+0000", 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3229999999999997e-06 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782399+0000", 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2466000000000001e-05 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782406+0000", 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9460000000000001e-06 2024-09-10T10:28:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782468+0000", 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2347999999999993e-05 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6016e-05 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785096+0000", 2024-09-10T10:28:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026118629999999999 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785135+0000", 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8779000000000003e-05 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:24 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.984863189000002, 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062416909999999997, 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877564+0000", 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877576+0000", 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2411e-05 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877592+0000", 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5812e-05 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877600+0000", 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3590000000000002e-06 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877661+0000", 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1749999999999997e-05 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877785+0000", 2024-09-10T10:28:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123807 2024-09-10T10:28:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877936+0000", 2024-09-10T10:28:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015084300000000001 2024-09-10T10:28:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.879317+0000", 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001381415 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883686+0000", 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004368701 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883755+0000", 2024-09-10T10:28:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9140999999999994e-05 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883804+0000", 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8772e-05 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.984037225000002, 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056063609999999998, 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878389+0000", 2024-09-10T10:28:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699999999999999e-07 2024-09-10T10:28:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878394+0000", 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6120000000000001e-06 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878407+0000", 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3704e-05 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878415+0000", 2024-09-10T10:28:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7100000000000007e-06 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878475+0000", 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878603+0000", 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127525 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883995+0000", 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053920460000000002 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.983986858999998, 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057312819999999999, 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5000000000000003e-07 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878442+0000", 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.632e-06 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878450+0000", 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0509999999999997e-06 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878456+0000", 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7200000000000003e-06 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878847+0000", 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039182099999999999 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878949+0000", 2024-09-10T10:28:03.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101458 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.884170+0000", 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052210499999999996 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:03.901 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.915321540999997, 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033173069999999998, 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:03.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947105+0000", 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4670000000000001e-06 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947111+0000", 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6230000000000002e-06 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947123+0000", 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1883e-05 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947132+0000", 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8740000000000001e-06 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947220+0000", 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7800000000000006e-05 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947375+0000", 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015566399999999999 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947501+0000", 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125609 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948393+0000", 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00089165399999999997 2024-09-10T10:28:03.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950323+0000", 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019308769999999999 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950366+0000", 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2688999999999997e-05 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950421+0000", 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5167000000000002e-05 2024-09-10T10:28:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.915265988999998, 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0070341989999999997, 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:03.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947163+0000", 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3440000000000001e-06 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947174+0000", 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0248000000000001e-05 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947180+0000", 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0639999999999999e-06 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947958+0000", 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077789700000000005 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948031+0000", 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3503000000000004e-05 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948151+0000", 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119976 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953625+0000", 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054739360000000004 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954124+0000", 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049858900000000002 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954157+0000", 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3652999999999998e-05 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954194+0000", 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6455000000000002e-05 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.912 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.915174145999998, 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032449549999999999, 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947252+0000", 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000003e-07 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947256+0000", 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3760000000000001e-06 2024-09-10T10:28:03.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947267+0000", 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0259e-05 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947273+0000", 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2190000000000001e-06 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947332+0000", 2024-09-10T10:28:03.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9086000000000003e-05 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947452+0000", 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120146 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950496+0000", 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003044171 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:13 2.1 e26/18 2:81735765:::619.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.914764015000003, 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006029375, 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947662+0000", 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7199999999999997e-07 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:03.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6990000000000003e-06 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947681+0000", 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0682999999999999e-05 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947687+0000", 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7479999999999996e-06 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947742+0000", 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4487000000000002e-05 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947761+0000", 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9497999999999999e-05 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953653+0000", 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058916530000000002 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953691+0000", 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7734999999999997e-05 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:14 2.1 e26/18 2:81735765:::619.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.913443678, 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052525849999999997, 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.921 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5700000000000002e-07 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948988+0000", 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7490000000000004e-06 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949003+0000", 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4214e-05 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949010+0000", 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3499999999999999e-06 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949059+0000", 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8758000000000003e-05 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949078+0000", 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9806e-05 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954198+0000", 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051191509999999997 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954234+0000", 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.68e-05 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.925 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:1 2.1 e26/18 2:85bbe569:::202.00000000:head v 26'4, mlcod=0'0)", 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.913215313999999, 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005049258, 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949211+0000", 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4999999999999998e-07 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949228+0000", 2024-09-10T10:28:03.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7808999999999998e-05 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949240+0000", 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1226000000000001e-05 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949246+0000", 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0689999999999998e-06 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949292+0000", 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6211999999999997e-05 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949311+0000", 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8873e-05 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954236+0000", 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049253980000000001 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954259+0000", 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3221e-05 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:26 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.920353231999997, 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039596270000000003, 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.930 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942075+0000", 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2680000000000001e-06 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942086+0000", 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1361e-05 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942098+0000", 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2221e-05 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942106+0000", 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0960000000000001e-06 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942209+0000", 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102944 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942346+0000", 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136976 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942516+0000", 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017015699999999999 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945195+0000", 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026790920000000001 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945919+0000", 2024-09-10T10:28:03.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00072374500000000003 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945983+0000", 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3418999999999997e-05 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946032+0000", 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9348000000000003e-05 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:2 2.1 e28/18 2:82971f97:::404.00000000:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.918694076000001, 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054707489999999996, 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:03.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943732+0000", 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1999999999999998e-07 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943740+0000", 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1910000000000003e-06 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943755+0000", 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5014000000000001e-05 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943763+0000", 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9519999999999994e-06 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5569999999999997e-05 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943862+0000", 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4150000000000003e-05 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949157+0000", 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052949340000000003 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949202+0000", 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5018000000000002e-05 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:6 2.e e28/18 2:75ee5fd8:::629.00000000:head v 28'5, mlcod=24'2)", 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:03.940 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.918632248000002, 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027501769999999999, 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:03.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943794+0000", 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943799+0000", 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5620000000000003e-06 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943807+0000", 2024-09-10T10:28:03.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.486e-06 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943813+0000", 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6419999999999999e-05 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943888+0000", 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9309000000000001e-05 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946498+0000", 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026094669999999999 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946543+0000", 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:17 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'12, mlcod=26'10)", 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.918588487000001, 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027214219999999998, 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:03.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943849+0000", 2024-09-10T10:28:03.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2023e-05 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943857+0000", 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6009999999999997e-06 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943863+0000", 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2899999999999999e-06 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943927+0000", 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3496000000000003e-05 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943949+0000", 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1801e-05 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946490+0000", 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025409579999999998 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946558+0000", 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8830000000000003e-05 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:18 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'13, mlcod=26'10)", 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.918542285000001, 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027193059999999999, 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.949 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943884+0000", 2024-09-10T10:28:03.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943889+0000", 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.95e-06 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943896+0000", 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3819999999999998e-06 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943901+0000", 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1719999999999998e-06 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944666+0000", 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00076445300000000001 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944686+0000", 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-05 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018748879999999999 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946603+0000", 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1947999999999999e-05 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:25 2.12 e28/18 2:4dad84a4:::mds4_sessionmap:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:03.953 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 37.915628322000003, 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002978994, 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:03.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946798+0000", 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0100000000000001e-06 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946807+0000", 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9700000000000005e-06 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:03.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946820+0000", 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2458e-05 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946856+0000", 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6427999999999997e-05 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946938+0000", 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2106000000000003e-05 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:03.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946958+0000", 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0162000000000001e-05 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949723+0000", 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027645899999999999 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949776+0000", 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3270000000000003e-05 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.958 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:03.958 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:03.958 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:03.959 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:03.959 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:03.958+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:04.060 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:04.059+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:04.074 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:04.074 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:04.074 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:04.074 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:04.083 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:04.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.089+0000 7f7de6ffd640 1 -- 172.21.15.120:0/2953821309 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap-items", "pgid": "4.4", "id": [3, 6, 5, 7, 0, 4, 2, 1]}]=0 set 4.4 pg_upmap_items mapping to [3->6,5->7,0->4,2->1] v42) ==== 168+0+0 (secure 0 0 0) 0x7f7de8022070 con 0x7f7df0152cf0 2024-09-10T10:28:04.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.089+0000 7f543a7fc640 1 -- 172.21.15.120:0/4003382731 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v42) ==== 97+0+0 (secure 0 0 0) 0x7f54400799d0 con 0x7f54440c3df0 2024-09-10T10:28:04.091 INFO:teuthology.orchestra.run.smithi120.stderr:set 4.4 pg_upmap_items mapping to [3->6,5->7,0->4,2->1] 2024-09-10T10:28:04.091 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-10T10:28:04.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 -- 172.21.15.120:0/4003382731 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f542806be30 msgr2=0x7f542806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f542806be30 0x7f542806e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5434007f70 tx=0x7f5434007f00 comp rx=0 tx=0).stop 2024-09-10T10:28:04.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 -- 172.21.15.120:0/4003382731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54440c3df0 msgr2=0x7f54441f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 -- 172.21.15.120:0/2953821309 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7dd406be30 msgr2=0x7f7dd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7dd406be30 0x7f7dd406e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7dec034000 tx=0x7f7dec02e000 comp rx=0 tx=0).stop 2024-09-10T10:28:04.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 -- 172.21.15.120:0/2953821309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df0152cf0 msgr2=0x7f7df00c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df0152cf0 0x7f7df00c6ec0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7de800efc0 tx=0x7f7de800c490 comp rx=0 tx=0).stop 2024-09-10T10:28:04.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54440c3df0 0x7f54441f4100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5440000c00 tx=0x7f54400046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 -- 172.21.15.120:0/4003382731 shutdown_connections 2024-09-10T10:28:04.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f542806be30 0x7f542806e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5444152d60 0x7f54441f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54440c8df0 0x7f54441f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 --2- 172.21.15.120:0/4003382731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54440c3df0 0x7f54441f4100 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 -- 172.21.15.120:0/4003382731 >> 172.21.15.120:0/4003382731 conn(0x7f54440716e0 msgr2=0x7f5444156d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:04.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 -- 172.21.15.120:0/2953821309 shutdown_connections 2024-09-10T10:28:04.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7dd406be30 0x7f7dd406e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7df0152cf0 0x7f7df00c6ec0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df00ca8a0 0x7f7df014cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 --2- 172.21.15.120:0/2953821309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7df00ca4c0 0x7f7df0155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 -- 172.21.15.120:0/2953821309 >> 172.21.15.120:0/2953821309 conn(0x7f7df006c240 msgr2=0x7f7df0070900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:04.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 -- 172.21.15.120:0/2953821309 shutdown_connections 2024-09-10T10:28:04.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 -- 172.21.15.120:0/4003382731 shutdown_connections 2024-09-10T10:28:04.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f7df872a640 1 -- 172.21.15.120:0/2953821309 wait complete. 2024-09-10T10:28:04.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.092+0000 7f544bb8d640 1 -- 172.21.15.120:0/4003382731 wait complete. 2024-09-10T10:28:04.161 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:04.160+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:04.188 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:04.188 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:04.189 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:04.189 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:04.189 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:04.197 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:04.262 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:04.261+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.521578762999994, 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047203150000000001, 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.303 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.304 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779674+0000", 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0359999999999995e-06 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779702+0000", 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.7970000000000002e-05 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779724+0000", 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2544e-05 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.305 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779741+0000", 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6781000000000001e-05 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779981+0000", 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000239764 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780147+0000", 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016605100000000001 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.306 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017096200000000001 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781749+0000", 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001431288 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784234+0000", 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024850940000000002 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784338+0000", 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104078 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784385+0000", 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6746999999999998e-05 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.1a epoch 24/18 rep_tid 2 entries 24'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.520100653, 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021396129999999998, 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.309 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781144+0000", 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.393e-06 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781156+0000", 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1514e-05 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8445999999999998e-05 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781202+0000", 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7439e-05 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781286+0000", 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4652000000000002e-05 2024-09-10T10:28:04.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783282+0000", 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0019961689999999999 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:31 2.d e24/18 2:b50e409b:::mds_snaptable:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.519181161999995, 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030643699999999999, 2024-09-10T10:28:04.312 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.313 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782063+0000", 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3699999999999999e-07 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782072+0000", 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9430000000000006e-06 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782083+0000", 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0998e-05 2024-09-10T10:28:04.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782092+0000", 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.3270000000000001e-06 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782158+0000", 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6830999999999998e-05 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782176+0000", 2024-09-10T10:28:04.315 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7370000000000001e-05 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785091+0000", 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029157879999999999 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785127+0000", 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5176000000000003e-05 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.316 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.519122271000001, 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002013515, 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.317 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782122+0000", 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.229e-06 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.318 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782137+0000", 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4895999999999999e-05 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782151+0000", 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3229e-05 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782198+0000", 2024-09-10T10:28:04.319 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7301e-05 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782271+0000", 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3516000000000006e-05 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782290+0000", 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8921000000000001e-05 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.320 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784094+0000", 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00180331 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784135+0000", 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1112999999999998e-05 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:28 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:04.321 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.519010657999999, 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028487970000000001, 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.322 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7000000000000005e-07 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782247+0000", 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3740000000000001e-05 2024-09-10T10:28:04.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782258+0000", 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0552e-05 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782265+0000", 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0700000000000001e-06 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.324 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782752+0000", 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048707399999999999 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782771+0000", 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9046999999999999e-05 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785037+0000", 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002265806 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785082+0000", 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4938000000000002e-05 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:29 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.517464662999998, 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001932907, 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.326 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.327 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783780+0000", 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9900000000000009e-07 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783789+0000", 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0140000000000004e-06 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783802+0000", 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2751e-05 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783811+0000", 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3950000000000007e-06 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783894+0000", 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2887999999999997e-05 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783913+0000", 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9052999999999999e-05 2024-09-10T10:28:04.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785668+0000", 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001754633 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785712+0000", 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4174000000000001e-05 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.330 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:7 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 55.428832622999998, 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026422899999999998, 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.331 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.561e-06 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872426+0000", 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3577999999999999e-05 2024-09-10T10:28:04.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872462+0000", 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6145e-05 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872473+0000", 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1109e-05 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872543+0000", 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9319000000000005e-05 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872566+0000", 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3768999999999999e-05 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875010+0000", 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024433910000000001 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875053+0000", 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3417999999999997e-05 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:8 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 55.428144234000001, 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0044428820000000004, 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.335 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.336 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873100+0000", 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3030000000000005e-06 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873121+0000", 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1471000000000001e-05 2024-09-10T10:28:04.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873129+0000", 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7339999999999997e-06 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048786499999999998 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873638+0000", 2024-09-10T10:28:04.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0735000000000001e-05 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877493+0000", 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0038550189999999999 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877542+0000", 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9135000000000003e-05 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:17 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 55.427926900999999, 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042663060000000001, 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873318+0000", 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7999999999999993e-07 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873326+0000", 2024-09-10T10:28:04.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8279999999999992e-06 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873341+0000", 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4688999999999999e-05 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873348+0000", 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4810000000000003e-06 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873843+0000", 2024-09-10T10:28:04.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000495083 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873860+0000", 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7609999999999999e-05 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877545+0000", 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036848839999999998 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877583+0000", 2024-09-10T10:28:04.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7750999999999998e-05 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:18 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 55.427204863999997, 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035807780000000002, 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.344 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:04.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874045+0000", 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9909999999999999e-06 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874055+0000", 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0147e-05 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874070+0000", 2024-09-10T10:28:04.346 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4184e-05 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874123+0000", 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3504999999999997e-05 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874141+0000", 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7524999999999999e-05 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.347 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877586+0000", 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034458700000000002 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877619+0000", 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2962000000000001e-05 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.18 epoch 26/18 rep_tid 5 entries 26'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.9:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:04.348 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 55.426143218999997, 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048872359999999997, 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:04.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875101+0000", 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875110+0000", 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2609999999999999e-06 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875129+0000", 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8685000000000002e-05 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875143+0000", 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3723000000000001e-05 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875222+0000", 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9498000000000004e-05 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879987+0000", 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047652800000000002 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:7 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'5, mlcod=26'3)", 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 50.351380921999997, 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031425310000000001, 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.352 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1100000000000005e-07 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949871+0000", 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8620000000000003e-06 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949885+0000", 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4072e-05 2024-09-10T10:28:04.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949893+0000", 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5769999999999996e-06 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949953+0000", 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.978e-05 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949972+0000", 2024-09-10T10:28:04.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9188e-05 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.952959+0000", 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029873859999999999 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953005+0000", 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5855000000000001e-05 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.356 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:8 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'6, mlcod=26'3)", 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 50.351328332999998, 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034236359999999999, 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:04.357 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949916+0000", 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6199999999999998e-07 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.358 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949922+0000", 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5390000000000004e-06 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949929+0000", 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4339999999999997e-06 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949934+0000", 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2159999999999997e-06 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950192+0000", 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00025864699999999998 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950213+0000", 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0761999999999999e-05 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953297+0000", 2024-09-10T10:28:04.360 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030842999999999999 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953339+0000", 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1276000000000003e-05 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:34 2.11 e26/18 2:8975f766:::202.00000001:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 50.347355493999999, 2024-09-10T10:28:04.361 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020619190000000002, 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:04.362 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953889+0000", 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.879999999999999e-07 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953895+0000", 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2249999999999997e-06 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.363 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953906+0000", 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953912+0000", 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4840000000000003e-06 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953959+0000", 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7768000000000001e-05 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.364 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953976+0000", 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6365999999999999e-05 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955917+0000", 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001941524 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955950+0000", 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2573999999999997e-05 2024-09-10T10:28:04.365 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:4 2.1a e26/18 2:5fdc322d:::61e.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.378287966000002, 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0018931340000000001, 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.366 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922956+0000", 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-07 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922962+0000", 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1890000000000004e-06 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922971+0000", 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922978+0000", 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7830000000000001e-06 2024-09-10T10:28:04.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923032+0000", 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3637999999999997e-05 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923049+0000", 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7249999999999999e-05 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.369 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924803+0000", 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001753788 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924849+0000", 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6050000000000001e-05 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:20 2.11 e26/18 2:890f50f7:::626.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:04.370 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.378180260000001, 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003328012, 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:04.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923064+0000", 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9900000000000001e-07 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923070+0000", 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7499999999999997e-06 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.372 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923079+0000", 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9080000000000001e-06 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923089+0000", 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0078000000000001e-05 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923150+0000", 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0300999999999997e-05 2024-09-10T10:28:04.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923507+0000", 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00035713599999999998 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926368+0000", 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028615469999999999 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926391+0000", 2024-09-10T10:28:04.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2793e-05 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:23 2.18 e26/18 2:1c1db442:::103.00000000.inode:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.378120791000001, 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032793050000000002, 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.375 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.376 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923124+0000", 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0599999999999999e-07 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923130+0000", 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.81e-06 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923141+0000", 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.377 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923149+0000", 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2679999999999997e-06 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4744000000000002e-05 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923233+0000", 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9190000000000001e-05 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926353+0000", 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031206419999999999 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926402+0000", 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8755000000000003e-05 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:13 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'6, mlcod=26'2)", 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.378068161000002, 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031818969999999999, 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923176+0000", 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4799999999999998e-07 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923182+0000", 2024-09-10T10:28:04.381 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.592e-06 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923191+0000", 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4029999999999996e-06 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923197+0000", 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1850000000000001e-06 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.382 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923230+0000", 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3290000000000001e-05 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923247+0000", 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6810999999999999e-05 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926308+0000", 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030606159999999999 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.383 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926357+0000", 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9351999999999998e-05 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.d epoch 26/18 rep_tid 20 entries 26'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.15:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.377515662999997, 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034028819999999999, 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.384 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.385 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923729+0000", 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.5900000000000005e-07 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923739+0000", 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8439999999999993e-06 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923761+0000", 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2821999999999999e-05 2024-09-10T10:28:04.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923776+0000", 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4763000000000001e-05 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923889+0000", 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000112518 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927131+0000", 2024-09-10T10:28:04.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032419760000000001 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:14 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'7, mlcod=26'2)", 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 44.376800701999997, 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027184890000000001, 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:04.388 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:04.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5099999999999999e-07 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924449+0000", 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0310000000000004e-06 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924482+0000", 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2845999999999997e-05 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924500+0000", 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.787e-05 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924594+0000", 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3566999999999994e-05 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924619+0000", 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4978999999999999e-05 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927118+0000", 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024995479999999999 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927161+0000", 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2997000000000002e-05 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:04.392 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:04.394 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:04.394 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:04.393+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:04.425 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:28:04.494 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:04.493+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:04.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.521+0000 7fece7074640 1 Processor -- start 2024-09-10T10:28:04.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.521+0000 7fece7074640 1 -- start start 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fece7074640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fece00c78b0 0x7fece015e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fece7074640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece00c7d80 con 0x7fece00c90b0 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fece7074640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece006c670 con 0x7fece00c9490 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fece7074640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fece006d280 con 0x7fece00c78b0 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fece4e10640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fece00c90b0 0x7fece0154a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:53126/0 (socket says 172.21.15.120:53126) 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fece4e10640 1 -- 172.21.15.120:0/2178251257 learned_addr learned my addr 172.21.15.120:0/2178251257 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fecdffff640 1 --2- 172.21.15.120:0/2178251257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fece00c78b0 0x7fece015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fecdf7fe640 1 -- 172.21.15.120:0/2178251257 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 208085785 0 0) 0x7fece00c7d80 con 0x7fece00c90b0 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fecdf7fe640 1 -- 172.21.15.120:0/2178251257 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecd0003520 con 0x7fece00c90b0 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fecdf7fe640 1 -- 172.21.15.120:0/2178251257 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2724427617 0 0) 0x7fece006c670 con 0x7fece00c9490 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fecdf7fe640 1 -- 172.21.15.120:0/2178251257 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fece00c7d80 con 0x7fece00c9490 2024-09-10T10:28:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fecdf7fe640 1 -- 172.21.15.120:0/2178251257 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2017011858 0 0) 0x7fecd0003520 con 0x7fece00c90b0 2024-09-10T10:28:04.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fecdffff640 1 -- 172.21.15.120:0/2178251257 >> v1:172.21.15.134:6789/0 conn(0x7fece00c9490 legacy=0x7fece01653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fecdffff640 1 -- 172.21.15.120:0/2178251257 >> v1:172.21.15.120:6789/0 conn(0x7fece00c90b0 legacy=0x7fece0154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.522+0000 7fecdffff640 1 -- 172.21.15.120:0/2178251257 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece006c950 con 0x7fece00c78b0 2024-09-10T10:28:04.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.523+0000 7fecdffff640 1 --2- 172.21.15.120:0/2178251257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fece00c78b0 0x7fece015e370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feccc0097c0 tx=0x7feccc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1696ffb1ccb88e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.523+0000 7fecdf7fe640 1 -- 172.21.15.120:0/2178251257 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc037070 con 0x7fece00c78b0 2024-09-10T10:28:04.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.523+0000 7fecdf7fe640 1 -- 172.21.15.120:0/2178251257 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feccc02aca0 con 0x7fece00c78b0 2024-09-10T10:28:04.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.523+0000 7fecdf7fe640 1 -- 172.21.15.120:0/2178251257 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4001670 con 0x7fece00c78b0 2024-09-10T10:28:04.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.524+0000 7fece7074640 1 -- 172.21.15.120:0/2178251257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fece00c78b0 msgr2=0x7fece015e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.524+0000 7fece7074640 1 --2- 172.21.15.120:0/2178251257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fece00c78b0 0x7fece015e370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feccc0097c0 tx=0x7feccc02a210 comp rx=0 tx=0).stop 2024-09-10T10:28:04.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.524+0000 7fece7074640 1 -- 172.21.15.120:0/2178251257 shutdown_connections 2024-09-10T10:28:04.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.524+0000 7fece7074640 1 --2- 172.21.15.120:0/2178251257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fece00c78b0 0x7fece015e370 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.524+0000 7fece7074640 1 -- 172.21.15.120:0/2178251257 >> 172.21.15.120:0/2178251257 conn(0x7fece00728b0 msgr2=0x7fece0072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:04.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.524+0000 7fece7074640 1 -- 172.21.15.120:0/2178251257 shutdown_connections 2024-09-10T10:28:04.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.524+0000 7fece7074640 1 -- 172.21.15.120:0/2178251257 wait complete. 2024-09-10T10:28:04.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.525+0000 7fece7074640 1 Processor -- start 2024-09-10T10:28:04.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.525+0000 7fece7074640 1 -- start start 2024-09-10T10:28:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fece7074640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fece00c78b0 0x7fece0156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fece7074640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fece00c90b0 0x7fece0157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fece7074640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fece00c9490 0x7fece0157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fece7074640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fece006bda0 con 0x7fece00c78b0 2024-09-10T10:28:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fece7074640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fece006cf10 con 0x7fece00c9490 2024-09-10T10:28:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fece7074640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fece0155150 con 0x7fece00c90b0 2024-09-10T10:28:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fece4e10640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fece00c78b0 0x7fece0156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fece4e10640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fece00c78b0 0x7fece0156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35034/0 (socket says 172.21.15.120:35034) 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fece4e10640 1 -- 172.21.15.120:0/3388694564 learned_addr learned my addr 172.21.15.120:0/3388694564 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fecdffff640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fece00c90b0 0x7fece0157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.526+0000 7fece5611640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fece00c9490 0x7fece0157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fece4e10640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fece00c90b0 msgr2=0x7fece0157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fece4e10640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fece00c90b0 0x7fece0157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fece4e10640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fece00c9490 msgr2=0x7fece0157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fece4e10640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fece00c9490 0x7fece0157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fece4e10640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece00c3f90 con 0x7fece00c78b0 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fece4e10640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fece00c78b0 0x7fece0156640 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c5d0 tx=0x7fecd400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4017070 con 0x7fece00c78b0 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd4005ce0 con 0x7fece00c78b0 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd40072b0 con 0x7fece00c78b0 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece01555a0 con 0x7fece00c78b0 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.527+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece0155b60 con 0x7fece00c78b0 2024-09-10T10:28:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.528+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fece00c8980 con 0x7fece00c78b0 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.528+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fecd401a070 con 0x7fece00c78b0 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.529+0000 7fecdd7fa640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fecc406be60 0x7fecc406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.529+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6801+0+0 (secure 0 0 0) 0x7fecd408a6c0 con 0x7fece00c78b0 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.529+0000 7fecdffff640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fecc406be60 0x7fecc406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.529+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v42) ==== 54+0+16 (secure 0 0 0) 0x7fecd4090170 con 0x7fece00c78b0 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.529+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fece0156b80 0x7fece0153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.529+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fece00c6f50 con 0x7fece0156b80 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.529+0000 7fecdffff640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fecc406be60 0x7fecc406e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feccc034000 tx=0x7feccc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.529+0000 7fece5611640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fece0156b80 0x7fece0153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.530+0000 7fece5611640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fece0156b80 0x7fece0153830 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.530+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fece00c6f50 con 0x7fece0156b80 2024-09-10T10:28:04.538 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:04.538 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:04.538 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:04.538 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:04.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.544+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fece00008d0 con 0x7fece0156b80 2024-09-10T10:28:04.547 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:04.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.546+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fece00008d0 con 0x7fece0156b80 2024-09-10T10:28:04.548 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:04.548 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:04.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.546+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fece01f3640 0x7fece01f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.546+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fece0154280 con 0x7fece01f3640 2024-09-10T10:28:04.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.546+0000 7fece4e10640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fece01f3640 0x7fece01f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.547+0000 7fece4e10640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fece01f3640 0x7fece01f3a20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.547+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fece0154280 con 0x7fece01f3640 2024-09-10T10:28:04.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.561+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fece01f0b50 con 0x7fece01f3640 2024-09-10T10:28:04.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.563+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fece01f0b50 con 0x7fece01f3640 2024-09-10T10:28:04.564 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:04.565 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:04.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.563+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fece0062630 0x7fece0062a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.563+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fece01f4470 con 0x7fece0062630 2024-09-10T10:28:04.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.563+0000 7fecdffff640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fece0062630 0x7fece0062a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.564+0000 7fecdffff640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fece0062630 0x7fece0062a70 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.565+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fece01f4470 con 0x7fece0062630 2024-09-10T10:28:04.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.579+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fece0063280 con 0x7fece0062630 2024-09-10T10:28:04.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.581+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fece0063280 con 0x7fece0062630 2024-09-10T10:28:04.582 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:04.583 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:04.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.581+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fece00c5370 0x7fece00c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.581+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fece0203b30 con 0x7fece00c5370 2024-09-10T10:28:04.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.581+0000 7fece5611640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fece00c5370 0x7fece00c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.582+0000 7fece5611640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fece00c5370 0x7fece00c5790 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.582+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fece0203b30 con 0x7fece00c5370 2024-09-10T10:28:04.595 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:04.595+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.596+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fece002c890 con 0x7fece00c5370 2024-09-10T10:28:04.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.597+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fece002c890 con 0x7fece00c5370 2024-09-10T10:28:04.599 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:04.599 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:04.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.597+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fece002cfb0 0x7fece002d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.597+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fece002d950 con 0x7fece002cfb0 2024-09-10T10:28:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.598+0000 7fece4e10640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fece002cfb0 0x7fece002d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.598+0000 7fece4e10640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fece002cfb0 0x7fece002d410 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.599+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fece002d950 con 0x7fece002cfb0 2024-09-10T10:28:04.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.612+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fece002d950 con 0x7fece002cfb0 2024-09-10T10:28:04.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.614+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fece002d950 con 0x7fece002cfb0 2024-09-10T10:28:04.615 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:04.616 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:04.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.614+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fece002d950 0x7fece0206ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.614+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fece0209cb0 con 0x7fece002d950 2024-09-10T10:28:04.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.614+0000 7fecdffff640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fece002d950 0x7fece0206ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.614+0000 7fecdffff640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fece002d950 0x7fece0206ee0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.615+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fece0209cb0 con 0x7fece002d950 2024-09-10T10:28:04.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.628+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fece0207420 con 0x7fece002d950 2024-09-10T10:28:04.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.630+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fece0207420 con 0x7fece002d950 2024-09-10T10:28:04.631 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:04.632 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:04.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.630+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fece0207aa0 0x7fece0209e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.630+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fece020cc60 con 0x7fece0207aa0 2024-09-10T10:28:04.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.630+0000 7fece5611640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fece0207aa0 0x7fece0209e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.631+0000 7fece5611640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fece0207aa0 0x7fece0209e90 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.631+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fece020cc60 con 0x7fece0207aa0 2024-09-10T10:28:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.646+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fece020a3d0 con 0x7fece0207aa0 2024-09-10T10:28:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.647+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fece020a3d0 con 0x7fece0207aa0 2024-09-10T10:28:04.649 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.647+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fece020aa50 0x7fece020ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.647+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fece020fc10 con 0x7fece020aa50 2024-09-10T10:28:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.648+0000 7fece4e10640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fece020aa50 0x7fece020ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.648+0000 7fece4e10640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fece020aa50 0x7fece020ce40 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.649+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fece020fc10 con 0x7fece020aa50 2024-09-10T10:28:04.652 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:04.653 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:04.661 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:04.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.662+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fece020d380 con 0x7fece020aa50 2024-09-10T10:28:04.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fecdd7fa640 1 -- 172.21.15.120:0/3388694564 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fece020d380 con 0x7fece020aa50 2024-09-10T10:28:04.665 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:04.666 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fece0156b80 msgr2=0x7fece0153830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fece0156b80 0x7fece0153830 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fece01f3640 msgr2=0x7fece01f3a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fece01f3640 0x7fece01f3a20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fece0062630 msgr2=0x7fece0062a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fece0062630 0x7fece0062a70 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fece00c5370 msgr2=0x7fece00c5790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fece00c5370 0x7fece00c5790 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fece002cfb0 msgr2=0x7fece002d410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fece002cfb0 0x7fece002d410 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.664+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fece002d950 msgr2=0x7fece0206ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fece002d950 0x7fece0206ee0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fece0207aa0 msgr2=0x7fece0209e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fece0207aa0 0x7fece0209e90 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fece020aa50 msgr2=0x7fece020ce40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fece020aa50 0x7fece020ce40 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece5611640 1 -- 172.21.15.120:0/3388694564 reap_dead start 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece5611640 1 -- 172.21.15.120:0/3388694564 reap_dead start 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece5611640 1 -- 172.21.15.120:0/3388694564 reap_dead start 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece5611640 1 -- 172.21.15.120:0/3388694564 reap_dead start 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece5611640 1 -- 172.21.15.120:0/3388694564 reap_dead start 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fecc406be60 msgr2=0x7fecc406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fecc406be60 0x7fecc406e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feccc034000 tx=0x7feccc02e000 comp rx=0 tx=0).stop 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fece00c78b0 msgr2=0x7fece0156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:04.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fece00c78b0 0x7fece0156640 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c5d0 tx=0x7fecd400ca90 comp rx=0 tx=0).stop 2024-09-10T10:28:04.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 shutdown_connections 2024-09-10T10:28:04.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fecc406be60 0x7fecc406e320 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 --2- 172.21.15.120:0/3388694564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fece00c78b0 0x7fece0156640 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:04.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 >> 172.21.15.120:0/3388694564 conn(0x7fece00728b0 msgr2=0x7fece00cc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:04.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 shutdown_connections 2024-09-10T10:28:04.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:04.665+0000 7fece7074640 1 -- 172.21.15.120:0/3388694564 wait complete. 2024-09-10T10:28:04.703 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:04.702+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:04.797 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:04.796+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:04.806 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:04.806 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:04.814 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:04.899 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:04.898+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:04.917 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:04.917 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:04.926 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:04.999 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:04.999+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:05.068 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:05.069 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:05.077 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:05.121 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:05.120+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:05.180 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:05.180 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:05.189 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:05.201 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:05.200+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:05.302 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:05.301+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:05.333 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:05.333 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:05.341 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:05.403 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:05.402+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:05.485 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:05.485 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:05.494 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:05.538 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:05.537+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:05.597 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:05.597 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:05.606 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:05.605+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:05.607 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:05.706 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:05.705+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:05.714 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:05.714 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:05.724 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:05.808 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:05.807+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:05.888 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:05.889 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:05.899 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:05.908 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:05.907+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:06.008 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:06.007+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:06.072 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:06.072 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:06.081 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:06.107 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-10T10:28:06.122 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:06.121+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:06.209 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:06.208+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:06.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.245+0000 7f24b4786640 1 Processor -- start 2024-09-10T10:28:06.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.245+0000 7f24b4786640 1 -- start start 2024-09-10T10:28:06.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b4786640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f24ac154f10 0x7f24ac06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b4786640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f24ac0c8df0 0x7f24ac15e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b4786640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24ac0cf380 con 0x7f24ac0ce380 2024-09-10T10:28:06.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b4786640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f24ac06da70 con 0x7f24ac154f10 2024-09-10T10:28:06.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b4786640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f24ac06dd50 con 0x7f24ac0c8df0 2024-09-10T10:28:06.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b2522640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f24ac154f10 0x7f24ac06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b2522640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f24ac154f10 0x7f24ac06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35044/0 (socket says 172.21.15.120:35044) 2024-09-10T10:28:06.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b2522640 1 -- 172.21.15.120:0/3651484634 learned_addr learned my addr 172.21.15.120:0/3651484634 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:06.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b1d21640 1 --2- 172.21.15.120:0/3651484634 >> v2:172.21.15.134:3300/0 conn(0x7f24ac0c8df0 0x7f24ac15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b2522640 1 -- 172.21.15.120:0/3651484634 >> v2:172.21.15.134:3300/0 conn(0x7f24ac0c8df0 msgr2=0x7f24ac15e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:06.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b2522640 1 --2- 172.21.15.120:0/3651484634 >> v2:172.21.15.134:3300/0 conn(0x7f24ac0c8df0 0x7f24ac15e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b2522640 1 -- 172.21.15.120:0/3651484634 >> v1:172.21.15.120:6789/0 conn(0x7f24ac0ce380 legacy=0x7f24ac06fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:06.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b2522640 1 -- 172.21.15.120:0/3651484634 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac15ae00 con 0x7f24ac154f10 2024-09-10T10:28:06.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.246+0000 7f24b1d21640 1 --2- 172.21.15.120:0/3651484634 >> v2:172.21.15.134:3300/0 conn(0x7f24ac0c8df0 0x7f24ac15e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:06.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.247+0000 7f24b2522640 1 --2- 172.21.15.120:0/3651484634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 0x7f24ac06cd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f24ac06d260 tx=0x7f24a802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab88d3b49c1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.247+0000 7f24b1520640 1 -- 172.21.15.120:0/3651484634 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8037070 con 0x7f24ac154f10 2024-09-10T10:28:06.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.247+0000 7f24b1520640 1 -- 172.21.15.120:0/3651484634 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a802ae50 con 0x7f24ac154f10 2024-09-10T10:28:06.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.247+0000 7f24b1520640 1 -- 172.21.15.120:0/3651484634 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8032860 con 0x7f24ac154f10 2024-09-10T10:28:06.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.248+0000 7f24b4786640 1 -- 172.21.15.120:0/3651484634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 msgr2=0x7f24ac06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:06.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.248+0000 7f24b4786640 1 --2- 172.21.15.120:0/3651484634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 0x7f24ac06cd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f24ac06d260 tx=0x7f24a802a300 comp rx=0 tx=0).stop 2024-09-10T10:28:06.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.248+0000 7f24b4786640 1 -- 172.21.15.120:0/3651484634 shutdown_connections 2024-09-10T10:28:06.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.248+0000 7f24b4786640 1 --2- 172.21.15.120:0/3651484634 >> v2:172.21.15.134:3300/0 conn(0x7f24ac0c8df0 0x7f24ac15e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.248+0000 7f24b4786640 1 --2- 172.21.15.120:0/3651484634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 0x7f24ac06cd20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.248+0000 7f24b4786640 1 -- 172.21.15.120:0/3651484634 >> 172.21.15.120:0/3651484634 conn(0x7f24ac0716e0 msgr2=0x7f24ac071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:06.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.248+0000 7f24b4786640 1 -- 172.21.15.120:0/3651484634 shutdown_connections 2024-09-10T10:28:06.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.248+0000 7f24b4786640 1 -- 172.21.15.120:0/3651484634 wait complete. 2024-09-10T10:28:06.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.249+0000 7f24b4786640 1 Processor -- start 2024-09-10T10:28:06.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.249+0000 7f24b4786640 1 -- start start 2024-09-10T10:28:06.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b4786640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac0c8df0 0x7f24ac1e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b4786640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24ac0ce380 0x7f24ac15e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b4786640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 0x7f24ac1f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b4786640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac0c9210 con 0x7f24ac154f10 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b4786640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac15b3c0 con 0x7f24ac0c8df0 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b4786640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f24ac06d730 con 0x7f24ac0ce380 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b2d23640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 0x7f24ac1f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b2d23640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 0x7f24ac1f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35046/0 (socket says 172.21.15.120:35046) 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b2d23640 1 -- 172.21.15.120:0/826545059 learned_addr learned my addr 172.21.15.120:0/826545059 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b1d21640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24ac0ce380 0x7f24ac15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.250+0000 7f24b2522640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac0c8df0 0x7f24ac1e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.251+0000 7f24b2d23640 1 -- 172.21.15.120:0/826545059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24ac0ce380 msgr2=0x7f24ac15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:06.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.251+0000 7f24b2d23640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24ac0ce380 0x7f24ac15e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.251+0000 7f24b2d23640 1 -- 172.21.15.120:0/826545059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac0c8df0 msgr2=0x7f24ac1e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:06.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.251+0000 7f24b2d23640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac0c8df0 0x7f24ac1e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.251+0000 7f24b2d23640 1 -- 172.21.15.120:0/826545059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac153e20 con 0x7f24ac154f10 2024-09-10T10:28:06.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.251+0000 7f24b2d23640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 0x7f24ac1f3c70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f24a400c740 tx=0x7f24a400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.251+0000 7f24a2ffd640 1 -- 172.21.15.120:0/826545059 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4016020 con 0x7f24ac154f10 2024-09-10T10:28:06.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.251+0000 7f24b4786640 1 -- 172.21.15.120:0/826545059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac1540e0 con 0x7f24ac154f10 2024-09-10T10:28:06.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.251+0000 7f24a2ffd640 1 -- 172.21.15.120:0/826545059 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a400ce50 con 0x7f24ac154f10 2024-09-10T10:28:06.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.251+0000 7f24b4786640 1 -- 172.21.15.120:0/826545059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac15bf90 con 0x7f24ac154f10 2024-09-10T10:28:06.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.252+0000 7f24a2ffd640 1 -- 172.21.15.120:0/826545059 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4003970 con 0x7f24ac154f10 2024-09-10T10:28:06.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.253+0000 7f24b4786640 1 -- 172.21.15.120:0/826545059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ac15aab0 con 0x7f24ac154f10 2024-09-10T10:28:06.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.254+0000 7f24a2ffd640 1 -- 172.21.15.120:0/826545059 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24a4003b10 con 0x7f24ac154f10 2024-09-10T10:28:06.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.254+0000 7f24a2ffd640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f249006be80 0x7f249006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.254+0000 7f24a2ffd640 1 -- 172.21.15.120:0/826545059 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6801+0+0 (secure 0 0 0) 0x7f24a408ac80 con 0x7f24ac154f10 2024-09-10T10:28:06.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.254+0000 7f24b2522640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f249006be80 0x7f249006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.255+0000 7f24b2522640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f249006be80 0x7f249006e340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24a8034000 tx=0x7f24a802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.263+0000 7f24a2ffd640 1 -- 172.21.15.120:0/826545059 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f24a4020020 con 0x7f24ac154f10 2024-09-10T10:28:06.266 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:06.266 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:06.276 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:06.317 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:06.316+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:06.411 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:06.410+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:06.465 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:06.465 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:06.477 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:06.511 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:06.511+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:06.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.512+0000 7f24b4786640 1 -- 172.21.15.120:0/826545059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f24ac0c9470 con 0x7f24ac154f10 2024-09-10T10:28:06.612 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:06.611+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:06.622 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:06.622 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:06.631 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:06.677 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:28:06.713 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:06.712+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:06.796 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:06.796 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:06.809 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:06.813 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:06.812+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:06.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.843+0000 7ffa485e8640 1 Processor -- start 2024-09-10T10:28:06.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.843+0000 7ffa485e8640 1 -- start start 2024-09-10T10:28:06.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa485e8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa400c78b0 0x7ffa4015e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa485e8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ffa400c9490 0x7ffa4015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa46384640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ffa400c90b0 0x7ffa40154a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:53150/0 (socket says 172.21.15.120:53150) 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa46b85640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ffa400c9490 0x7ffa4015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa46b85640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ffa400c9490 0x7ffa4015e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35050/0 (socket says 172.21.15.120:35050) 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa45b83640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa400c78b0 0x7ffa4015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa45b83640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa400c78b0 0x7ffa4015e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:48070/0 (socket says 172.21.15.120:48070) 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa45b83640 1 -- 172.21.15.120:0/1916001383 learned_addr learned my addr 172.21.15.120:0/1916001383 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa485e8640 1 -- 172.21.15.120:0/1916001383 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa400c7d80 con 0x7ffa400c90b0 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa485e8640 1 -- 172.21.15.120:0/1916001383 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ffa4006d280 con 0x7ffa400c9490 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa485e8640 1 -- 172.21.15.120:0/1916001383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4006c670 con 0x7ffa400c78b0 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa45382640 1 -- 172.21.15.120:0/1916001383 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 27409235 0 0) 0x7ffa400c7d80 con 0x7ffa400c90b0 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa45382640 1 -- 172.21.15.120:0/1916001383 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa2c003520 con 0x7ffa400c90b0 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa46b85640 1 -- 172.21.15.120:0/1916001383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa400c78b0 msgr2=0x7ffa4015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa46b85640 1 --2- 172.21.15.120:0/1916001383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa400c78b0 0x7ffa4015e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa46b85640 1 -- 172.21.15.120:0/1916001383 >> v1:172.21.15.120:6789/0 conn(0x7ffa400c90b0 legacy=0x7ffa40154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.845+0000 7ffa46b85640 1 -- 172.21.15.120:0/1916001383 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4006c950 con 0x7ffa400c9490 2024-09-10T10:28:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.846+0000 7ffa46b85640 1 --2- 172.21.15.120:0/1916001383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa400c9490 0x7ffa4015e8b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009d90 tx=0x7ffa3802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad5e9cdb87a5cba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.847+0000 7ffa45382640 1 -- 172.21.15.120:0/1916001383 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38007740 con 0x7ffa400c9490 2024-09-10T10:28:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.847+0000 7ffa45382640 1 -- 172.21.15.120:0/1916001383 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa380078e0 con 0x7ffa400c9490 2024-09-10T10:28:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.847+0000 7ffa45382640 1 -- 172.21.15.120:0/1916001383 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38007cb0 con 0x7ffa400c9490 2024-09-10T10:28:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.847+0000 7ffa485e8640 1 -- 172.21.15.120:0/1916001383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa400c9490 msgr2=0x7ffa4015e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.847+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1916001383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa400c9490 0x7ffa4015e8b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009d90 tx=0x7ffa3802a6f0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.848+0000 7ffa485e8640 1 -- 172.21.15.120:0/1916001383 shutdown_connections 2024-09-10T10:28:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.848+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1916001383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa400c9490 0x7ffa4015e8b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.848+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1916001383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa400c78b0 0x7ffa4015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.848+0000 7ffa485e8640 1 -- 172.21.15.120:0/1916001383 >> 172.21.15.120:0/1916001383 conn(0x7ffa400728b0 msgr2=0x7ffa40072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.848+0000 7ffa485e8640 1 -- 172.21.15.120:0/1916001383 shutdown_connections 2024-09-10T10:28:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.848+0000 7ffa485e8640 1 -- 172.21.15.120:0/1916001383 wait complete. 2024-09-10T10:28:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.849+0000 7ffa485e8640 1 Processor -- start 2024-09-10T10:28:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.849+0000 7ffa485e8640 1 -- start start 2024-09-10T10:28:06.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa485e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa400c78b0 0x7ffa401e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa485e8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa400c90b0 0x7ffa401f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa485e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa400c9490 0x7ffa401f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa485e8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4006bda0 con 0x7ffa400c78b0 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa46384640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa400c78b0 0x7ffa401e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa46384640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa400c78b0 0x7ffa401e44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35056/0 (socket says 172.21.15.120:35056) 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa46384640 1 -- 172.21.15.120:0/1753707347 learned_addr learned my addr 172.21.15.120:0/1753707347 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4006cf10 con 0x7ffa400c9490 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffa40155150 con 0x7ffa400c90b0 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa46b85640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa400c9490 0x7ffa401f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa45b83640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa400c90b0 0x7ffa401f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa46384640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa400c90b0 msgr2=0x7ffa401f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa46384640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa400c90b0 0x7ffa401f2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa46384640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa400c9490 msgr2=0x7ffa401f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa46384640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa400c9490 0x7ffa401f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa46384640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa400bd410 con 0x7ffa400c78b0 2024-09-10T10:28:06.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.851+0000 7ffa46384640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa400c78b0 0x7ffa401e44d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00c460 tx=0x7ffa3c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.852+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c018070 con 0x7ffa400c78b0 2024-09-10T10:28:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.852+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3c0074a0 con 0x7ffa400c78b0 2024-09-10T10:28:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.852+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c004a20 con 0x7ffa400c78b0 2024-09-10T10:28:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.852+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa400bd6b0 con 0x7ffa400c78b0 2024-09-10T10:28:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.852+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa400c3c10 con 0x7ffa400c78b0 2024-09-10T10:28:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.853+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ffa400c8980 con 0x7ffa400c78b0 2024-09-10T10:28:06.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.854+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa3c01b070 con 0x7ffa400c78b0 2024-09-10T10:28:06.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.855+0000 7ffa36ffd640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffa2406be30 0x7ffa2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.855+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6801+0+0 (secure 0 0 0) 0x7ffa3c08a4b0 con 0x7ffa400c78b0 2024-09-10T10:28:06.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.855+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v42) ==== 54+0+16 (secure 0 0 0) 0x7ffa3c007650 con 0x7ffa400c78b0 2024-09-10T10:28:06.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.855+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ffa401f16a0 0x7ffa401f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.855+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa400c8190 con 0x7ffa401f16a0 2024-09-10T10:28:06.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.855+0000 7ffa46b85640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ffa401f16a0 0x7ffa401f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.856+0000 7ffa46b85640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ffa401f16a0 0x7ffa401f1c60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.856+0000 7ffa45b83640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffa2406be30 0x7ffa2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.856+0000 7ffa45b83640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffa2406be30 0x7ffa2406e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa400c75b0 tx=0x7ffa30009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.857+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa400c8190 con 0x7ffa401f16a0 2024-09-10T10:28:06.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.883+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ffa400756f0 con 0x7ffa401f16a0 2024-09-10T10:28:06.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.885+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa400756f0 con 0x7ffa401f16a0 2024-09-10T10:28:06.886 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:06.887 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:06.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.885+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ffa401f37a0 0x7ffa401f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.885+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ffa400c3de0 con 0x7ffa401f37a0 2024-09-10T10:28:06.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.885+0000 7ffa46384640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ffa401f37a0 0x7ffa401f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.886+0000 7ffa46384640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ffa401f37a0 0x7ffa401f3b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.887+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa400c3de0 con 0x7ffa401f37a0 2024-09-10T10:28:06.914 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:06.913+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:06.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.916+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ffa401f4310 con 0x7ffa401f37a0 2024-09-10T10:28:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.918+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa401f4310 con 0x7ffa401f37a0 2024-09-10T10:28:06.919 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.918+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ffa400c2560 0x7ffa40062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.918+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ffa401f4520 con 0x7ffa400c2560 2024-09-10T10:28:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.918+0000 7ffa45b83640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ffa400c2560 0x7ffa40062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.919+0000 7ffa45b83640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ffa400c2560 0x7ffa40062630 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.919+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa401f4520 con 0x7ffa400c2560 2024-09-10T10:28:06.942 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:06.942 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:06.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.943+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ffa40062b70 con 0x7ffa400c2560 2024-09-10T10:28:06.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.946+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa40062b70 con 0x7ffa400c2560 2024-09-10T10:28:06.947 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:06.947 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:06.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.946+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ffa4015bf10 0x7ffa4015c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.946+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ffa4015c890 con 0x7ffa4015bf10 2024-09-10T10:28:06.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.946+0000 7ffa46b85640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ffa4015bf10 0x7ffa4015c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.947+0000 7ffa46b85640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ffa4015bf10 0x7ffa4015c350 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.947+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa4015c890 con 0x7ffa4015bf10 2024-09-10T10:28:06.953 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:06.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.963+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ffa4015cda0 con 0x7ffa4015bf10 2024-09-10T10:28:06.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.965+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa4015cda0 con 0x7ffa4015bf10 2024-09-10T10:28:06.966 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:06.967 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:06.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.965+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ffa4002cb50 0x7ffa4002cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.965+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ffa4002d4b0 con 0x7ffa4002cb50 2024-09-10T10:28:06.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.965+0000 7ffa46384640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ffa4002cb50 0x7ffa4002cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.966+0000 7ffa46384640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ffa4002cb50 0x7ffa4002cf70 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.967+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa4002d4b0 con 0x7ffa4002cb50 2024-09-10T10:28:06.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.983+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ffa4002d4b0 con 0x7ffa4002cb50 2024-09-10T10:28:06.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.985+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa4002d4b0 con 0x7ffa4002cb50 2024-09-10T10:28:06.986 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:06.986 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:06.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.985+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ffa40204b20 0x7ffa40206f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:06.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.985+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ffa4002d9c0 con 0x7ffa40204b20 2024-09-10T10:28:06.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.985+0000 7ffa45b83640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ffa40204b20 0x7ffa40206f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:06.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.986+0000 7ffa45b83640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ffa40204b20 0x7ffa40206f10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:06.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:06.987+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa4002d9c0 con 0x7ffa40204b20 2024-09-10T10:28:07.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.012+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ffa4002d9c0 con 0x7ffa40204b20 2024-09-10T10:28:07.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.014+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa4002d9c0 con 0x7ffa40204b20 2024-09-10T10:28:07.015 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:07.015 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:07.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.014+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ffa40207960 0x7ffa40209d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:07.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.014+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ffa4020cb20 con 0x7ffa40207960 2024-09-10T10:28:07.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.014+0000 7ffa46b85640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ffa40207960 0x7ffa40209d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:07.016 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:07.014+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:07.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.015+0000 7ffa46b85640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ffa40207960 0x7ffa40209d50 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:07.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.016+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa4020cb20 con 0x7ffa40207960 2024-09-10T10:28:07.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.029+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ffa4020a290 con 0x7ffa40207960 2024-09-10T10:28:07.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.031+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa4020a290 con 0x7ffa40207960 2024-09-10T10:28:07.032 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.031+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ffa4020a910 0x7ffa4020ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.031+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7ffa4020feb0 con 0x7ffa4020a910 2024-09-10T10:28:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.031+0000 7ffa46384640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ffa4020a910 0x7ffa4020ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.032+0000 7ffa46384640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ffa4020a910 0x7ffa4020ad70 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.033+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa4020feb0 con 0x7ffa4020a910 2024-09-10T10:28:07.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.046+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ffa4020b3c0 con 0x7ffa4020a910 2024-09-10T10:28:07.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.048+0000 7ffa36ffd640 1 -- 172.21.15.120:0/1753707347 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ffa4020b3c0 con 0x7ffa4020a910 2024-09-10T10:28:07.049 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:07.051 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:07.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.048+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ffa401f16a0 msgr2=0x7ffa401f1c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.048+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ffa401f16a0 0x7ffa401f1c60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.048+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ffa401f37a0 msgr2=0x7ffa401f3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.048+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ffa401f37a0 0x7ffa401f3b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.048+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ffa400c2560 msgr2=0x7ffa40062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.048+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ffa400c2560 0x7ffa40062630 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ffa4015bf10 msgr2=0x7ffa4015c350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ffa4015bf10 0x7ffa4015c350 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ffa4002cb50 msgr2=0x7ffa4002cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa46b85640 1 -- 172.21.15.120:0/1753707347 reap_dead start 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ffa4002cb50 0x7ffa4002cf70 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa46b85640 1 -- 172.21.15.120:0/1753707347 reap_dead start 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ffa40204b20 msgr2=0x7ffa40206f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ffa40204b20 0x7ffa40206f10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ffa40207960 msgr2=0x7ffa40209d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ffa40207960 0x7ffa40209d50 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ffa4020a910 msgr2=0x7ffa4020ad70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ffa4020a910 0x7ffa4020ad70 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffa2406be30 msgr2=0x7ffa2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ffa2406be30 0x7ffa2406e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa400c75b0 tx=0x7ffa30009210 comp rx=0 tx=0).stop 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa400c78b0 msgr2=0x7ffa401e44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 --2- 172.21.15.120:0/1753707347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa400c78b0 0x7ffa401e44d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00c460 tx=0x7ffa3c00c920 comp rx=0 tx=0).stop 2024-09-10T10:28:07.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa46b85640 1 -- 172.21.15.120:0/1753707347 reap_dead start 2024-09-10T10:28:07.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 shutdown_connections 2024-09-10T10:28:07.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 >> 172.21.15.120:0/1753707347 conn(0x7ffa400728b0 msgr2=0x7ffa400cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:07.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 shutdown_connections 2024-09-10T10:28:07.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.049+0000 7ffa485e8640 1 -- 172.21.15.120:0/1753707347 wait complete. 2024-09-10T10:28:07.097 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:07.097 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:07.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.314747975000003, 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078690670000000004, 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:07.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2200000000000004e-07 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780048+0000", 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2427999999999999e-05 2024-09-10T10:28:07.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780057+0000", 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1090000000000004e-06 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780063+0000", 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9780000000000002e-06 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011213480000000001 2024-09-10T10:28:07.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781281+0000", 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6728e-05 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781382+0000", 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101455 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784820+0000", 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034376200000000002 2024-09-10T10:28:07.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787779+0000", 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002959373 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787852+0000", 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3135999999999999e-05 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787894+0000", 2024-09-10T10:28:07.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1369999999999999e-05 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.314700434000002, 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079289789999999992, 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780073+0000", 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:07.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780075+0000", 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5129999999999999e-06 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780082+0000", 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9410000000000001e-06 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780087+0000", 2024-09-10T10:28:07.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1880000000000001e-06 2024-09-10T10:28:07.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781620+0000", 2024-09-10T10:28:07.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001532346 2024-09-10T10:28:07.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781698+0000", 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8232999999999994e-05 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781816+0000", 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011825800000000001 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784824+0000", 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030076349999999998 2024-09-10T10:28:07.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787937+0000", 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031134829999999998 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787969+0000", 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1745999999999997e-05 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788001+0000", 2024-09-10T10:28:07.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2132000000000003e-05 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.314670219000007, 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079888619999999994, 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780103+0000", 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-09-10T10:28:07.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780106+0000", 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1379999999999999e-06 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780115+0000", 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4299999999999995e-06 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780122+0000", 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3380000000000001e-06 2024-09-10T10:28:07.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782041+0000", 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019193700000000001 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782149+0000", 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107493 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782246+0000", 2024-09-10T10:28:07.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999996e-05 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785205+0000", 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002958535 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788041+0000", 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028359650000000002 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788069+0000", 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8123000000000002e-05 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788091+0000", 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2506000000000001e-05 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.314641678000001, 2024-09-10T10:28:07.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080531070000000003, 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780134+0000", 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0570000000000001e-06 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780143+0000", 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-06 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780156+0000", 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3179e-05 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023278280000000001 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782555+0000", 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0494000000000006e-05 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782675+0000", 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120463 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785209+0000", 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002533455 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788127+0000", 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029181260000000001 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788155+0000", 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8495000000000001e-05 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788184+0000", 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8736999999999999e-05 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.314609161000007, 2024-09-10T10:28:07.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008097524, 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780164+0000", 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.13e-07 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9780000000000001e-06 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780174+0000", 2024-09-10T10:28:07.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7510000000000002e-06 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780179+0000", 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0869999999999997e-06 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782890+0000", 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027113509999999999 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782985+0000", 2024-09-10T10:28:07.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5152e-05 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783090+0000", 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104947 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785222+0000", 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021320269999999999 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788216+0000", 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029940629999999999 2024-09-10T10:28:07.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788242+0000", 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5259999999999999e-05 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788261+0000", 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9395e-05 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.124 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.314586790000007, 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081619529999999996, 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780189+0000", 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.926e-06 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780197+0000", 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2039999999999997e-06 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780202+0000", 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783326+0000", 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003123855 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783402+0000", 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6548e-05 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783558+0000", 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015539199999999999 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785225+0000", 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001666989 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788290+0000", 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030647890000000001 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788316+0000", 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6662e-05 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788348+0000", 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1417000000000001e-05 2024-09-10T10:28:07.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:3 2.10 2:0f65c9b8:::60a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.222993277999997, 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066829990000000002, 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871781+0000", 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2129999999999998e-06 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871790+0000", 2024-09-10T10:28:07.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3380000000000005e-06 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871805+0000", 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5353999999999999e-05 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871817+0000", 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1399999999999999e-05 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871932+0000", 2024-09-10T10:28:07.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114982 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872069+0000", 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013713199999999999 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872208+0000", 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013976900000000001 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874946+0000", 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027378789999999999 2024-09-10T10:28:07.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878355+0000", 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034089139999999999 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878413+0000", 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7540000000000001e-05 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878462+0000", 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9478000000000003e-05 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.134 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.147550146, 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067332750000000004, 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:07.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947223+0000", 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7299999999999995e-07 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947227+0000", 2024-09-10T10:28:07.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.895e-06 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947235+0000", 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947246+0000", 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0738e-05 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948505+0000", 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001259402 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948578+0000", 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2788999999999997e-05 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948718+0000", 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013953699999999999 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953391+0000", 2024-09-10T10:28:07.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046736670000000003 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953881+0000", 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048956900000000003 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953923+0000", 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2153000000000001e-05 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953956+0000", 2024-09-10T10:28:07.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2526e-05 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.147491358000003, 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066654160000000004, 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947282+0000", 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947285+0000", 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7689999999999998e-06 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947292+0000", 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6200000000000001e-06 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947304+0000", 2024-09-10T10:28:07.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1702000000000001e-05 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947363+0000", 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9426000000000003e-05 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947485+0000", 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012227599999999999 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947588+0000", 2024-09-10T10:28:07.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102641 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950690+0000", 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003102264 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953856+0000", 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031656200000000001 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953907+0000", 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1430000000000001e-05 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953947+0000", 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9164e-05 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:07.145 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.147454185000001, 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071148679999999999, 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947319+0000", 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947322+0000", 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.523e-06 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947331+0000", 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3240000000000003e-06 2024-09-10T10:28:07.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947336+0000", 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7189999999999998e-06 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947994+0000", 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065770599999999998 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948080+0000", 2024-09-10T10:28:07.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5649000000000006e-05 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111783 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953965+0000", 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057732089999999996 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954343+0000", 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037846199999999998 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954395+0000", 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.126e-05 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954433+0000", 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8693999999999999e-05 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.147323989, 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067554449999999997, 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6899999999999996e-07 2024-09-10T10:28:07.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947454+0000", 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947463+0000", 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0310000000000004e-06 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947477+0000", 2024-09-10T10:28:07.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4328000000000001e-05 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3869999999999998e-05 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947631+0000", 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9874e-05 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947750+0000", 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118483 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953644+0000", 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058946889999999998 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954111+0000", 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00046712099999999998 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954162+0000", 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000003e-05 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954204+0000", 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1718000000000003e-05 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.156 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.146936951000001, 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064551169999999998, 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3e-07 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947841+0000", 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3460000000000004e-06 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947856+0000", 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947864+0000", 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4939999999999997e-06 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947997+0000", 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013305399999999999 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948076+0000", 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9146000000000004e-05 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115586 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953707+0000", 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055153470000000003 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954215+0000", 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050850400000000001 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954258+0000", 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2932000000000002e-05 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954291+0000", 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2322e-05 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.146897631000002, 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088919089999999999, 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7199999999999999e-07 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947879+0000", 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8380000000000004e-06 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947887+0000", 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5410000000000003e-06 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947893+0000", 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6930000000000004e-06 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948405+0000", 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051236000000000001 2024-09-10T10:28:07.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948502+0000", 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6916999999999994e-05 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948647+0000", 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000145265 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954479+0000", 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058316970000000003 2024-09-10T10:28:07.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956680+0000", 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002201494 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956728+0000", 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8161999999999997e-05 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956767+0000", 2024-09-10T10:28:07.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.837e-05 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.146334969999998, 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083279530000000008, 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8999999999999995e-07 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948443+0000", 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2189999999999998e-06 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948450+0000", 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8879999999999997e-06 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948457+0000", 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0750000000000003e-06 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948501+0000", 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948594+0000", 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3009000000000003e-05 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948696+0000", 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101461 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953756+0000", 2024-09-10T10:28:07.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050607129999999997 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956690+0000", 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029334109999999999 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956737+0000", 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7080999999999999e-05 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956766+0000", 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8778000000000001e-05 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.145795296000003, 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0077788589999999999, 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.172 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-10T10:28:07.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948986+0000", 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948998+0000", 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.19e-05 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949007+0000", 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.081e-06 2024-09-10T10:28:07.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949062+0000", 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754999999999997e-05 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949172+0000", 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110128 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949282+0000", 2024-09-10T10:28:07.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109289 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955941+0000", 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066594990000000001 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956654+0000", 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071279399999999999 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956706+0000", 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2027000000000002e-05 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956756+0000", 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0012999999999997e-05 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:07.177 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.145257202000003, 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073200349999999999, 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:07.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949516+0000", 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999995e-07 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949522+0000", 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3980000000000004e-06 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949534+0000", 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1562e-05 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949541+0000", 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1380000000000001e-06 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950700+0000", 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011593459999999999 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950786+0000", 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5556999999999994e-05 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950898+0000", 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111743 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954326+0000", 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034280510000000001 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956774+0000", 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024477430000000001 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956815+0000", 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1752999999999998e-05 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956835+0000", 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0032999999999999e-05 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.145214901000003, 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073303700000000001, 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:07.183 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949571+0000", 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2298e-05 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949579+0000", 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5909999999999996e-06 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949585+0000", 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7370000000000003e-06 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953802+0000", 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042170929999999999 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953915+0000", 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011256699999999999 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956888+0000", 2024-09-10T10:28:07.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029734639999999999 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:24 2.3 e26/18 2:c216c0d1:::mds2_inotable:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.144396899999997, 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0063254399999999999, 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950380+0000", 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2590000000000002e-06 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950387+0000", 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950392+0000", 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950438+0000", 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5592e-05 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950457+0000", 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8569000000000001e-05 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956657+0000", 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062003409999999998 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956701+0000", 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4140000000000001e-05 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:29 2.10 2:0e4da237:::203.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 47.172736096999998, 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064228799999999997, 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.191 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922037+0000", 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922040+0000", 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.694e-06 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922054+0000", 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906e-05 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922065+0000", 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922125+0000", 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.982e-05 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922233+0000", 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108175 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928459+0000", 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062264570000000003 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.153472467, 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081214100000000008, 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.195 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941301+0000", 2024-09-10T10:28:07.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.062e-06 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941306+0000", 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0919999999999997e-06 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941316+0000", 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5640000000000002e-06 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941323+0000", 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941405+0000", 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1650000000000006e-05 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941549+0000", 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014447599999999999 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:07.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012929 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943549+0000", 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018704049999999999 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949343+0000", 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057943300000000003 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949378+0000", 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4474999999999998e-05 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949421+0000", 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3955999999999997e-05 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.200 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:07.202 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:07.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.163+0000 7f24a2ffd640 1 -- 172.21.15.120:0/826545059 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v43) ==== 91+0+0 (secure 0 0 0) 0x7f24ac0cf380 con 0x7f24ac154f10 2024-09-10T10:28:07.203 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-10T10:28:07.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 -- 172.21.15.120:0/826545059 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f249006be80 msgr2=0x7f249006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f249006be80 0x7f249006e340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24a8034000 tx=0x7f24a802e000 comp rx=0 tx=0).stop 2024-09-10T10:28:07.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 -- 172.21.15.120:0/826545059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 msgr2=0x7f24ac1f3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:07.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 0x7f24ac1f3c70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f24a400c740 tx=0x7f24a400cc00 comp rx=0 tx=0).stop 2024-09-10T10:28:07.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 -- 172.21.15.120:0/826545059 shutdown_connections 2024-09-10T10:28:07.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f249006be80 0x7f249006e340 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24ac154f10 0x7f24ac1f3c70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24ac0ce380 0x7f24ac15e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 --2- 172.21.15.120:0/826545059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac0c8df0 0x7f24ac1e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:07.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 -- 172.21.15.120:0/826545059 >> 172.21.15.120:0/826545059 conn(0x7f24ac0716e0 msgr2=0x7f24ac0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:07.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 -- 172.21.15.120:0/826545059 shutdown_connections 2024-09-10T10:28:07.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:07.167+0000 7f24b4786640 1 -- 172.21.15.120:0/826545059 wait complete. 2024-09-10T10:28:07.204 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:07.200+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:07.302 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:07.301+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:07.352 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:07.353 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:07.353 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:07.353 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:07.364 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:07.405 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:07.404+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:07.505 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:07.504+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:07.523 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:07.523 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:07.523 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:07.523 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:07.523 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:07.535 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:07.606 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:07.605+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:07.694 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:07.694 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.911794950000001, 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054521709999999996, 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780407+0000", 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.517e-06 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780414+0000", 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3910000000000003e-06 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780429+0000", 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5379999999999998e-05 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780438+0000", 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9230000000000008e-06 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780885+0000", 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044711300000000001 2024-09-10T10:28:07.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780979+0000", 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4312000000000001e-05 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785856+0000", 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048765350000000004 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:07.698 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.911720223000003, 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028177609999999998, 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780479+0000", 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9800000000000003e-07 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780488+0000", 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3349999999999996e-06 2024-09-10T10:28:07.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780512+0000", 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3156999999999999e-05 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780518+0000", 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.764e-06 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780576+0000", 2024-09-10T10:28:07.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7813999999999999e-05 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780700+0000", 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123733 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783296+0000", 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025963599999999998 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:19 2.14 e24/18 2:2f55791f:::606.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.910601063000001, 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003505083, 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9400000000000005e-07 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781606+0000", 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6559999999999992e-06 2024-09-10T10:28:07.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781624+0000", 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7575e-05 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781630+0000", 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8630000000000002e-06 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781683+0000", 2024-09-10T10:28:07.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475000000000002e-05 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781700+0000", 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7393000000000001e-05 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785058+0000", 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033575660000000002 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785103+0000", 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:1 2.14 e24/18 2:292cf221:::200.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.910432557000007, 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033666960000000002, 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781767+0000", 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781772+0000", 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5539999999999997e-06 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781781+0000", 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6149999999999997e-06 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781787+0000", 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1589999999999998e-06 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2859000000000002e-05 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781876+0000", 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5916000000000001e-05 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785108+0000", 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003232146 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785133+0000", 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5015999999999999e-05 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.711 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:23 2.e e24/18 2:717a0223:::608.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.909818021000007, 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027543580000000002, 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1699999999999998e-07 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782387+0000", 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3229999999999997e-06 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782399+0000", 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2466000000000001e-05 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782406+0000", 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9460000000000001e-06 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782468+0000", 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2347999999999993e-05 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6016e-05 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785096+0000", 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026118629999999999 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785135+0000", 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8779000000000003e-05 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:24 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.814636315000001, 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062416909999999997, 2024-09-10T10:28:07.716 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877564+0000", 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877576+0000", 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2411e-05 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877592+0000", 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5812e-05 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877600+0000", 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3590000000000002e-06 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877661+0000", 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1749999999999997e-05 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877785+0000", 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123807 2024-09-10T10:28:07.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877936+0000", 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015084300000000001 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.879317+0000", 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001381415 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883686+0000", 2024-09-10T10:28:07.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004368701 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883755+0000", 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9140999999999994e-05 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883804+0000", 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8772e-05 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.813810351000001, 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056063609999999998, 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878389+0000", 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699999999999999e-07 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878394+0000", 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6120000000000001e-06 2024-09-10T10:28:07.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878407+0000", 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3704e-05 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878415+0000", 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7100000000000007e-06 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878475+0000", 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878603+0000", 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127525 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883995+0000", 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053920460000000002 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.725 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.813759984999997, 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057312819999999999, 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5000000000000003e-07 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878442+0000", 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.632e-06 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878450+0000", 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0509999999999997e-06 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878456+0000", 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7200000000000003e-06 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878847+0000", 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039182099999999999 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878949+0000", 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101458 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.884170+0000", 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052210499999999996 2024-09-10T10:28:07.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.745094667000004, 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033173069999999998, 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947105+0000", 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4670000000000001e-06 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947111+0000", 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6230000000000002e-06 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947123+0000", 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1883e-05 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947132+0000", 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8740000000000001e-06 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947220+0000", 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7800000000000006e-05 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947375+0000", 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015566399999999999 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947501+0000", 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125609 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948393+0000", 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00089165399999999997 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950323+0000", 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019308769999999999 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950366+0000", 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2688999999999997e-05 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950421+0000", 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5167000000000002e-05 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.745039114999997, 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0070341989999999997, 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947163+0000", 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3440000000000001e-06 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947174+0000", 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0248000000000001e-05 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947180+0000", 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0639999999999999e-06 2024-09-10T10:28:07.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947958+0000", 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077789700000000005 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948031+0000", 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3503000000000004e-05 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:07.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948151+0000", 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119976 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953625+0000", 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054739360000000004 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954124+0000", 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049858900000000002 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954157+0000", 2024-09-10T10:28:07.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3652999999999998e-05 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954194+0000", 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6455000000000002e-05 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.744947271999997, 2024-09-10T10:28:07.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032449549999999999, 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947252+0000", 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000003e-07 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947256+0000", 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3760000000000001e-06 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947267+0000", 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0259e-05 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947273+0000", 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2190000000000001e-06 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947332+0000", 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9086000000000003e-05 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947452+0000", 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120146 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950496+0000", 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003044171 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:13 2.1 e26/18 2:81735765:::619.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.744537141000002, 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006029375, 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.744 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947662+0000", 2024-09-10T10:28:07.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7199999999999997e-07 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6990000000000003e-06 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947681+0000", 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0682999999999999e-05 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947687+0000", 2024-09-10T10:28:07.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7479999999999996e-06 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947742+0000", 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4487000000000002e-05 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947761+0000", 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9497999999999999e-05 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953653+0000", 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058916530000000002 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953691+0000", 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7734999999999997e-05 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:14 2.1 e26/18 2:81735765:::619.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:07.748 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.743216803999999, 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052525849999999997, 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:07.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5700000000000002e-07 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948988+0000", 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7490000000000004e-06 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949003+0000", 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4214e-05 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949010+0000", 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3499999999999999e-06 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949059+0000", 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8758000000000003e-05 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949078+0000", 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9806e-05 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954198+0000", 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051191509999999997 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954234+0000", 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.68e-05 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.752 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:1 2.1 e26/18 2:85bbe569:::202.00000000:head v 26'4, mlcod=0'0)", 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 53.742988439999998, 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005049258, 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949211+0000", 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4999999999999998e-07 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949228+0000", 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7808999999999998e-05 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949240+0000", 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1226000000000001e-05 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949246+0000", 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0689999999999998e-06 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949292+0000", 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6211999999999997e-05 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949311+0000", 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8873e-05 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954236+0000", 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049253980000000001 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954259+0000", 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3221e-05 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:26 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.750126358000003, 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039596270000000003, 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.757 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942075+0000", 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2680000000000001e-06 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942086+0000", 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1361e-05 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942098+0000", 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2221e-05 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942106+0000", 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0960000000000001e-06 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942209+0000", 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102944 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942346+0000", 2024-09-10T10:28:07.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136976 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942516+0000", 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017015699999999999 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945195+0000", 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026790920000000001 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:07.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945919+0000", 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00072374500000000003 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945983+0000", 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3418999999999997e-05 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946032+0000", 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9348000000000003e-05 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:2 2.1 e28/18 2:82971f97:::404.00000000:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.748467202, 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054707489999999996, 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943732+0000", 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1999999999999998e-07 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943740+0000", 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1910000000000003e-06 2024-09-10T10:28:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943755+0000", 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5014000000000001e-05 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943763+0000", 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9519999999999994e-06 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:07.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5569999999999997e-05 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943862+0000", 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4150000000000003e-05 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949157+0000", 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052949340000000003 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949202+0000", 2024-09-10T10:28:07.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5018000000000002e-05 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:6 2.e e28/18 2:75ee5fd8:::629.00000000:head v 28'5, mlcod=24'2)", 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.748405374000001, 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027501769999999999, 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943794+0000", 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943799+0000", 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5620000000000003e-06 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943807+0000", 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.486e-06 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943813+0000", 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-10T10:28:07.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6419999999999999e-05 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943888+0000", 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9309000000000001e-05 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946498+0000", 2024-09-10T10:28:07.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026094669999999999 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946543+0000", 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:17 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'12, mlcod=26'10)", 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.748361613, 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027214219999999998, 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.771 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:07.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943849+0000", 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2023e-05 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943857+0000", 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6009999999999997e-06 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943863+0000", 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2899999999999999e-06 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943927+0000", 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3496000000000003e-05 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943949+0000", 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1801e-05 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946490+0000", 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025409579999999998 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946558+0000", 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8830000000000003e-05 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:18 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'13, mlcod=26'10)", 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.748315411, 2024-09-10T10:28:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027193059999999999, 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943884+0000", 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943889+0000", 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.95e-06 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943896+0000", 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3819999999999998e-06 2024-09-10T10:28:07.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943901+0000", 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1719999999999998e-06 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944666+0000", 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00076445300000000001 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944686+0000", 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-05 2024-09-10T10:28:07.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018748879999999999 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946603+0000", 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1947999999999999e-05 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.779 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:25 2.12 e28/18 2:4dad84a4:::mds4_sessionmap:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 41.745401448000003, 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002978994, 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946798+0000", 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0100000000000001e-06 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946807+0000", 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9700000000000005e-06 2024-09-10T10:28:07.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946820+0000", 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2458e-05 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946856+0000", 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6427999999999997e-05 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946938+0000", 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2106000000000003e-05 2024-09-10T10:28:07.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946958+0000", 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0162000000000001e-05 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949723+0000", 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027645899999999999 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949776+0000", 2024-09-10T10:28:07.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3270000000000003e-05 2024-09-10T10:28:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:07.784 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:07.785 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:07.786 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:07.784+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:07.886 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:07.885+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:07.902 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:07.902 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:07.902 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:07.902 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:07.911 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:07.986 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:07.986+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:08.055 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:08.055 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:08.055 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:08.055 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:08.056 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:08.064 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:08.087 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:08.086+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:08.170 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:08.170 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:08.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:08.170 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:08.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.170 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:08.170 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.388698448, 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047203150000000001, 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.171 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779674+0000", 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0359999999999995e-06 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779702+0000", 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.7970000000000002e-05 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779724+0000", 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2544e-05 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779741+0000", 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6781000000000001e-05 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779981+0000", 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000239764 2024-09-10T10:28:08.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780147+0000", 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016605100000000001 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017096200000000001 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781749+0000", 2024-09-10T10:28:08.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001431288 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784234+0000", 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024850940000000002 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784338+0000", 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104078 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784385+0000", 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6746999999999998e-05 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.1a epoch 24/18 rep_tid 2 entries 24'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.387220338000006, 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021396129999999998, 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781144+0000", 2024-09-10T10:28:08.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.393e-06 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781156+0000", 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1514e-05 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8445999999999998e-05 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781202+0000", 2024-09-10T10:28:08.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7439e-05 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781286+0000", 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4652000000000002e-05 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783282+0000", 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0019961689999999999 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:31 2.d e24/18 2:b50e409b:::mds_snaptable:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.386300847000001, 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030643699999999999, 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782063+0000", 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3699999999999999e-07 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782072+0000", 2024-09-10T10:28:08.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9430000000000006e-06 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782083+0000", 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0998e-05 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782092+0000", 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.3270000000000001e-06 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.182 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782158+0000", 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6830999999999998e-05 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782176+0000", 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7370000000000001e-05 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785091+0000", 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029157879999999999 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.183 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785127+0000", 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5176000000000003e-05 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.386241956000006, 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002013515, 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.184 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782122+0000", 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.229e-06 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782137+0000", 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4895999999999999e-05 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782151+0000", 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3229e-05 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782198+0000", 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7301e-05 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782271+0000", 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3516000000000006e-05 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782290+0000", 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8921000000000001e-05 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784094+0000", 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00180331 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784135+0000", 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1112999999999998e-05 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.188 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:28 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.386130343000005, 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028487970000000001, 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7000000000000005e-07 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782247+0000", 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3740000000000001e-05 2024-09-10T10:28:08.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782258+0000", 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0552e-05 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782265+0000", 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0700000000000001e-06 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782752+0000", 2024-09-10T10:28:08.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048707399999999999 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782771+0000", 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9046999999999999e-05 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785037+0000", 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002265806 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.192 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785082+0000", 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4938000000000002e-05 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:29 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.384584348000004, 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001932907, 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.193 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783780+0000", 2024-09-10T10:28:08.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9900000000000009e-07 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783789+0000", 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0140000000000004e-06 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783802+0000", 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2751e-05 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.195 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783811+0000", 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3950000000000007e-06 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783894+0000", 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2887999999999997e-05 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783913+0000", 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9052999999999999e-05 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.196 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785668+0000", 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001754633 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785712+0000", 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4174000000000001e-05 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:7 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.295952307999997, 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026422899999999998, 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:08.198 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.561e-06 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872426+0000", 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3577999999999999e-05 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.199 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872462+0000", 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6145e-05 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872473+0000", 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1109e-05 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872543+0000", 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9319000000000005e-05 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872566+0000", 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3768999999999999e-05 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875010+0000", 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024433910000000001 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875053+0000", 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3417999999999997e-05 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.201 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:8 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.295263919, 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0044428820000000004, 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:08.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873100+0000", 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.203 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3030000000000005e-06 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873121+0000", 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1471000000000001e-05 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873129+0000", 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7339999999999997e-06 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.204 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048786499999999998 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873638+0000", 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0735000000000001e-05 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877493+0000", 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0038550189999999999 2024-09-10T10:28:08.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877542+0000", 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9135000000000003e-05 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:17 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.295046585999998, 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042663060000000001, 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.206 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.207 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873318+0000", 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7999999999999993e-07 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873326+0000", 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8279999999999992e-06 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873341+0000", 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4688999999999999e-05 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873348+0000", 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4810000000000003e-06 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873843+0000", 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000495083 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873860+0000", 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7609999999999999e-05 2024-09-10T10:28:08.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.210 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.210 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877545+0000", 2024-09-10T10:28:08.210 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036848839999999998 2024-09-10T10:28:08.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.210 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.210 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877583+0000", 2024-09-10T10:28:08.210 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7750999999999998e-05 2024-09-10T10:28:08.210 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:18 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.294324549000002, 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035807780000000002, 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.211 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874045+0000", 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9909999999999999e-06 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874055+0000", 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0147e-05 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874070+0000", 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4184e-05 2024-09-10T10:28:08.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874123+0000", 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3504999999999997e-05 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874141+0000", 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7524999999999999e-05 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877586+0000", 2024-09-10T10:28:08.214 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034458700000000002 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877619+0000", 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2962000000000001e-05 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.18 epoch 26/18 rep_tid 5 entries 26'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.9:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.293262904000002, 2024-09-10T10:28:08.215 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048872359999999997, 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875101+0000", 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875110+0000", 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2609999999999999e-06 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875129+0000", 2024-09-10T10:28:08.217 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8685000000000002e-05 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875143+0000", 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3723000000000001e-05 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875222+0000", 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9498000000000004e-05 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879987+0000", 2024-09-10T10:28:08.218 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047652800000000002 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:7 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'5, mlcod=26'3)", 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 54.218500607000003, 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031425310000000001, 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1100000000000005e-07 2024-09-10T10:28:08.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949871+0000", 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8620000000000003e-06 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949885+0000", 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4072e-05 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949893+0000", 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5769999999999996e-06 2024-09-10T10:28:08.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949953+0000", 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.978e-05 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949972+0000", 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9188e-05 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.952959+0000", 2024-09-10T10:28:08.222 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029873859999999999 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953005+0000", 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5855000000000001e-05 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:8 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'6, mlcod=26'3)", 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 54.218448017999997, 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034236359999999999, 2024-09-10T10:28:08.223 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949916+0000", 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6199999999999998e-07 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949922+0000", 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5390000000000004e-06 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949929+0000", 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4339999999999997e-06 2024-09-10T10:28:08.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949934+0000", 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2159999999999997e-06 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950192+0000", 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00025864699999999998 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950213+0000", 2024-09-10T10:28:08.226 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0761999999999999e-05 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953297+0000", 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030842999999999999 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953339+0000", 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1276000000000003e-05 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:34 2.11 e26/18 2:8975f766:::202.00000001:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 54.214475178999997, 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020619190000000002, 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953889+0000", 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.879999999999999e-07 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953895+0000", 2024-09-10T10:28:08.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2249999999999997e-06 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953906+0000", 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953912+0000", 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4840000000000003e-06 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.230 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953959+0000", 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7768000000000001e-05 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953976+0000", 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6365999999999999e-05 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955917+0000", 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001941524 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.231 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955950+0000", 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2573999999999997e-05 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:4 2.1a e26/18 2:5fdc322d:::61e.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.245407651000001, 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0018931340000000001, 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.232 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922956+0000", 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-07 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922962+0000", 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1890000000000004e-06 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922971+0000", 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922978+0000", 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7830000000000001e-06 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923032+0000", 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3637999999999997e-05 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923049+0000", 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7249999999999999e-05 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.235 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924803+0000", 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001753788 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924849+0000", 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6050000000000001e-05 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:20 2.11 e26/18 2:890f50f7:::626.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:08.236 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.245299944999999, 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003328012, 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923064+0000", 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9900000000000001e-07 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923070+0000", 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7499999999999997e-06 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923079+0000", 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9080000000000001e-06 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923089+0000", 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0078000000000001e-05 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923150+0000", 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0300999999999997e-05 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923507+0000", 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00035713599999999998 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926368+0000", 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028615469999999999 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926391+0000", 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2793e-05 2024-09-10T10:28:08.240 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:23 2.18 e26/18 2:1c1db442:::103.00000000.inode:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.245240475999999, 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032793050000000002, 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.241 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923124+0000", 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0599999999999999e-07 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923130+0000", 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.81e-06 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923141+0000", 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923149+0000", 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2679999999999997e-06 2024-09-10T10:28:08.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4744000000000002e-05 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923233+0000", 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9190000000000001e-05 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926353+0000", 2024-09-10T10:28:08.244 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031206419999999999 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926402+0000", 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8755000000000003e-05 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:13 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'6, mlcod=26'2)", 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.245187846, 2024-09-10T10:28:08.245 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031818969999999999, 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923176+0000", 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4799999999999998e-07 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923182+0000", 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.592e-06 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923191+0000", 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4029999999999996e-06 2024-09-10T10:28:08.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923197+0000", 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1850000000000001e-06 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923230+0000", 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3290000000000001e-05 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923247+0000", 2024-09-10T10:28:08.248 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6810999999999999e-05 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926308+0000", 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030606159999999999 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926357+0000", 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9351999999999998e-05 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.d epoch 26/18 rep_tid 20 entries 26'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.15:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.244635348000003, 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034028819999999999, 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923729+0000", 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.5900000000000005e-07 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923739+0000", 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8439999999999993e-06 2024-09-10T10:28:08.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923761+0000", 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2821999999999999e-05 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923776+0000", 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4763000000000001e-05 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.252 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923889+0000", 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000112518 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927131+0000", 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032419760000000001 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:14 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'7, mlcod=26'2)", 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.243920387000003, 2024-09-10T10:28:08.253 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027184890000000001, 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:08.254 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5099999999999999e-07 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924449+0000", 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0310000000000004e-06 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.255 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924482+0000", 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2845999999999997e-05 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924500+0000", 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.787e-05 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924594+0000", 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3566999999999994e-05 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924619+0000", 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4978999999999999e-05 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927118+0000", 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024995479999999999 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927161+0000", 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2997000000000002e-05 2024-09-10T10:28:08.257 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.258 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.258 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.258 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.258 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.258 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:08.259 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:08.260 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:08.258+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:08.360 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:08.359+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:08.363 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:08.363 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:08.363 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:08.363 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:08.371 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:08.461 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:08.460+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:08.474 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:08.474 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:08.474 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:08.474 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:08.474 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:08.483 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:08.567 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:08.566+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:08.627 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:08.627 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:08.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:08.627 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:08.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.627 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.753003845000002, 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.006807553, 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.628 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872290+0000", 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2029999999999999e-06 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872310+0000", 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0508000000000001e-05 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872331+0000", 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0577e-05 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872341+0000", 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0421999999999999e-05 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872457+0000", 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115618 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872575+0000", 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118563 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872690+0000", 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011438900000000001 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875281+0000", 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002590995 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879035+0000", 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037544879999999998 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879069+0000", 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4034e-05 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879095+0000", 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5755999999999999e-05 2024-09-10T10:28:08.632 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.751807223999997, 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0061030149999999998, 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873485+0000", 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2419999999999999e-06 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.669e-06 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873505+0000", 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.325e-05 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873514+0000", 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1949999999999992e-06 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873575+0000", 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873647+0000", 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2346000000000002e-05 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873778+0000", 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013125600000000001 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875444+0000", 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001665962 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879528+0000", 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040837809999999999 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879563+0000", 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5329e-05 2024-09-10T10:28:08.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879587+0000", 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3947999999999998e-05 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.751753174000001, 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096274789999999996, 2024-09-10T10:28:08.638 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873539+0000", 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0299999999999998e-07 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873554+0000", 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5481999999999998e-05 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873565+0000", 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0733e-05 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873573+0000", 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9270000000000005e-06 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873633+0000", 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9864000000000001e-05 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873752+0000", 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118946 2024-09-10T10:28:08.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873866+0000", 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000113757 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875462+0000", 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015966699999999999 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883084+0000", 2024-09-10T10:28:08.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076214280000000004 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883136+0000", 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2182000000000003e-05 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883166+0000", 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.9787000000000001e-05 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.751695202999997, 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097094669999999994, 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873597+0000", 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8899999999999998e-07 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873602+0000", 2024-09-10T10:28:08.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9230000000000001e-06 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873611+0000", 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9150000000000002e-06 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5239999999999997e-06 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874719+0000", 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0011012769999999999 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874791+0000", 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2305000000000007e-05 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874962+0000", 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000171099 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877786+0000", 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028241059999999998 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883211+0000", 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054250019999999999 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883263+0000", 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1563000000000001e-05 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.648 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883306+0000", 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2864000000000002e-05 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.750774513000003, 2024-09-10T10:28:08.649 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005164681, 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874518+0000", 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3799999999999996e-07 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874521+0000", 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8550000000000004e-06 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874531+0000", 2024-09-10T10:28:08.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8079999999999999e-06 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874538+0000", 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2760000000000003e-06 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874945+0000", 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000406516 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875045+0000", 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9570999999999999e-05 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879682+0000", 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046369169999999999 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:08.653 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 59.750740618000002, 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056651770000000004, 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:08.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4600000000000005e-07 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874565+0000", 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4017e-05 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.655 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874573+0000", 2024-09-10T10:28:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9689999999999994e-06 2024-09-10T10:28:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:28:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874678+0000", 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7483000000000002e-05 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874786+0000", 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107914 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.880216+0000", 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005429499 2024-09-10T10:28:08.657 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:24 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 54.676432294999998, 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0079888200000000006, 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.658 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948861+0000", 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9269999999999999e-06 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948869+0000", 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7749999999999993e-06 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948884+0000", 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5713e-05 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948894+0000", 2024-09-10T10:28:08.660 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2050000000000008e-06 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948958+0000", 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4398000000000002e-05 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949093+0000", 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013508300000000001 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:08.661 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949223+0000", 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012949500000000001 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953128+0000", 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039058360000000002 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956775+0000", 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036467439999999999 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956812+0000", 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6510000000000001e-05 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956848+0000", 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6134000000000003e-05 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:23 2.c e26/18 2:3631e767:::102.00000000.inode:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:08.663 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 54.674948045000001, 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054176620000000002, 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.664 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950344+0000", 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9400000000000001e-07 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950362+0000", 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8655999999999999e-05 2024-09-10T10:28:08.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950384+0000", 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1929999999999998e-05 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950391+0000", 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7020000000000004e-06 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.666 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950624+0000", 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000233031 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950643+0000", 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9459000000000001e-05 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955726+0000", 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050826389999999999 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4851000000000002e-05 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.704299059999997, 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0070980829999999998, 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.668 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920994+0000", 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.697e-06 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921001+0000", 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3719999999999999e-06 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921014+0000", 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2529e-05 2024-09-10T10:28:08.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921021+0000", 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9519999999999997e-06 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921097+0000", 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6499000000000001e-05 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921261+0000", 2024-09-10T10:28:08.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016399599999999999 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921396+0000", 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013448400000000001 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922936+0000", 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015405970000000001 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927988+0000", 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00505158 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928047+0000", 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9011000000000001e-05 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928090+0000", 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3365999999999998e-05 2024-09-10T10:28:08.673 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.704247549000002, 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007356853, 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5699999999999998e-07 2024-09-10T10:28:08.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921048+0000", 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.754e-06 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921056+0000", 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0469999999999994e-06 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921063+0000", 2024-09-10T10:28:08.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5980000000000002e-06 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921747+0000", 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00068475800000000005 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921852+0000", 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104347 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922002+0000", 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000150233 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924357+0000", 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0023545599999999999 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928286+0000", 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039299540000000003 2024-09-10T10:28:08.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928349+0000", 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2044000000000004e-05 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928401+0000", 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2101000000000001e-05 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 48.704208090999998, 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005625166, 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921084+0000", 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0100000000000005e-07 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921093+0000", 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9910000000000008e-06 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921099+0000", 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2269999999999998e-06 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921103+0000", 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0929999999999996e-06 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921167+0000", 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4030999999999997e-05 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921301+0000", 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013388699999999999 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921471+0000", 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017000899999999999 2024-09-10T10:28:08.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924553+0000", 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030818199999999999 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926600+0000", 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020471339999999999 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926663+0000", 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3274000000000004e-05 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926708+0000", 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5099000000000003e-05 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:08.685 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 42.681656421, 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010012200000000001, 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:08.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943637+0000", 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5390000000000002e-06 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943646+0000", 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1340000000000004e-06 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943661+0000", 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4958e-05 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943669+0000", 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5629999999999992e-06 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943785+0000", 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011584800000000001 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943960+0000", 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000174611 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944078+0000", 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118381 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948022+0000", 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039442599999999998 2024-09-10T10:28:08.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953546+0000", 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055242449999999997 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953604+0000", 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7555e-05 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953647+0000", 2024-09-10T10:28:08.690 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3105999999999997e-05 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 42.681462861999997, 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099609380000000008, 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.691 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943829+0000", 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3099999999999996e-07 2024-09-10T10:28:08.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943835+0000", 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8610000000000003e-06 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943846+0000", 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0686999999999999e-05 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.693 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943854+0000", 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944413+0000", 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055893399999999997 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944505+0000", 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2788999999999995e-05 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:08.694 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944639+0000", 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013387699999999999 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950701+0000", 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0060612360000000002 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953700+0000", 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029997689999999998 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.695 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953743+0000", 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3024e-05 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953789+0000", 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6109999999999997e-05 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.696 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 42.681422840000003, 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054304089999999998, 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.697 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1399999999999996e-07 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943874+0000", 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3830000000000002e-06 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.698 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943881+0000", 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1369999999999996e-06 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943886+0000", 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-06 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944057+0000", 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000170781 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944182+0000", 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012506299999999999 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944309+0000", 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127081 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.947998+0000", 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003689323 2024-09-10T10:28:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949225+0000", 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001227005 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949267+0000", 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1391999999999999e-05 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.701 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949299+0000", 2024-09-10T10:28:08.702 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2169999999999999e-05 2024-09-10T10:28:08.702 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.702 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.702 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.704 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.704 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:08.704 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 42.681362358000001, 2024-09-10T10:28:08.704 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0068819399999999996, 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:08.705 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943930+0000", 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1200000000000002e-07 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943934+0000", 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0969999999999999e-06 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.706 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944782+0000", 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00084778299999999996 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944793+0000", 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0993e-05 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944864+0000", 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1241000000000005e-05 2024-09-10T10:28:08.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944940+0000", 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6044999999999994e-05 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945068+0000", 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012846 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.708 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949683+0000", 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046151040000000001 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950706+0000", 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010222149999999999 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950764+0000", 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8180999999999997e-05 2024-09-10T10:28:08.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950811+0000", 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7108999999999999e-05 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:24 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 42.679988731999998, 2024-09-10T10:28:08.710 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055052950000000003, 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.712 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.712 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945304+0000", 2024-09-10T10:28:08.712 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4799999999999997e-07 2024-09-10T10:28:08.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945311+0000", 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1269999999999997e-06 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945322+0000", 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1134e-05 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.713 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945330+0000", 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9039999999999992e-06 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945408+0000", 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8200000000000003e-05 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945533+0000", 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000125518 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945659+0000", 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012583599999999999 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949147+0000", 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003487675 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950716+0000", 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015695279999999999 2024-09-10T10:28:08.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950767+0000", 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0577e-05 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950808+0000", 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0847999999999999e-05 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:25 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 42.679347362999998, 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050290129999999997, 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:08.717 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945945+0000", 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1119999999999999e-06 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.718 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945952+0000", 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5189999999999998e-06 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945965+0000", 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3643e-05 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945974+0000", 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5750000000000001e-06 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0532999999999998e-05 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946192+0000", 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015717699999999999 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946341+0000", 2024-09-10T10:28:08.720 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014897200000000001 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949885+0000", 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035449729999999999 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950896+0000", 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00101023 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.721 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950938+0000", 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2284999999999999e-05 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950973+0000", 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4993999999999997e-05 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.722 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 42.679294290999998, 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048920780000000002, 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.723 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945998+0000", 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3900000000000004e-07 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946012+0000", 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4799e-05 2024-09-10T10:28:08.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946021+0000", 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.6000000000000007e-06 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946029+0000", 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9209999999999992e-06 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946093+0000", 2024-09-10T10:28:08.725 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3608000000000005e-05 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946242+0000", 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014889699999999999 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950889+0000", 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046476140000000004 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 42.679250263999997, 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047976679999999997, 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946042+0000", 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5199999999999996e-07 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946046+0000", 2024-09-10T10:28:08.728 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1439999999999996e-06 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946054+0000", 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1140000000000006e-06 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946070+0000", 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6232e-05 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.729 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946137+0000", 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6956000000000001e-05 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946260+0000", 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012240299999999999 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950839+0000", 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045791670000000003 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:23 2.1c e28/18 2:399c8d31:::104.00000000.inode:head v 28'5, mlcod=24'1)", 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 42.678730924999996, 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042418079999999997, 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3e-07 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946574+0000", 2024-09-10T10:28:08.732 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.277e-05 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946584+0000", 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7710000000000001e-06 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946590+0000", 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6769999999999998e-06 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.733 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946666+0000", 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5420999999999995e-05 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946687+0000", 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1665999999999999e-05 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950762+0000", 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040749030000000004 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:08.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:08.735 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:08.735 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950802+0000", 2024-09-10T10:28:08.735 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9969999999999998e-05 2024-09-10T10:28:08.735 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.735 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.735 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.735 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:08.735 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:08.735 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:08.736 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:08.749 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:08.748+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:08.837 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:08.836+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:08.857 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:08.857 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:08.857 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:08.857 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:08.866 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:08.938 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:08.937+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:08.971 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:08.971 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:08.971 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:08.971 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:08.971 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:08.979 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:09.064 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:28:09.066 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:09.065+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:09.112 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:28:09.112 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:28:09.112 INFO:tasks.thrashosds.thrasher:Reweighting osd 4 to 0.9353561255939 2024-09-10T10:28:09.113 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 0.9353561255939 2024-09-10T10:28:09.126 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:09.126 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:09.126 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:09.126 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.345004736999996, 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:09.127 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:28:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.131 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.132 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.344674237000007, 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:09.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:09.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:09.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.135 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:09.136 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.138 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.344627693999996, 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.141 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.142 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 60.253185481999999, 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:09.143 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 60.253104139000001, 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:28:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:28:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 60.253077584000003, 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.155 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.155 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:09.155 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.155 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:28:09.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:28:09.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.158 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:28:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.160 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 60.252934934999999, 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:28:09.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:28:09.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:28:09.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.166 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.166 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:28:09.166 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:28:09.166 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.166 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.166 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 60.250288001999998, 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:28:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 60.249437262000001, 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.171 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:28:09.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:28:09.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 55.177285578999999, 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.176 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:28:09.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.181 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 55.177236442999998, 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:09.182 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:28:09.184 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:09.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:28:09.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.187 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 55.177124964999997, 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:28:09.189 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 49.203724446000003, 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.192 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:28:09.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.195 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.195 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:28:09.195 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:09.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.195 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.195 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:28:09.195 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:28:09.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.196 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.196 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:28:09.196 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:28:09.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.198 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 49.203694274999997, 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.199 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:28:09.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:28:09.203 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 49.203637782999998, 2024-09-10T10:28:09.204 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.206 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:28:09.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:28:09.209 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 49.203536614999997, 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:28:09.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.213 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:28:09.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:28:09.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.213 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:28:09.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:28:09.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.214 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:09.214 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:28:09.214 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:28:09.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:28:09.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 43.183269518000003, 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:28:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:28:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:28:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:28:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:28:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 43.179204298999998, 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.224 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.224 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:09.224 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.225 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.225 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:28:09.225 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:28:09.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.226 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:28:09.226 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:28:09.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.226 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.226 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:28:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:28:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:28:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:28:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:28:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:28:09.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:28:09.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:28:09.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.230 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.230 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:28:09.230 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:28:09.230 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.230 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.230 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:28:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 43.179186428999998, 2024-09-10T10:28:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:28:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:28:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:28:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:28:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 43.179140191000002, 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:28:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:28:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:28:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:28:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:09.240 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:09.241 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-10T10:28:09.243 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:09.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.185+0000 7fe6f03df640 1 Processor -- start 2024-09-10T10:28:09.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.185+0000 7fe6f03df640 1 -- start start 2024-09-10T10:28:09.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.187+0000 7fe6f03df640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe6e8154c10 0x7fe6e815b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.187+0000 7fe6f03df640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe6e80cafa0 0x7fe6e80cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.187+0000 7fe6f03df640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e815e0b0 con 0x7fe6e80cca70 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.187+0000 7fe6f03df640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e815de60 con 0x7fe6e8154c10 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.187+0000 7fe6f03df640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e815e390 con 0x7fe6e80cafa0 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.187+0000 7fe6ee17b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe6e8154c10 0x7fe6e815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.187+0000 7fe6ee17b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe6e8154c10 0x7fe6e815b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56986/0 (socket says 172.21.15.120:56986) 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.187+0000 7fe6ee17b640 1 -- 172.21.15.120:0/196750709 learned_addr learned my addr 172.21.15.120:0/196750709 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.187+0000 7fe6ee97c640 1 --2- 172.21.15.120:0/196750709 >> v2:172.21.15.134:3300/0 conn(0x7fe6e80cafa0 0x7fe6e80cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.188+0000 7fe6ee97c640 1 -- 172.21.15.120:0/196750709 >> v2:172.21.15.120:3300/0 conn(0x7fe6e8154c10 msgr2=0x7fe6e815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.188+0000 7fe6ee97c640 1 --2- 172.21.15.120:0/196750709 >> v2:172.21.15.120:3300/0 conn(0x7fe6e8154c10 0x7fe6e815b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.188+0000 7fe6ee97c640 1 -- 172.21.15.120:0/196750709 >> v1:172.21.15.134:6789/0 conn(0x7fe6e80cca70 legacy=0x7fe6e8073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.188+0000 7fe6ee97c640 1 -- 172.21.15.120:0/196750709 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e815e670 con 0x7fe6e80cafa0 2024-09-10T10:28:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.189+0000 7fe6ee97c640 1 --2- 172.21.15.120:0/196750709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80cafa0 0x7fe6e80cb6f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe6e815dfe0 tx=0x7fe6e002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7697cd6275fc87dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.189+0000 7fe6ed179640 1 -- 172.21.15.120:0/196750709 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0007740 con 0x7fe6e80cafa0 2024-09-10T10:28:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.189+0000 7fe6ed179640 1 -- 172.21.15.120:0/196750709 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e00078e0 con 0x7fe6e80cafa0 2024-09-10T10:28:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.189+0000 7fe6ed179640 1 -- 172.21.15.120:0/196750709 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0007cb0 con 0x7fe6e80cafa0 2024-09-10T10:28:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.190+0000 7fe6f03df640 1 -- 172.21.15.120:0/196750709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80cafa0 msgr2=0x7fe6e80cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.190+0000 7fe6f03df640 1 --2- 172.21.15.120:0/196750709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80cafa0 0x7fe6e80cb6f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe6e815dfe0 tx=0x7fe6e002a5f0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.190+0000 7fe6f03df640 1 -- 172.21.15.120:0/196750709 shutdown_connections 2024-09-10T10:28:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.190+0000 7fe6f03df640 1 --2- 172.21.15.120:0/196750709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80cafa0 0x7fe6e80cb6f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.190+0000 7fe6f03df640 1 --2- 172.21.15.120:0/196750709 >> v2:172.21.15.120:3300/0 conn(0x7fe6e8154c10 0x7fe6e815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.190+0000 7fe6f03df640 1 -- 172.21.15.120:0/196750709 >> 172.21.15.120:0/196750709 conn(0x7fe6e806ce80 msgr2=0x7fe6e806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.190+0000 7fe6f03df640 1 -- 172.21.15.120:0/196750709 shutdown_connections 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.190+0000 7fe6f03df640 1 -- 172.21.15.120:0/196750709 wait complete. 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.191+0000 7fe6f03df640 1 Processor -- start 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.191+0000 7fe6f03df640 1 -- start start 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6f03df640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e80cafa0 0x7fe6e80cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6f03df640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80cca70 0x7fe6e815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6f03df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6e8154c10 0x7fe6e8145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6f03df640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8154ff0 con 0x7fe6e8154c10 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6f03df640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e815ec30 con 0x7fe6e80cca70 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6f03df640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e8073a50 con 0x7fe6e80cafa0 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6ee17b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e80cafa0 0x7fe6e80cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6ed97a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80cca70 0x7fe6e815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6ee97c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6e8154c10 0x7fe6e8145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6ed97a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80cca70 0x7fe6e815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:58774/0 (socket says 172.21.15.120:58774) 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.192+0000 7fe6ed97a640 1 -- 172.21.15.120:0/2765039209 learned_addr learned my addr 172.21.15.120:0/2765039209 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6ee97c640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e80cafa0 msgr2=0x7fe6e80cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6ee97c640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e80cafa0 0x7fe6e80cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6ee97c640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80cca70 msgr2=0x7fe6e815ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6ee97c640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80cca70 0x7fe6e815ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6ee97c640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e8153c50 con 0x7fe6e8154c10 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6ee97c640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6e8154c10 0x7fe6e8145770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0031000 tx=0x7fe6e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0040070 con 0x7fe6e8154c10 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e0031bc0 con 0x7fe6e8154c10 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0009450 con 0x7fe6e8154c10 2024-09-10T10:28:09.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e815d130 con 0x7fe6e8154c10 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.193+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e815d6f0 con 0x7fe6e8154c10 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.194+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe6e815cf20 con 0x7fe6e8154c10 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.196+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6e0039040 con 0x7fe6e8154c10 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.196+0000 7fe6deffd640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe6cc06be30 0x7fe6cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.196+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6801+0+0 (secure 0 0 0) 0x7fe6e00abe80 con 0x7fe6e8154c10 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.197+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v43) ==== 54+0+16 (secure 0 0 0) 0x7fe6e004b070 con 0x7fe6e8154c10 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.197+0000 7fe6ee17b640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe6cc06be30 0x7fe6cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.197+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fe6e815bf90 0x7fe6e8143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.197+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6e8154430 con 0x7fe6e815bf90 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.197+0000 7fe6ed97a640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fe6e815bf90 0x7fe6e8143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.197+0000 7fe6ee17b640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe6cc06be30 0x7fe6cc06e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009b30 tx=0x7fe6e4009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.197+0000 7fe6ed97a640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fe6e815bf90 0x7fe6e8143ea0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.198+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6e8154430 con 0x7fe6e815bf90 2024-09-10T10:28:09.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.212+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe6e8159c20 con 0x7fe6e815bf90 2024-09-10T10:28:09.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.215+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe6e8159c20 con 0x7fe6e815bf90 2024-09-10T10:28:09.252 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:09.253 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:09.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.215+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fe6e815b370 0x7fe6e8062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.215+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe6e8159940 con 0x7fe6e815b370 2024-09-10T10:28:09.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.215+0000 7fe6ee17b640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fe6e815b370 0x7fe6e8062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.216+0000 7fe6ee17b640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fe6e815b370 0x7fe6e8062630 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.216+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6e8159940 con 0x7fe6e815b370 2024-09-10T10:28:09.254 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:09.240+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:09.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.243+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe6e8159660 con 0x7fe6e815b370 2024-09-10T10:28:09.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.246+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe6e8159660 con 0x7fe6e815b370 2024-09-10T10:28:09.255 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:09.256 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:09.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.246+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fe6e8145cb0 0x7fe6e8063080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.246+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe6e80635c0 con 0x7fe6e8145cb0 2024-09-10T10:28:09.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.246+0000 7fe6ed97a640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fe6e8145cb0 0x7fe6e8063080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.247+0000 7fe6ed97a640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fe6e8145cb0 0x7fe6e8063080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.248+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6e80635c0 con 0x7fe6e8145cb0 2024-09-10T10:28:09.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.249+0000 7f7f28b9f640 1 Processor -- start 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.249+0000 7f7f28b9f640 1 -- start start 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.250+0000 7f7f28b9f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7f2415c680 0x7f7f240c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.250+0000 7f7f28b9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f2415ce10 0x7f7f2415d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.250+0000 7f7f28b9f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f2415d7e0 con 0x7f7f2415f8b0 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.250+0000 7f7f28b9f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7f240c4880 con 0x7f7f2415c680 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.250+0000 7f7f28b9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7f240c7f70 con 0x7f7f2415ce10 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.250+0000 7f7f21d9b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f7f2415f8b0 0x7f7f2406e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:44262/0 (socket says 172.21.15.120:44262) 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.251+0000 7f7f2259c640 1 --2- 172.21.15.120:0/440894006 >> v2:172.21.15.134:3300/0 conn(0x7f7f2415c680 0x7f7f240c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.250+0000 7f7f21d9b640 1 -- 172.21.15.120:0/440894006 learned_addr learned my addr 172.21.15.120:0/440894006 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.251+0000 7f7f22d9d640 1 --2- 172.21.15.120:0/440894006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f2415ce10 0x7f7f2415d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.251+0000 7f7f2259c640 1 -- 172.21.15.120:0/440894006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f2415ce10 msgr2=0x7f7f2415d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:09.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.251+0000 7f7f2259c640 1 --2- 172.21.15.120:0/440894006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f2415ce10 0x7f7f2415d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.251+0000 7f7f2259c640 1 -- 172.21.15.120:0/440894006 >> v1:172.21.15.120:6789/0 conn(0x7f7f2415f8b0 legacy=0x7f7f2406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.251+0000 7f7f2259c640 1 -- 172.21.15.120:0/440894006 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2406c4a0 con 0x7f7f2415c680 2024-09-10T10:28:09.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.252+0000 7f7f2259c640 1 --2- 172.21.15.120:0/440894006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 0x7f7f240c78e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f140097a0 tx=0x7f7f1402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e8410f166e3a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.252+0000 7f7f2159a640 1 -- 172.21.15.120:0/440894006 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14037070 con 0x7f7f2415c680 2024-09-10T10:28:09.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.252+0000 7f7f2159a640 1 -- 172.21.15.120:0/440894006 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f14032440 con 0x7f7f2415c680 2024-09-10T10:28:09.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.252+0000 7f7f2159a640 1 -- 172.21.15.120:0/440894006 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14032810 con 0x7f7f2415c680 2024-09-10T10:28:09.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.253+0000 7f7f28b9f640 1 -- 172.21.15.120:0/440894006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 msgr2=0x7f7f240c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.253+0000 7f7f28b9f640 1 --2- 172.21.15.120:0/440894006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 0x7f7f240c78e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f140097a0 tx=0x7f7f1402a210 comp rx=0 tx=0).stop 2024-09-10T10:28:09.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.253+0000 7f7f28b9f640 1 -- 172.21.15.120:0/440894006 shutdown_connections 2024-09-10T10:28:09.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.253+0000 7f7f28b9f640 1 --2- 172.21.15.120:0/440894006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f2415ce10 0x7f7f2415d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.253+0000 7f7f28b9f640 1 --2- 172.21.15.120:0/440894006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 0x7f7f240c78e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.253+0000 7f7f28b9f640 1 -- 172.21.15.120:0/440894006 >> 172.21.15.120:0/440894006 conn(0x7f7f240728b0 msgr2=0x7f7f24072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:09.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.253+0000 7f7f28b9f640 1 -- 172.21.15.120:0/440894006 shutdown_connections 2024-09-10T10:28:09.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.253+0000 7f7f28b9f640 1 -- 172.21.15.120:0/440894006 wait complete. 2024-09-10T10:28:09.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.255+0000 7f7f28b9f640 1 Processor -- start 2024-09-10T10:28:09.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.255+0000 7f7f28b9f640 1 -- start start 2024-09-10T10:28:09.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.256+0000 7f7f28b9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 0x7f7f240c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.256+0000 7f7f28b9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f2415ce10 0x7f7f241eba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.256+0000 7f7f28b9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2415f8b0 0x7f7f241ead70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.256+0000 7f7f28b9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2406cd40 con 0x7f7f2415f8b0 2024-09-10T10:28:09.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f28b9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2406ca60 con 0x7f7f2415c680 2024-09-10T10:28:09.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f28b9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2406d020 con 0x7f7f2415ce10 2024-09-10T10:28:09.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f2259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 0x7f7f240c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f21d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f2415ce10 0x7f7f241eba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f2259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 0x7f7f240c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:58792/0 (socket says 172.21.15.120:58792) 2024-09-10T10:28:09.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f2259c640 1 -- 172.21.15.120:0/4035152709 learned_addr learned my addr 172.21.15.120:0/4035152709 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:09.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f22d9d640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2415f8b0 0x7f7f241ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f2259c640 1 -- 172.21.15.120:0/4035152709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f2415ce10 msgr2=0x7f7f241eba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f2259c640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f2415ce10 0x7f7f241eba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f2259c640 1 -- 172.21.15.120:0/4035152709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2415f8b0 msgr2=0x7f7f241ead70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f2259c640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2415f8b0 0x7f7f241ead70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.257+0000 7f7f2259c640 1 -- 172.21.15.120:0/4035152709 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f240ca220 con 0x7f7f2415c680 2024-09-10T10:28:09.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.258+0000 7f7f2259c640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 0x7f7f240c9ce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7f14035040 tx=0x7f7f140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.258+0000 7f7f12ffd640 1 -- 172.21.15.120:0/4035152709 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14037040 con 0x7f7f2415c680 2024-09-10T10:28:09.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.258+0000 7f7f12ffd640 1 -- 172.21.15.120:0/4035152709 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f14030500 con 0x7f7f2415c680 2024-09-10T10:28:09.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.258+0000 7f7f28b9f640 1 -- 172.21.15.120:0/4035152709 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f241f5ad0 con 0x7f7f2415c680 2024-09-10T10:28:09.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.258+0000 7f7f12ffd640 1 -- 172.21.15.120:0/4035152709 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14030910 con 0x7f7f2415c680 2024-09-10T10:28:09.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.258+0000 7f7f28b9f640 1 -- 172.21.15.120:0/4035152709 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f240c86b0 con 0x7f7f2415c680 2024-09-10T10:28:09.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.260+0000 7f7f12ffd640 1 -- 172.21.15.120:0/4035152709 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f14044020 con 0x7f7f2415c680 2024-09-10T10:28:09.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.260+0000 7f7f28b9f640 1 -- 172.21.15.120:0/4035152709 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f240735e0 con 0x7f7f2415c680 2024-09-10T10:28:09.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.261+0000 7f7f12ffd640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7f0406be60 0x7f7f0406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.261+0000 7f7f21d9b640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7f0406be60 0x7f7f0406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.261+0000 7f7f12ffd640 1 -- 172.21.15.120:0/4035152709 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6801+0+0 (secure 0 0 0) 0x7f7f140ab490 con 0x7f7f2415c680 2024-09-10T10:28:09.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.262+0000 7f7f21d9b640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7f0406be60 0x7f7f0406e320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f241e6f10 tx=0x7f7f18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.265+0000 7f7f12ffd640 1 -- 172.21.15.120:0/4035152709 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7f14081e40 con 0x7f7f2415c680 2024-09-10T10:28:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.269+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe6e8144e30 con 0x7fe6e8145cb0 2024-09-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.271+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe6d80019e0 con 0x7fe6e8145cb0 2024-09-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.272+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fe6e80b48c0 0x7fe6e8152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.272+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe6e80b4ca0 con 0x7fe6e80b48c0 2024-09-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.272+0000 7fe6ee97c640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fe6e80b48c0 0x7fe6e8152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.272+0000 7fe6ee97c640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fe6e80b48c0 0x7fe6e8152430 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.273+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6e80b4ca0 con 0x7fe6e80b48c0 2024-09-10T10:28:09.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.299+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe6e80b4ca0 con 0x7fe6e80b48c0 2024-09-10T10:28:09.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.302+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe6e80b4ca0 con 0x7fe6e80b48c0 2024-09-10T10:28:09.303 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:09.303 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:09.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.302+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe6e8152e80 0x7fe6e802c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.302+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe6e81532e0 con 0x7fe6e8152e80 2024-09-10T10:28:09.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.302+0000 7fe6ee17b640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe6e8152e80 0x7fe6e802c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.303+0000 7fe6ee17b640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe6e8152e80 0x7fe6e802c640 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.303+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6e81532e0 con 0x7fe6e8152e80 2024-09-10T10:28:09.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.328+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe6e81532e0 con 0x7fe6e8152e80 2024-09-10T10:28:09.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.331+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe6e81532e0 con 0x7fe6e8152e80 2024-09-10T10:28:09.332 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:09.333 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:09.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.331+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe6e802d090 0x7fe6e802d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.331+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe6e8153520 con 0x7fe6e802d090 2024-09-10T10:28:09.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.332+0000 7fe6ed97a640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe6e802d090 0x7fe6e802d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.332+0000 7fe6ed97a640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe6e802d090 0x7fe6e802d4f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.333+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6d8019560 con 0x7fe6e802d090 2024-09-10T10:28:09.342 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:09.341+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:09.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.359+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe6e80008d0 con 0x7fe6e802d090 2024-09-10T10:28:09.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.361+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe6e80008d0 con 0x7fe6e802d090 2024-09-10T10:28:09.362 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.361+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fe6e8207900 0x7fe6e8209cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.361+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe6e820cc10 con 0x7fe6e8207900 2024-09-10T10:28:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.361+0000 7fe6ee97c640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fe6e8207900 0x7fe6e8209cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.362+0000 7fe6ee97c640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fe6e8207900 0x7fe6e8209cf0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.362+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6e820cc10 con 0x7fe6e8207900 2024-09-10T10:28:09.389 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:09.389 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:09.389 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:09.389 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:09.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.388+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe6e820cbe0 con 0x7fe6e8207900 2024-09-10T10:28:09.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.390+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe6e820cbe0 con 0x7fe6e8207900 2024-09-10T10:28:09.391 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:09.392 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:09.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.390+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fe6e820c240 0x7fe6e820a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.390+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe6e820c740 con 0x7fe6e820c240 2024-09-10T10:28:09.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.391+0000 7fe6ee17b640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fe6e820c240 0x7fe6e820a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.391+0000 7fe6ee17b640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fe6e820c240 0x7fe6e820a230 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.392+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6e820c740 con 0x7fe6e820c240 2024-09-10T10:28:09.398 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:09.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.405+0000 7f3dd2e1e640 1 Processor -- start 2024-09-10T10:28:09.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.405+0000 7f3dd2e1e640 1 -- start start 2024-09-10T10:28:09.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dd2e1e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dd2e1e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc15b1d0 con 0x7f3dcc0cc4a0 2024-09-10T10:28:09.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dd2e1e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc15b4b0 con 0x7f3dcc0cc0c0 2024-09-10T10:28:09.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dd2e1e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc0cbd00 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dd13bb640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f3dcc0cc4a0 0x7f3dcc165430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:44264/0 (socket says 172.21.15.120:44264) 2024-09-10T10:28:09.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dd13bb640 1 -- 172.21.15.120:0/184713604 learned_addr learned my addr 172.21.15.120:0/184713604 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:09.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dcbfff640 1 --2- 172.21.15.120:0/184713604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dcbfff640 1 -- 172.21.15.120:0/184713604 >> v1:172.21.15.134:6789/0 conn(0x7f3dcc0cc0c0 legacy=0x7f3dcc15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dcbfff640 1 -- 172.21.15.120:0/184713604 >> v1:172.21.15.120:6789/0 conn(0x7f3dcc0cc4a0 legacy=0x7f3dcc165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dcb7fe640 1 -- 172.21.15.120:0/184713604 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021176923 0 0) 0x7f3dcc15b1d0 con 0x7f3dcc0cc4a0 2024-09-10T10:28:09.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.407+0000 7f3dcbfff640 1 -- 172.21.15.120:0/184713604 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc15b790 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.409+0000 7f3dcbfff640 1 --2- 172.21.15.120:0/184713604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15aab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009750 tx=0x7f3dc002a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49b4fd6eec02614a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.409+0000 7f3dcb7fe640 1 -- 172.21.15.120:0/184713604 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0037070 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.409+0000 7f3dcb7fe640 1 -- 172.21.15.120:0/184713604 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc002adb0 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.409+0000 7f3dcb7fe640 1 -- 172.21.15.120:0/184713604 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0032660 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.410+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/184713604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 msgr2=0x7f3dcc15aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.410+0000 7f3dd2e1e640 1 --2- 172.21.15.120:0/184713604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15aab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009750 tx=0x7f3dc002a0a0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.410+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/184713604 shutdown_connections 2024-09-10T10:28:09.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.410+0000 7f3dd2e1e640 1 --2- 172.21.15.120:0/184713604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15aab0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.410+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/184713604 >> 172.21.15.120:0/184713604 conn(0x7f3dcc06d560 msgr2=0x7f3dcc06d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:09.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.410+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/184713604 shutdown_connections 2024-09-10T10:28:09.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.410+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/184713604 wait complete. 2024-09-10T10:28:09.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.412+0000 7f3dd2e1e640 1 Processor -- start 2024-09-10T10:28:09.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.412+0000 7f3dd2e1e640 1 -- start start 2024-09-10T10:28:09.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.413+0000 7f3dd2e1e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.413+0000 7f3dd2e1e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dcc0cc0c0 0x7f3dcc150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.413+0000 7f3dd2e1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dcc0cc4a0 0x7f3dcc14f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.413+0000 7f3dd0bba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.413+0000 7f3dd0bba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:58024/0 (socket says 172.21.15.120:58024) 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd13bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dcc0cc4a0 0x7f3dcc14f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd13bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dcc0cc4a0 0x7f3dcc14f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57008/0 (socket says 172.21.15.120:57008) 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dcbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dcc0cc0c0 0x7f3dcc150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd0bba640 1 -- 172.21.15.120:0/1044104768 learned_addr learned my addr 172.21.15.120:0/1044104768 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc15e540 con 0x7f3dcc0cc4a0 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc15ba70 con 0x7f3dcc0cc0c0 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc0cc8b0 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd0bba640 1 -- 172.21.15.120:0/1044104768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dcc0cc0c0 msgr2=0x7f3dcc150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd0bba640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dcc0cc0c0 0x7f3dcc150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd0bba640 1 -- 172.21.15.120:0/1044104768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dcc0cc4a0 msgr2=0x7f3dcc14f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd0bba640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dcc0cc4a0 0x7f3dcc14f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd0bba640 1 -- 172.21.15.120:0/1044104768 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc0ca730 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.414+0000 7f3dd13bb640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dcc0cc4a0 0x7f3dcc14f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:09.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.415+0000 7f3dd0bba640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15a180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c550 tx=0x7f3dbc00ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.415+0000 7f3dc97fa640 1 -- 172.21.15.120:0/1044104768 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc017070 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.415+0000 7f3dc97fa640 1 -- 172.21.15.120:0/1044104768 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dbc005480 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.415+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc0ca490 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.416+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc1f1930 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.416+0000 7f3dc97fa640 1 -- 172.21.15.120:0/1044104768 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc003830 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.417+0000 7f3dc97fa640 1 -- 172.21.15.120:0/1044104768 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3dbc01a070 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.417+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc0c60c0 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.418+0000 7f3dc97fa640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3db806be60 0x7f3db806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:09.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.418+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe6e820c950 con 0x7fe6e820c240 2024-09-10T10:28:09.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.418+0000 7f3dcbfff640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3db806be60 0x7f3db806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:09.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.418+0000 7f3dc97fa640 1 -- 172.21.15.120:0/1044104768 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(43..43 src has 1..43) ==== 6801+0+0 (secure 0 0 0) 0x7f3dbc08a600 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.419+0000 7f3dcbfff640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3db806be60 0x7f3db806e320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0034000 tx=0x7f3dc002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:09.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.419+0000 7fe6deffd640 1 -- 172.21.15.120:0/2765039209 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe6e4028e50 con 0x7fe6e820c240 2024-09-10T10:28:09.421 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:09.423 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:09.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fe6e815bf90 msgr2=0x7fe6e8143ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fe6e815bf90 0x7fe6e8143ea0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fe6e815b370 msgr2=0x7fe6e8062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fe6e815b370 0x7fe6e8062630 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fe6e8145cb0 msgr2=0x7fe6e8063080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fe6e8145cb0 0x7fe6e8063080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fe6e80b48c0 msgr2=0x7fe6e8152430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fe6e80b48c0 0x7fe6e8152430 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe6e8152e80 msgr2=0x7fe6e802c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe6e8152e80 0x7fe6e802c640 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe6e802d090 msgr2=0x7fe6e802d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe6e802d090 0x7fe6e802d4f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fe6e8207900 msgr2=0x7fe6e8209cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fe6e8207900 0x7fe6e8209cf0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fe6e820c240 msgr2=0x7fe6e820a230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fe6e820c240 0x7fe6e820a230 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6ee97c640 1 -- 172.21.15.120:0/2765039209 reap_dead start 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6ee97c640 1 -- 172.21.15.120:0/2765039209 reap_dead start 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6ee97c640 1 -- 172.21.15.120:0/2765039209 reap_dead start 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6ee97c640 1 -- 172.21.15.120:0/2765039209 reap_dead start 2024-09-10T10:28:09.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6ee97c640 1 -- 172.21.15.120:0/2765039209 reap_dead start 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe6cc06be30 msgr2=0x7fe6cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe6cc06be30 0x7fe6cc06e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009b30 tx=0x7fe6e4009240 comp rx=0 tx=0).stop 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6e8154c10 msgr2=0x7fe6e8145770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6e8154c10 0x7fe6e8145770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0031000 tx=0x7fe6e0004a40 comp rx=0 tx=0).stop 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 shutdown_connections 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe6cc06be30 0x7fe6cc06e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 --2- 172.21.15.120:0/2765039209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6e8154c10 0x7fe6e8145770 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.420+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 >> 172.21.15.120:0/2765039209 conn(0x7fe6e806ce80 msgr2=0x7fe6e80cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.421+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 shutdown_connections 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.421+0000 7fe6f03df640 1 -- 172.21.15.120:0/2765039209 wait complete. 2024-09-10T10:28:09.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.423+0000 7f3dc97fa640 1 -- 172.21.15.120:0/1044104768 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3dcc0c60c0 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.443 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:09.442+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:09.501 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:09.502 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:09.510 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:09.551 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:09.550+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:09.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.581+0000 7f7f28b9f640 1 -- 172.21.15.120:0/4035152709 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 0.9353561255939} v 0) -- 0x7f7f240c7160 con 0x7f7f2415c680 2024-09-10T10:28:09.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:09.629+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f3dcc0763a0 con 0x7f3dcc0cb2f0 2024-09-10T10:28:09.644 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:09.643+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:09.654 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:09.654 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:09.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:09.745 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:09.744+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:09.808 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:09.808 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:09.820 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:09.845 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:09.844+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:09.923 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:09.924 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:09.932 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:09.946 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:09.945+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:10.037 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:10.037 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:10.046 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:10.047 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:10.045+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:10.148 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:10.147+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:10.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.170+0000 7f7f12ffd640 1 -- 172.21.15.120:0/4035152709 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 0.9353561255939}]=0 reweighted osd.4 to 0.935356 (ef73) v44) ==== 131+0+0 (secure 0 0 0) 0x7f7f14079310 con 0x7f7f2415c680 2024-09-10T10:28:10.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.170+0000 7f3dc97fa640 1 -- 172.21.15.120:0/1044104768 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v44) ==== 101+0+0 (secure 0 0 0) 0x7f3dcc0763a0 con 0x7f3dcc0cb2f0 2024-09-10T10:28:10.172 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.4 to 0.935356 (ef73) 2024-09-10T10:28:10.172 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-10T10:28:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f7f28b9f640 1 -- 172.21.15.120:0/4035152709 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7f0406be60 msgr2=0x7f7f0406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f7f28b9f640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7f0406be60 0x7f7f0406e320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f241e6f10 tx=0x7f7f18009040 comp rx=0 tx=0).stop 2024-09-10T10:28:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f7f28b9f640 1 -- 172.21.15.120:0/4035152709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 msgr2=0x7f7f240c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f7f28b9f640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 0x7f7f240c9ce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7f14035040 tx=0x7f7f140046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f7f28b9f640 1 -- 172.21.15.120:0/4035152709 shutdown_connections 2024-09-10T10:28:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f7f28b9f640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7f0406be60 0x7f7f0406e320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f7f28b9f640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2415f8b0 0x7f7f241ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f7f28b9f640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f2415ce10 0x7f7f241eba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f7f28b9f640 1 --2- 172.21.15.120:0/4035152709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f2415c680 0x7f7f240c9ce0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f7f28b9f640 1 -- 172.21.15.120:0/4035152709 >> 172.21.15.120:0/4035152709 conn(0x7f7f240728b0 msgr2=0x7f7f240c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3db806be60 msgr2=0x7f3db806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f3dd2e1e640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3db806be60 0x7f3db806e320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0034000 tx=0x7f3dc002e000 comp rx=0 tx=0).stop 2024-09-10T10:28:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 msgr2=0x7f3dcc15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.174+0000 7f3dd2e1e640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15a180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c550 tx=0x7f3dbc00ca10 comp rx=0 tx=0).stop 2024-09-10T10:28:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.175+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 shutdown_connections 2024-09-10T10:28:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.175+0000 7f3dd2e1e640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3db806be60 0x7f3db806e320 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0034000 tx=0x7f3dc002e000 comp rx=0 tx=0).stop 2024-09-10T10:28:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.175+0000 7f3dd2e1e640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dcc0cc4a0 0x7f3dcc14f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.175+0000 7f3dd2e1e640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dcc0cc0c0 0x7f3dcc150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.175+0000 7f3dd2e1e640 1 --2- 172.21.15.120:0/1044104768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dcc0cb2f0 0x7f3dcc15a180 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c550 tx=0x7f3dbc00ca10 comp rx=0 tx=0).stop 2024-09-10T10:28:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.175+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 >> 172.21.15.120:0/1044104768 conn(0x7f3dcc06d560 msgr2=0x7f3dcc0cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.177+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 shutdown_connections 2024-09-10T10:28:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.177+0000 7f3dd2e1e640 1 -- 172.21.15.120:0/1044104768 wait complete. 2024-09-10T10:28:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.175+0000 7f7f28b9f640 1 -- 172.21.15.120:0/4035152709 shutdown_connections 2024-09-10T10:28:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:10.177+0000 7f7f28b9f640 1 -- 172.21.15.120:0/4035152709 wait complete. 2024-09-10T10:28:10.214 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:10.214 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:10.223 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:10.248 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:10.248+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:10.349 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:10.349+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:10.367 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:10.367 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:10.376 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:10.450 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:10.449+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:10.483 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:10.484 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:10.493 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:10.577 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:10.576+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:10.652 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:10.651+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:10.677 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:10.677 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:10.688 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:10.753 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:10.753+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:10.833 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:10.833 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:10.842 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:10.887 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:10.886+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:10.955 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:10.954+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:10.983 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:10.983 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:10.995 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:11.056 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:11.055+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:11.158 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:11.157+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:11.168 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:11.168 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:11.182 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:11.259 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:11.258+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:11.325 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:11.325 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:11.337 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:11.381 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:11.380+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:11.436 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:28:11.460 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:11.459+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:11.485 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:11.485 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:11.498 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:11.561 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:11.560+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.565+0000 7f4ef088c640 1 Processor -- start 2024-09-10T10:28:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.565+0000 7f4ef088c640 1 -- start start 2024-09-10T10:28:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.566+0000 7f4ef088c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4eec154c50 0x7f4eec0ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.566+0000 7f4ef088c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4eec0c6210 0x7f4eec0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.566+0000 7f4ef088c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eec0cbc10 con 0x7f4eec0caaf0 2024-09-10T10:28:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.566+0000 7f4ef088c640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4eec0cb030 con 0x7f4eec154c50 2024-09-10T10:28:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.566+0000 7f4ef088c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4eec0cc820 con 0x7f4eec0c6210 2024-09-10T10:28:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.566+0000 7f4eead9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4eec154c50 0x7f4eec0ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.566+0000 7f4eead9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4eec154c50 0x7f4eec0ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57024/0 (socket says 172.21.15.120:57024) 2024-09-10T10:28:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.566+0000 7f4eead9d640 1 -- 172.21.15.120:0/2663038745 learned_addr learned my addr 172.21.15.120:0/2663038745 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.566+0000 7f4eea59c640 1 --2- 172.21.15.120:0/2663038745 >> v2:172.21.15.134:3300/0 conn(0x7f4eec0c6210 0x7f4eec0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.567+0000 7f4eead9d640 1 -- 172.21.15.120:0/2663038745 >> v2:172.21.15.134:3300/0 conn(0x7f4eec0c6210 msgr2=0x7f4eec0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.567+0000 7f4eead9d640 1 --2- 172.21.15.120:0/2663038745 >> v2:172.21.15.134:3300/0 conn(0x7f4eec0c6210 0x7f4eec0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.567+0000 7f4eead9d640 1 -- 172.21.15.120:0/2663038745 >> v1:172.21.15.120:6789/0 conn(0x7f4eec0caaf0 legacy=0x7f4eec15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.567+0000 7f4ee9d9b640 1 -- 172.21.15.120:0/2663038745 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114441498 0 0) 0x7f4eec0cbc10 con 0x7f4eec0caaf0 2024-09-10T10:28:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.567+0000 7f4eead9d640 1 -- 172.21.15.120:0/2663038745 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec0cbef0 con 0x7f4eec154c50 2024-09-10T10:28:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.568+0000 7f4eead9d640 1 --2- 172.21.15.120:0/2663038745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 0x7f4eec0ccb80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4eec158370 tx=0x7f4edc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65198c074c37159 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.568+0000 7f4ee9d9b640 1 -- 172.21.15.120:0/2663038745 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc037070 con 0x7f4eec154c50 2024-09-10T10:28:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.568+0000 7f4ee9d9b640 1 -- 172.21.15.120:0/2663038745 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc032440 con 0x7f4eec154c50 2024-09-10T10:28:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.568+0000 7f4ee9d9b640 1 -- 172.21.15.120:0/2663038745 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc032810 con 0x7f4eec154c50 2024-09-10T10:28:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.569+0000 7f4ef088c640 1 -- 172.21.15.120:0/2663038745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 msgr2=0x7f4eec0ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.569+0000 7f4ef088c640 1 --2- 172.21.15.120:0/2663038745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 0x7f4eec0ccb80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4eec158370 tx=0x7f4edc02a270 comp rx=0 tx=0).stop 2024-09-10T10:28:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.569+0000 7f4ef088c640 1 -- 172.21.15.120:0/2663038745 shutdown_connections 2024-09-10T10:28:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.569+0000 7f4ef088c640 1 --2- 172.21.15.120:0/2663038745 >> v2:172.21.15.134:3300/0 conn(0x7f4eec0c6210 0x7f4eec0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.569+0000 7f4ef088c640 1 --2- 172.21.15.120:0/2663038745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 0x7f4eec0ccb80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.569+0000 7f4ef088c640 1 -- 172.21.15.120:0/2663038745 >> 172.21.15.120:0/2663038745 conn(0x7f4eec06e5e0 msgr2=0x7f4eec067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.569+0000 7f4ef088c640 1 -- 172.21.15.120:0/2663038745 shutdown_connections 2024-09-10T10:28:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.569+0000 7f4ef088c640 1 -- 172.21.15.120:0/2663038745 wait complete. 2024-09-10T10:28:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.571+0000 7f4ef088c640 1 Processor -- start 2024-09-10T10:28:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.571+0000 7f4ef088c640 1 -- start start 2024-09-10T10:28:11.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4ef088c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0c6210 0x7f4eec157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4ef088c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eec0caaf0 0x7f4eec06c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4ef088c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 0x7f4eec1f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4ef088c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0c7170 con 0x7f4eec154c50 2024-09-10T10:28:11.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4ef088c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0cc4b0 con 0x7f4eec0caaf0 2024-09-10T10:28:11.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4ef088c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec15b070 con 0x7f4eec0c6210 2024-09-10T10:28:11.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eeb59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 0x7f4eec1f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eea59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eec0caaf0 0x7f4eec06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eeb59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 0x7f4eec1f3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57040/0 (socket says 172.21.15.120:57040) 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eeb59e640 1 -- 172.21.15.120:0/1383719660 learned_addr learned my addr 172.21.15.120:0/1383719660 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eead9d640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0c6210 0x7f4eec157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eeb59e640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0c6210 msgr2=0x7f4eec157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eeb59e640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0c6210 0x7f4eec157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eeb59e640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eec0caaf0 msgr2=0x7f4eec06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eeb59e640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eec0caaf0 0x7f4eec06c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eeb59e640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec1f3840 con 0x7f4eec154c50 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eea59c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eec0caaf0 0x7f4eec06c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.572+0000 7f4eeb59e640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 0x7f4eec1f3fd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400c7f0 tx=0x7f4ee400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.573+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4017070 con 0x7f4eec154c50 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.573+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee4003dc0 con 0x7f4eec154c50 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.573+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40052b0 con 0x7f4eec154c50 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.573+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec14c920 con 0x7f4eec154c50 2024-09-10T10:28:11.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.573+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec150b20 con 0x7f4eec154c50 2024-09-10T10:28:11.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.574+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f4eec0ca7c0 con 0x7f4eec154c50 2024-09-10T10:28:11.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.575+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ee401a070 con 0x7f4eec154c50 2024-09-10T10:28:11.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.575+0000 7f4ed37fe640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ed806be60 0x7f4ed806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.576+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6801+0+0 (secure 0 0 0) 0x7f4ee408a5a0 con 0x7f4eec154c50 2024-09-10T10:28:11.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.576+0000 7f4eead9d640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ed806be60 0x7f4ed806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.576+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v45) ==== 54+0+16 (secure 0 0 0) 0x7f4eec0ca7c0 con 0x7f4eec154c50 2024-09-10T10:28:11.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.576+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f4eec1f3a10 0x7f4eec0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.576+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4eec0cb1d0 con 0x7f4eec1f3a10 2024-09-10T10:28:11.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.576+0000 7f4eead9d640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ed806be60 0x7f4ed806e320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4edc034000 tx=0x7f4edc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.576+0000 7f4eea59c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f4eec1f3a10 0x7f4eec0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.577+0000 7f4eea59c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f4eec1f3a10 0x7f4eec0cea90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.578+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec0cb1d0 con 0x7f4eec1f3a10 2024-09-10T10:28:11.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.605+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4eec158a00 con 0x7f4eec1f3a10 2024-09-10T10:28:11.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.607+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4eec158a00 con 0x7f4eec1f3a10 2024-09-10T10:28:11.608 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:11.609 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:11.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.607+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f4eec1f0e10 0x7f4eec14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.607+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f4eec15a5d0 con 0x7f4eec1f0e10 2024-09-10T10:28:11.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.607+0000 7f4eead9d640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f4eec1f0e10 0x7f4eec14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.608+0000 7f4eead9d640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f4eec1f0e10 0x7f4eec14f270 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.609+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec15a5d0 con 0x7f4eec1f0e10 2024-09-10T10:28:11.613 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:11.613 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:11.622 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:11.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.623+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4eec1f4510 con 0x7f4eec1f0e10 2024-09-10T10:28:11.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.625+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4eec1f4510 con 0x7f4eec1f0e10 2024-09-10T10:28:11.627 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:11.627 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:11.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.625+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f4eec14fcc0 0x7f4eec062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.625+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f4eec14eb50 con 0x7f4eec14fcc0 2024-09-10T10:28:11.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.626+0000 7f4eea59c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f4eec14fcc0 0x7f4eec062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.626+0000 7f4eea59c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f4eec14fcc0 0x7f4eec062630 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.627+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec14eb50 con 0x7f4eec14fcc0 2024-09-10T10:28:11.634 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:28:11.637 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:28:11.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.652+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4eec062e40 con 0x7f4eec14fcc0 2024-09-10T10:28:11.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.654+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4eec062e40 con 0x7f4eec14fcc0 2024-09-10T10:28:11.655 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:11.656 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:11.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.654+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f4eec159430 0x7f4eec159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.654+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4eec14ed30 con 0x7f4eec159430 2024-09-10T10:28:11.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.655+0000 7f4eeb59e640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f4eec159430 0x7f4eec159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.655+0000 7f4eeb59e640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f4eec159430 0x7f4eec159870 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.656+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec14ed30 con 0x7f4eec159430 2024-09-10T10:28:11.663 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:11.662+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:11.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.671+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4eec063410 con 0x7f4eec159430 2024-09-10T10:28:11.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.672+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4eec063410 con 0x7f4eec159430 2024-09-10T10:28:11.674 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:11.674 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:11.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.673+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f4eec02cb50 0x7f4eec02cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.673+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4eec063410 con 0x7f4eec02cb50 2024-09-10T10:28:11.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.673+0000 7f4eead9d640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f4eec02cb50 0x7f4eec02cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.673+0000 7f4eead9d640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f4eec02cb50 0x7f4eec02cf70 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.674+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec063410 con 0x7f4eec02cb50 2024-09-10T10:28:11.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.688+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4eec063410 con 0x7f4eec02cb50 2024-09-10T10:28:11.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.690+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4eec063410 con 0x7f4eec02cb50 2024-09-10T10:28:11.691 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:11.692 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:11.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.690+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f4eec204b20 0x7f4eec206f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.690+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f4eec02d9c0 con 0x7f4eec204b20 2024-09-10T10:28:11.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.690+0000 7f4eea59c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f4eec204b20 0x7f4eec206f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.690+0000 7f4eea59c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f4eec204b20 0x7f4eec206f10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.691+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec02d9c0 con 0x7f4eec204b20 2024-09-10T10:28:11.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.704+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4eec02d9c0 con 0x7f4eec204b20 2024-09-10T10:28:11.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.706+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4eec02d9c0 con 0x7f4eec204b20 2024-09-10T10:28:11.707 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:11.708 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:11.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.706+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f4eec207960 0x7f4eec209d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.706+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f4eec063410 con 0x7f4eec207960 2024-09-10T10:28:11.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.706+0000 7f4eeb59e640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f4eec207960 0x7f4eec209d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.707+0000 7f4eeb59e640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f4eec207960 0x7f4eec209d50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.708+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec063410 con 0x7f4eec207960 2024-09-10T10:28:11.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.721+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4eec063410 con 0x7f4eec207960 2024-09-10T10:28:11.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.724+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4eec063410 con 0x7f4eec207960 2024-09-10T10:28:11.725 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:11.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.724+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f4eec20a7a0 0x7f4eec20cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:11.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.724+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f4eec20fb60 con 0x7f4eec20a7a0 2024-09-10T10:28:11.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.724+0000 7f4eead9d640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f4eec20a7a0 0x7f4eec20cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:11.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.724+0000 7f4eead9d640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f4eec20a7a0 0x7f4eec20cb90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:11.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.725+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec20fb60 con 0x7f4eec20a7a0 2024-09-10T10:28:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.738+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4eec000950 con 0x7f4eec20a7a0 2024-09-10T10:28:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ed37fe640 1 -- 172.21.15.120:0/1383719660 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4eec000950 con 0x7f4eec20a7a0 2024-09-10T10:28:11.741 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f4eec1f3a10 msgr2=0x7f4eec0cea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f4eec1f3a10 0x7f4eec0cea90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f4eec1f0e10 msgr2=0x7f4eec14f270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f4eec1f0e10 0x7f4eec14f270 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f4eec14fcc0 msgr2=0x7f4eec062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f4eec14fcc0 0x7f4eec062630 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f4eec159430 msgr2=0x7f4eec159870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f4eec159430 0x7f4eec159870 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f4eec02cb50 msgr2=0x7f4eec02cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f4eec02cb50 0x7f4eec02cf70 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f4eec204b20 msgr2=0x7f4eec206f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f4eec204b20 0x7f4eec206f10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4eeb59e640 1 -- 172.21.15.120:0/1383719660 reap_dead start 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f4eec207960 msgr2=0x7f4eec209d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f4eec207960 0x7f4eec209d50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4eeb59e640 1 -- 172.21.15.120:0/1383719660 reap_dead start 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f4eec20a7a0 msgr2=0x7f4eec20cb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f4eec20a7a0 0x7f4eec20cb90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4eeb59e640 1 -- 172.21.15.120:0/1383719660 reap_dead start 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4eeb59e640 1 -- 172.21.15.120:0/1383719660 reap_dead start 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ed806be60 msgr2=0x7f4ed806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ed806be60 0x7f4ed806e320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4edc034000 tx=0x7f4edc02e000 comp rx=0 tx=0).stop 2024-09-10T10:28:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.740+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 msgr2=0x7f4eec1f3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.741+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 0x7f4eec1f3fd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400c7f0 tx=0x7f4ee400ccb0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.741+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 shutdown_connections 2024-09-10T10:28:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.741+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ed806be60 0x7f4ed806e320 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.741+0000 7f4ef088c640 1 --2- 172.21.15.120:0/1383719660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4eec154c50 0x7f4eec1f3fd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.741+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 >> 172.21.15.120:0/1383719660 conn(0x7f4eec06e5e0 msgr2=0x7f4eec153c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.741+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 shutdown_connections 2024-09-10T10:28:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:11.741+0000 7f4ef088c640 1 -- 172.21.15.120:0/1383719660 wait complete. 2024-09-10T10:28:11.769 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:11.769 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:11.769 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:11.769 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:11.769 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:11.780 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:11.794 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:11.793+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:11.863 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:11.862+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:11.885 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:11.885 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:11.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:11.885 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:11.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.885 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.103875848000001, 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047203150000000001, 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779674+0000", 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0359999999999995e-06 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779702+0000", 2024-09-10T10:28:11.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.7970000000000002e-05 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779724+0000", 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2544e-05 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779741+0000", 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6781000000000001e-05 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.888 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779981+0000", 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000239764 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780147+0000", 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016605100000000001 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017096200000000001 2024-09-10T10:28:11.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781749+0000", 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001431288 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784234+0000", 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024850940000000002 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784338+0000", 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104078 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784385+0000", 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6746999999999998e-05 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.1a epoch 24/18 rep_tid 2 entries 24'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:11.891 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.102397737999993, 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021396129999999998, 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.892 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781144+0000", 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.393e-06 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781156+0000", 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1514e-05 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8445999999999998e-05 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781202+0000", 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7439e-05 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781286+0000", 2024-09-10T10:28:11.894 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4652000000000002e-05 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783282+0000", 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0019961689999999999 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:31 2.d e24/18 2:b50e409b:::mds_snaptable:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:11.895 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.101478247000003, 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030643699999999999, 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:11.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782063+0000", 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3699999999999999e-07 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782072+0000", 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9430000000000006e-06 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.897 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782083+0000", 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0998e-05 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782092+0000", 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.3270000000000001e-06 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782158+0000", 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6830999999999998e-05 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782176+0000", 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7370000000000001e-05 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785091+0000", 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029157879999999999 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785127+0000", 2024-09-10T10:28:11.899 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5176000000000003e-05 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.101419355999994, 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002013515, 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.901 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782122+0000", 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.229e-06 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782137+0000", 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4895999999999999e-05 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782151+0000", 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3229e-05 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782198+0000", 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7301e-05 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782271+0000", 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3516000000000006e-05 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782290+0000", 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8921000000000001e-05 2024-09-10T10:28:11.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784094+0000", 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00180331 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784135+0000", 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1112999999999998e-05 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:28 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.101307743000007, 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028487970000000001, 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7000000000000005e-07 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782247+0000", 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3740000000000001e-05 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782258+0000", 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0552e-05 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782265+0000", 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0700000000000001e-06 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782752+0000", 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048707399999999999 2024-09-10T10:28:11.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782771+0000", 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9046999999999999e-05 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785037+0000", 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002265806 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785082+0000", 2024-09-10T10:28:11.908 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4938000000000002e-05 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:29 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.099761748000006, 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001932907, 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783780+0000", 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9900000000000009e-07 2024-09-10T10:28:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783789+0000", 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0140000000000004e-06 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783802+0000", 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2751e-05 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783811+0000", 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3950000000000007e-06 2024-09-10T10:28:11.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783894+0000", 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2887999999999997e-05 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783913+0000", 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9052999999999999e-05 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785668+0000", 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001754633 2024-09-10T10:28:11.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785712+0000", 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4174000000000001e-05 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:7 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.011129707999999, 2024-09-10T10:28:11.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026422899999999998, 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.561e-06 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872426+0000", 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3577999999999999e-05 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872462+0000", 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6145e-05 2024-09-10T10:28:11.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872473+0000", 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1109e-05 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872543+0000", 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9319000000000005e-05 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872566+0000", 2024-09-10T10:28:11.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3768999999999999e-05 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875010+0000", 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024433910000000001 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875053+0000", 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3417999999999997e-05 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.917 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:8 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.010441319000002, 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0044428820000000004, 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873100+0000", 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3030000000000005e-06 2024-09-10T10:28:11.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873121+0000", 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1471000000000001e-05 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873129+0000", 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7339999999999997e-06 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048786499999999998 2024-09-10T10:28:11.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873638+0000", 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0735000000000001e-05 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877493+0000", 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0038550189999999999 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877542+0000", 2024-09-10T10:28:11.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9135000000000003e-05 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:17 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.010223986, 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042663060000000001, 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.922 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873318+0000", 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7999999999999993e-07 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873326+0000", 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8279999999999992e-06 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.924 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.925 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873341+0000", 2024-09-10T10:28:11.925 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4688999999999999e-05 2024-09-10T10:28:11.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.925 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.925 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873348+0000", 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4810000000000003e-06 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873843+0000", 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000495083 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873860+0000", 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7609999999999999e-05 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877545+0000", 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036848839999999998 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877583+0000", 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7750999999999998e-05 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:18 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:11.927 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.009501948999997, 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035807780000000002, 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.928 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874045+0000", 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9909999999999999e-06 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874055+0000", 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0147e-05 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874070+0000", 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4184e-05 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874123+0000", 2024-09-10T10:28:11.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3504999999999997e-05 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874141+0000", 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7524999999999999e-05 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877586+0000", 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034458700000000002 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877619+0000", 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2962000000000001e-05 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.18 epoch 26/18 rep_tid 5 entries 26'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.9:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.008440303999997, 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048872359999999997, 2024-09-10T10:28:11.932 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875101+0000", 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875110+0000", 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2609999999999999e-06 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875129+0000", 2024-09-10T10:28:11.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8685000000000002e-05 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875143+0000", 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3723000000000001e-05 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875222+0000", 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9498000000000004e-05 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879987+0000", 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047652800000000002 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:7 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'5, mlcod=26'3)", 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 57.933678006999997, 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031425310000000001, 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.936 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.937 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1100000000000005e-07 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949871+0000", 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8620000000000003e-06 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949885+0000", 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4072e-05 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949893+0000", 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5769999999999996e-06 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949953+0000", 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.978e-05 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949972+0000", 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9188e-05 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.952959+0000", 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029873859999999999 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953005+0000", 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5855000000000001e-05 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.940 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:8 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'6, mlcod=26'3)", 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 57.933625417999998, 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034236359999999999, 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.941 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949916+0000", 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6199999999999998e-07 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949922+0000", 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5390000000000004e-06 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949929+0000", 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4339999999999997e-06 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949934+0000", 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2159999999999997e-06 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950192+0000", 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00025864699999999998 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950213+0000", 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0761999999999999e-05 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953297+0000", 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030842999999999999 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953339+0000", 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1276000000000003e-05 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.944 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:34 2.11 e26/18 2:8975f766:::202.00000001:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 57.929652578999999, 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020619190000000002, 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.945 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953889+0000", 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.879999999999999e-07 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.946 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953895+0000", 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2249999999999997e-06 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953906+0000", 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953912+0000", 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4840000000000003e-06 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953959+0000", 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7768000000000001e-05 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953976+0000", 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6365999999999999e-05 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955917+0000", 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001941524 2024-09-10T10:28:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955950+0000", 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2573999999999997e-05 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:4 2.1a e26/18 2:5fdc322d:::61e.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 51.960585051000002, 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0018931340000000001, 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.949 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.950 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922956+0000", 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-07 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922962+0000", 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1890000000000004e-06 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922971+0000", 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.951 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922978+0000", 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7830000000000001e-06 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923032+0000", 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3637999999999997e-05 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923049+0000", 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7249999999999999e-05 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924803+0000", 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001753788 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924849+0000", 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6050000000000001e-05 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:20 2.11 e26/18 2:890f50f7:::626.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 51.960477345000001, 2024-09-10T10:28:11.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003328012, 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923064+0000", 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9900000000000001e-07 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923070+0000", 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7499999999999997e-06 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923079+0000", 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9080000000000001e-06 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923089+0000", 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0078000000000001e-05 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923150+0000", 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0300999999999997e-05 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923507+0000", 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00035713599999999998 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926368+0000", 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028615469999999999 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926391+0000", 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2793e-05 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.957 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:23 2.18 e26/18 2:1c1db442:::103.00000000.inode:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 51.960417876000001, 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032793050000000002, 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923124+0000", 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0599999999999999e-07 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923130+0000", 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.81e-06 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923141+0000", 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923149+0000", 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2679999999999997e-06 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4744000000000002e-05 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923233+0000", 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9190000000000001e-05 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926353+0000", 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031206419999999999 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926402+0000", 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8755000000000003e-05 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:13 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'6, mlcod=26'2)", 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 51.960365246000002, 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031818969999999999, 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.962 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923176+0000", 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4799999999999998e-07 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923182+0000", 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.592e-06 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923191+0000", 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4029999999999996e-06 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923197+0000", 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1850000000000001e-06 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923230+0000", 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3290000000000001e-05 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923247+0000", 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6810999999999999e-05 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926308+0000", 2024-09-10T10:28:11.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030606159999999999 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926357+0000", 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9351999999999998e-05 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.d epoch 26/18 rep_tid 20 entries 26'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.15:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 51.959812747999997, 2024-09-10T10:28:11.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034028819999999999, 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923729+0000", 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.5900000000000005e-07 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923739+0000", 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8439999999999993e-06 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923761+0000", 2024-09-10T10:28:11.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2821999999999999e-05 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923776+0000", 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4763000000000001e-05 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923889+0000", 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000112518 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927131+0000", 2024-09-10T10:28:11.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032419760000000001 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:14 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'7, mlcod=26'2)", 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 51.959097786999997, 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027184890000000001, 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:11.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5099999999999999e-07 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924449+0000", 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0310000000000004e-06 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924482+0000", 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2845999999999997e-05 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924500+0000", 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.787e-05 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924594+0000", 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3566999999999994e-05 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924619+0000", 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4978999999999999e-05 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927118+0000", 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024995479999999999 2024-09-10T10:28:11.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:11.974 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:11.974 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:11.974 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927161+0000", 2024-09-10T10:28:11.974 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2997000000000002e-05 2024-09-10T10:28:11.974 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.974 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.974 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.974 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:11.974 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:11.974 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:11.976 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:11.976 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:11.974+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:12.076 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:12.075+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:12.121 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:12.121 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:12.121 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:12.121 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:12.130 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:12.177 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:12.176+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:12.198 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-10T10:28:12.273 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:12.273 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:12.273 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:12.273 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:12.273 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:12.278 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:12.277+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:12.282 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:12.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.308+0000 7fe7a6eb7640 1 Processor -- start 2024-09-10T10:28:12.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.308+0000 7fe7a6eb7640 1 -- start start 2024-09-10T10:28:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.309+0000 7fe7a6eb7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe7a015fa70 0x7fe7a00cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.309+0000 7fe7a6eb7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7a00c5600 con 0x7fe7a00cb2f0 2024-09-10T10:28:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.309+0000 7fe7a6eb7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7a015ef20 con 0x7fe7a00ccba0 2024-09-10T10:28:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.309+0000 7fe7a6eb7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe7a01596e0 con 0x7fe7a015fa70 2024-09-10T10:28:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.309+0000 7fe7a4c53640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe7a015fa70 0x7fe7a00cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.309+0000 7fe7a5454640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fe7a00cb2f0 0x7fe7a0169820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:44286/0 (socket says 172.21.15.120:44286) 2024-09-10T10:28:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.309+0000 7fe7a5454640 1 -- 172.21.15.120:0/791748767 learned_addr learned my addr 172.21.15.120:0/791748767 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.310+0000 7fe79f7fe640 1 -- 172.21.15.120:0/791748767 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3907081556 0 0) 0x7fe7a00c5600 con 0x7fe7a00cb2f0 2024-09-10T10:28:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.310+0000 7fe7a4c53640 1 -- 172.21.15.120:0/791748767 >> v1:172.21.15.134:6789/0 conn(0x7fe7a00ccba0 legacy=0x7fe7a00d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.310+0000 7fe7a4c53640 1 -- 172.21.15.120:0/791748767 >> v1:172.21.15.120:6789/0 conn(0x7fe7a00cb2f0 legacy=0x7fe7a0169820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.310+0000 7fe7a4c53640 1 -- 172.21.15.120:0/791748767 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a015fe50 con 0x7fe7a015fa70 2024-09-10T10:28:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.311+0000 7fe7a4c53640 1 --2- 172.21.15.120:0/791748767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a015fa70 0x7fe7a00cd770 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe794009720 tx=0x7fe79402a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be192f89d69e2c13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.311+0000 7fe79f7fe640 1 -- 172.21.15.120:0/791748767 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794037070 con 0x7fe7a015fa70 2024-09-10T10:28:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.311+0000 7fe79f7fe640 1 -- 172.21.15.120:0/791748767 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79402ade0 con 0x7fe7a015fa70 2024-09-10T10:28:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.311+0000 7fe79f7fe640 1 -- 172.21.15.120:0/791748767 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794032660 con 0x7fe7a015fa70 2024-09-10T10:28:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.312+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/791748767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a015fa70 msgr2=0x7fe7a00cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.312+0000 7fe7a6eb7640 1 --2- 172.21.15.120:0/791748767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a015fa70 0x7fe7a00cd770 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe794009720 tx=0x7fe79402a0d0 comp rx=0 tx=0).stop 2024-09-10T10:28:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.312+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/791748767 shutdown_connections 2024-09-10T10:28:12.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.312+0000 7fe7a6eb7640 1 --2- 172.21.15.120:0/791748767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a015fa70 0x7fe7a00cd770 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:12.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.312+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/791748767 >> 172.21.15.120:0/791748767 conn(0x7fe7a006d560 msgr2=0x7fe7a006d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:12.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.312+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/791748767 shutdown_connections 2024-09-10T10:28:12.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.312+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/791748767 wait complete. 2024-09-10T10:28:12.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.314+0000 7fe7a6eb7640 1 Processor -- start 2024-09-10T10:28:12.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.314+0000 7fe7a6eb7640 1 -- start start 2024-09-10T10:28:12.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe7a6eb7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7a00cb2f0 0x7fe7a00c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:12.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe7a6eb7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a00ccba0 0x7fe7a01f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:12.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe7a6eb7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7a015fa70 0x7fe7a0151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:12.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe7a6eb7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a00c5b50 con 0x7fe7a00ccba0 2024-09-10T10:28:12.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe7a6eb7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a00cd2e0 con 0x7fe7a00cb2f0 2024-09-10T10:28:12.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe7a6eb7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a00cb830 con 0x7fe7a015fa70 2024-09-10T10:28:12.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe79ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a00ccba0 0x7fe7a01f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe7a4c53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7a00cb2f0 0x7fe7a00c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe79ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a00ccba0 0x7fe7a01f8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57056/0 (socket says 172.21.15.120:57056) 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe79ffff640 1 -- 172.21.15.120:0/3501476536 learned_addr learned my addr 172.21.15.120:0/3501476536 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.315+0000 7fe7a5454640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7a015fa70 0x7fe7a0151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe79ffff640 1 -- 172.21.15.120:0/3501476536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7a015fa70 msgr2=0x7fe7a0151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe79ffff640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7a015fa70 0x7fe7a0151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe79ffff640 1 -- 172.21.15.120:0/3501476536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7a00cb2f0 msgr2=0x7fe7a00c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe79ffff640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7a00cb2f0 0x7fe7a00c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe79ffff640 1 -- 172.21.15.120:0/3501476536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a015bba0 con 0x7fe7a00ccba0 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe79ffff640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a00ccba0 0x7fe7a01f8280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c7b0 tx=0x7fe78c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe79d7fa640 1 -- 172.21.15.120:0/3501476536 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c005270 con 0x7fe7a00ccba0 2024-09-10T10:28:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe79d7fa640 1 -- 172.21.15.120:0/3501476536 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c005430 con 0x7fe7a00ccba0 2024-09-10T10:28:12.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/3501476536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a015c8c0 con 0x7fe7a00ccba0 2024-09-10T10:28:12.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe79d7fa640 1 -- 172.21.15.120:0/3501476536 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00a880 con 0x7fe7a00ccba0 2024-09-10T10:28:12.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.316+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/3501476536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a015c5e0 con 0x7fe7a00ccba0 2024-09-10T10:28:12.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.317+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/3501476536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a00b3270 con 0x7fe7a00ccba0 2024-09-10T10:28:12.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.318+0000 7fe79d7fa640 1 -- 172.21.15.120:0/3501476536 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe78c00aa20 con 0x7fe7a00ccba0 2024-09-10T10:28:12.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.319+0000 7fe79d7fa640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe78406be30 0x7fe78406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:12.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.319+0000 7fe79d7fa640 1 -- 172.21.15.120:0/3501476536 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6801+0+0 (secure 0 0 0) 0x7fe78c08ae80 con 0x7fe7a00ccba0 2024-09-10T10:28:12.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.319+0000 7fe7a4c53640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe78406be30 0x7fe78406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:12.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.319+0000 7fe7a4c53640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe78406be30 0x7fe78406e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe794034000 tx=0x7fe79402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:12.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.327+0000 7fe79d7fa640 1 -- 172.21.15.120:0/3501476536 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe78c0012b0 con 0x7fe7a00ccba0 2024-09-10T10:28:12.378 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:12.378+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:12.387 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:12.387 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:12.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:12.387 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:12.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.513099441000001, 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.006807553, 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872290+0000", 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2029999999999999e-06 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872310+0000", 2024-09-10T10:28:12.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0508000000000001e-05 2024-09-10T10:28:12.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872331+0000", 2024-09-10T10:28:12.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0577e-05 2024-09-10T10:28:12.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872341+0000", 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0421999999999999e-05 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872457+0000", 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115618 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872575+0000", 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118563 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:12.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872690+0000", 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011438900000000001 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875281+0000", 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002590995 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879035+0000", 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037544879999999998 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879069+0000", 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4034e-05 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879095+0000", 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5755999999999999e-05 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.395 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.511902820000003, 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0061030149999999998, 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873485+0000", 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2419999999999999e-06 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.669e-06 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873505+0000", 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.325e-05 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873514+0000", 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1949999999999992e-06 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873575+0000", 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873647+0000", 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2346000000000002e-05 2024-09-10T10:28:12.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873778+0000", 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013125600000000001 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875444+0000", 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001665962 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879528+0000", 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040837809999999999 2024-09-10T10:28:12.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879563+0000", 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5329e-05 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879587+0000", 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3947999999999998e-05 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.51184877, 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096274789999999996, 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873539+0000", 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0299999999999998e-07 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873554+0000", 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5481999999999998e-05 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873565+0000", 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0733e-05 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873573+0000", 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9270000000000005e-06 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873633+0000", 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9864000000000001e-05 2024-09-10T10:28:12.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873752+0000", 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118946 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873866+0000", 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000113757 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875462+0000", 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015966699999999999 2024-09-10T10:28:12.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883084+0000", 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076214280000000004 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883136+0000", 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2182000000000003e-05 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883166+0000", 2024-09-10T10:28:12.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.9787000000000001e-05 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.511790799000003, 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097094669999999994, 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873597+0000", 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8899999999999998e-07 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873602+0000", 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9230000000000001e-06 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873611+0000", 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9150000000000002e-06 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5239999999999997e-06 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.409 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874719+0000", 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0011012769999999999 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874791+0000", 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2305000000000007e-05 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874962+0000", 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000171099 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.410 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877786+0000", 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028241059999999998 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883211+0000", 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054250019999999999 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883263+0000", 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1563000000000001e-05 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883306+0000", 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2864000000000002e-05 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.510870109000003, 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005164681, 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874518+0000", 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3799999999999996e-07 2024-09-10T10:28:12.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874521+0000", 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8550000000000004e-06 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874531+0000", 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8079999999999999e-06 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874538+0000", 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2760000000000003e-06 2024-09-10T10:28:12.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874945+0000", 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000406516 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875045+0000", 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9570999999999999e-05 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879682+0000", 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046369169999999999 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.415 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 63.510836214000001, 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056651770000000004, 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.416 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4600000000000005e-07 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.417 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874565+0000", 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4017e-05 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874573+0000", 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9689999999999994e-06 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874678+0000", 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7483000000000002e-05 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874786+0000", 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107914 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.880216+0000", 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005429499 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.419 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:24 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 58.436527890999997, 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0079888200000000006, 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:12.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948861+0000", 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9269999999999999e-06 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948869+0000", 2024-09-10T10:28:12.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7749999999999993e-06 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948884+0000", 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5713e-05 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948894+0000", 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2050000000000008e-06 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948958+0000", 2024-09-10T10:28:12.422 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4398000000000002e-05 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949093+0000", 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013508300000000001 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949223+0000", 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012949500000000001 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953128+0000", 2024-09-10T10:28:12.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039058360000000002 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956775+0000", 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036467439999999999 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956812+0000", 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6510000000000001e-05 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956848+0000", 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6134000000000003e-05 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:23 2.c e26/18 2:3631e767:::102.00000000.inode:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 58.435043641, 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054176620000000002, 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.426 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950344+0000", 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9400000000000001e-07 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950362+0000", 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8655999999999999e-05 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950384+0000", 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1929999999999998e-05 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950391+0000", 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7020000000000004e-06 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950624+0000", 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000233031 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950643+0000", 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9459000000000001e-05 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955726+0000", 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050826389999999999 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4851000000000002e-05 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 52.464394656000003, 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0070980829999999998, 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920994+0000", 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.697e-06 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.431 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921001+0000", 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3719999999999999e-06 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921014+0000", 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2529e-05 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921021+0000", 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9519999999999997e-06 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921097+0000", 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6499000000000001e-05 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921261+0000", 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016399599999999999 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921396+0000", 2024-09-10T10:28:12.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013448400000000001 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922936+0000", 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015405970000000001 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927988+0000", 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00505158 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928047+0000", 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9011000000000001e-05 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928090+0000", 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3365999999999998e-05 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:12.435 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 52.464343145000001, 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007356853, 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.436 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5699999999999998e-07 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921048+0000", 2024-09-10T10:28:12.437 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.754e-06 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921056+0000", 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0469999999999994e-06 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921063+0000", 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5980000000000002e-06 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921747+0000", 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00068475800000000005 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921852+0000", 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104347 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922002+0000", 2024-09-10T10:28:12.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000150233 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924357+0000", 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0023545599999999999 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928286+0000", 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039299540000000003 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.440 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928349+0000", 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2044000000000004e-05 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928401+0000", 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2101000000000001e-05 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.441 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 52.464303686999997, 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005625166, 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.442 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921084+0000", 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0100000000000005e-07 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921093+0000", 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9910000000000008e-06 2024-09-10T10:28:12.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921099+0000", 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2269999999999998e-06 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921103+0000", 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0929999999999996e-06 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921167+0000", 2024-09-10T10:28:12.444 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4030999999999997e-05 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921301+0000", 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013388699999999999 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921471+0000", 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017000899999999999 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924553+0000", 2024-09-10T10:28:12.445 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030818199999999999 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926600+0000", 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020471339999999999 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926663+0000", 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3274000000000004e-05 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926708+0000", 2024-09-10T10:28:12.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5099000000000003e-05 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.441752016999999, 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010012200000000001, 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.447 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943637+0000", 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5390000000000002e-06 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943646+0000", 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1340000000000004e-06 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943661+0000", 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4958e-05 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943669+0000", 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5629999999999992e-06 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.449 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943785+0000", 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011584800000000001 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943960+0000", 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000174611 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944078+0000", 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118381 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.450 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948022+0000", 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039442599999999998 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953546+0000", 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055242449999999997 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953604+0000", 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7555e-05 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953647+0000", 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3105999999999997e-05 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.441558458000003, 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099609380000000008, 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.452 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.453 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943829+0000", 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3099999999999996e-07 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943835+0000", 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8610000000000003e-06 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943846+0000", 2024-09-10T10:28:12.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0686999999999999e-05 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943854+0000", 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944413+0000", 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055893399999999997 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944505+0000", 2024-09-10T10:28:12.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2788999999999995e-05 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944639+0000", 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013387699999999999 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950701+0000", 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0060612360000000002 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953700+0000", 2024-09-10T10:28:12.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029997689999999998 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953743+0000", 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3024e-05 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953789+0000", 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6109999999999997e-05 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.457 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.441518436000003, 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054304089999999998, 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.458 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1399999999999996e-07 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943874+0000", 2024-09-10T10:28:12.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3830000000000002e-06 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943881+0000", 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1369999999999996e-06 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943886+0000", 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-06 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944057+0000", 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000170781 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944182+0000", 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012506299999999999 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944309+0000", 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127081 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.461 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.947998+0000", 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003689323 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949225+0000", 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001227005 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949267+0000", 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1391999999999999e-05 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.462 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949299+0000", 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2169999999999999e-05 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.441457954000001, 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0068819399999999996, 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.463 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943930+0000", 2024-09-10T10:28:12.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1200000000000002e-07 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943934+0000", 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0969999999999999e-06 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944782+0000", 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00084778299999999996 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.465 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944793+0000", 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0993e-05 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944864+0000", 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1241000000000005e-05 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944940+0000", 2024-09-10T10:28:12.466 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6044999999999994e-05 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945068+0000", 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012846 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949683+0000", 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046151040000000001 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.467 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950706+0000", 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010222149999999999 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950764+0000", 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8180999999999997e-05 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950811+0000", 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7108999999999999e-05 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.468 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:24 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.440084327999998, 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055052950000000003, 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945304+0000", 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4799999999999997e-07 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945311+0000", 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1269999999999997e-06 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945322+0000", 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1134e-05 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945330+0000", 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9039999999999992e-06 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945408+0000", 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8200000000000003e-05 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945533+0000", 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000125518 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:12.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945659+0000", 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012583599999999999 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949147+0000", 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003487675 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950716+0000", 2024-09-10T10:28:12.473 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015695279999999999 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950767+0000", 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0577e-05 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950808+0000", 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0847999999999999e-05 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.474 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:25 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.439442958999997, 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050290129999999997, 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.475 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945945+0000", 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1119999999999999e-06 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.476 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945952+0000", 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5189999999999998e-06 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945965+0000", 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3643e-05 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945974+0000", 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5750000000000001e-06 2024-09-10T10:28:12.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0532999999999998e-05 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946192+0000", 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015717699999999999 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946341+0000", 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014897200000000001 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949885+0000", 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035449729999999999 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950896+0000", 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00101023 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950938+0000", 2024-09-10T10:28:12.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2284999999999999e-05 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950973+0000", 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4993999999999997e-05 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.439389886999997, 2024-09-10T10:28:12.480 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048920780000000002, 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945998+0000", 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3900000000000004e-07 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946012+0000", 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4799e-05 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946021+0000", 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.6000000000000007e-06 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946029+0000", 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9209999999999992e-06 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946093+0000", 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3608000000000005e-05 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946242+0000", 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014889699999999999 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950889+0000", 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046476140000000004 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.439345860000003, 2024-09-10T10:28:12.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047976679999999997, 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:12.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946042+0000", 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5199999999999996e-07 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946046+0000", 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1439999999999996e-06 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.486 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946054+0000", 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1140000000000006e-06 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946070+0000", 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6232e-05 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946137+0000", 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6956000000000001e-05 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946260+0000", 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012240299999999999 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950839+0000", 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045791670000000003 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:23 2.1c e28/18 2:399c8d31:::104.00000000.inode:head v 28'5, mlcod=24'1)", 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.438826521000003, 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042418079999999997, 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.489 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3e-07 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946574+0000", 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.277e-05 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946584+0000", 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7710000000000001e-06 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946590+0000", 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6769999999999998e-06 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946666+0000", 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5420999999999995e-05 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946687+0000", 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1665999999999999e-05 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950762+0000", 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040749030000000004 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950802+0000", 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9969999999999998e-05 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.492 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.493 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.493 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.493 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.493 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:12.494 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:12.495 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:12.493+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:12.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:12.509+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/3501476536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fe7a00cddd0 con 0x7fe7a00ccba0 2024-09-10T10:28:12.596 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:12.596+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:12.637 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:12.637 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:12.637 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:12.637 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:12.645 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:12.730 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:12.729+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:12.789 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:12.789 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:12.790 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:12.790 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:12.790 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:12.800 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:12.801 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:12.797+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:12.898 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:12.897+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:12.944 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.163136902999995, 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:28:12.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:28:12.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:28:12.950 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.162806403000005, 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.951 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.162759859999994, 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.956 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:28:12.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:28:12.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 64.071317648000004, 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.960 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:28:12.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:12.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.965 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 64.071236304999999, 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:28:12.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:28:12.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 64.071209749999994, 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.971 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:12.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.974 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:28:12.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:28:12.976 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 64.071067100999997, 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.977 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:12.978 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:28:12.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:28:12.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:28:12.981 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 64.068420168000003, 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.982 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.983 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.984 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.985 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:12.986 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 64.067569427999999, 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.987 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:28:12.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:28:12.990 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 58.995417744999997, 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.991 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.992 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:28:12.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:28:12.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:28:12.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.996 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 58.995368609000003, 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.997 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:28:12.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:12.999 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.000 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.001 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:28:13.001 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:28:13.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.001 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.001 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.001 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:28:13.004 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.005 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 58.995257131000002, 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.006 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:28:13.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.008 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:28:13.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 53.021856612000001, 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.010 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.011 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.012 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:13.015 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 53.021826441000002, 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.016 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.017 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.018 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:28:13.020 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 53.021769949000003, 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:28:13.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:28:13.023 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.024 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:28:13.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.027 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:13.027 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:13.027 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 53.021668781000002, 2024-09-10T10:28:13.027 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:28:13.027 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:28:13.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:28:13.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:28:13.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.032 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 47.001401684000001, 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.033 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.034 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.035 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:28:13.037 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.997336464999997, 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:28:13.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:28:13.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:13.042 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.997318595000003, 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:13.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.044 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:28:13.046 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 46.997272357, 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.047 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.048 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:28:13.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.051 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:13.053 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:13.053 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:13.052+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:13.153 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:13.152+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.193+0000 7fe79d7fa640 1 -- 172.21.15.120:0/3501476536 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v46) ==== 87+0+0 (secure 0 0 0) 0x7fe78c058e20 con 0x7fe7a00ccba0 2024-09-10T10:28:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.195+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/3501476536 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe78406be30 msgr2=0x7fe78406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.195+0000 7fe7a6eb7640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe78406be30 0x7fe78406e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe794034000 tx=0x7fe79402e000 comp rx=0 tx=0).stop 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.196+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/3501476536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a00ccba0 msgr2=0x7fe7a01f8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.196+0000 7fe7a6eb7640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a00ccba0 0x7fe7a01f8280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c7b0 tx=0x7fe78c00cc70 comp rx=0 tx=0).stop 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.196+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/3501476536 shutdown_connections 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.196+0000 7fe7a6eb7640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe78406be30 0x7fe78406e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.196+0000 7fe7a6eb7640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7a015fa70 0x7fe7a0151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.196+0000 7fe7a6eb7640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a00ccba0 0x7fe7a01f8280 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c7b0 tx=0x7fe78c00cc70 comp rx=0 tx=0).stop 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.196+0000 7fe7a6eb7640 1 --2- 172.21.15.120:0/3501476536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7a00cb2f0 0x7fe7a00c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.196+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/3501476536 >> 172.21.15.120:0/3501476536 conn(0x7fe7a006d560 msgr2=0x7fe7a00cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.196+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/3501476536 shutdown_connections 2024-09-10T10:28:13.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.196+0000 7fe7a6eb7640 1 -- 172.21.15.120:0/3501476536 wait complete. 2024-09-10T10:28:13.201 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:13.201 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:13.201 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:13.201 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:13.210 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:13.254 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:13.254+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:13.314 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:13.314 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:13.314 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:13.314 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:13.314 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:13.323 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:13.355 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:13.354+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:13.457 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:13.456+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:13.467 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:13.467 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:13.467 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:13.467 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:13.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.467 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.467 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:13.467 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 101.36852435, 2024-09-10T10:28:13.467 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.17492211699999999, 2024-09-10T10:28:13.467 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 1 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096318+0000", 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.657e-06 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096326+0000", 2024-09-10T10:28:13.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0560000000000005e-06 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096339+0000", 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2965999999999999e-05 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096349+0000", 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0784e-05 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096432+0000", 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2657999999999994e-05 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op must wait for map", 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096441+0000", 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4590000000000004e-06 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096935+0000", 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049331899999999998 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for peered", 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096944+0000", 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9539999999999993e-06 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267505+0000", 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.170561622 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267700+0000", 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019420400000000001 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267923+0000", 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000223451 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.270575+0000", 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026522030000000001 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271131+0000", 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055599000000000002 2024-09-10T10:28:13.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271191+0000", 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9947000000000001e-05 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271238+0000", 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6847000000000001e-05 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.474 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.97969010600001, 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078035889999999997, 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 115 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.475 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1300000000000001e-07 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485209+0000", 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8628999999999997e-05 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485214+0000", 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0849999999999996e-06 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485218+0000", 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3840000000000001e-06 2024-09-10T10:28:13.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490185+0000", 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0049675270000000002 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490231+0000", 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5488000000000002e-05 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490307+0000", 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6038000000000006e-05 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491263+0000", 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095625099999999998 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492908+0000", 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001645355 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.479 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492934+0000", 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5431000000000001e-05 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492954+0000", 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0088000000000001e-05 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.480 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.97961561699999, 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077741099999999999, 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 116 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.481 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1100000000000002e-07 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485227+0000", 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1890000000000001e-06 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485231+0000", 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4010000000000001e-06 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485233+0000", 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.728e-06 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490628+0000", 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0053952109999999996 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490674+0000", 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5946999999999999e-05 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490750+0000", 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5227000000000003e-05 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491443+0000", 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069346800000000004 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492959+0000", 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015154470000000001 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492983+0000", 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.406e-05 2024-09-10T10:28:13.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492999+0000", 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6121e-05 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.979603489, 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078059490000000004, 2024-09-10T10:28:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 117 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.23e-07 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485240+0000", 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780000000000001e-06 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485243+0000", 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8430000000000003e-06 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485247+0000", 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2279999999999998e-06 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490995+0000", 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057480730000000002 2024-09-10T10:28:13.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491043+0000", 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7936000000000003e-05 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491113+0000", 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0872999999999997e-05 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491636+0000", 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00052242300000000005 2024-09-10T10:28:13.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493002+0000", 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001365808 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493025+0000", 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2932999999999999e-05 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493043+0000", 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8031000000000001e-05 2024-09-10T10:28:13.491 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.647248347, 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095639920000000003, 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 131 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1199999999999998e-07 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817663+0000", 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1099000000000005e-05 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817670+0000", 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.8539999999999999e-06 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817674+0000", 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.574e-06 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818465+0000", 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000791445 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818509+0000", 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3569000000000002e-05 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818580+0000", 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1503999999999998e-05 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821292+0000", 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027120529999999999 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827108+0000", 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0058160759999999999 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827134+0000", 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5859000000000001e-05 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827156+0000", 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1546999999999999e-05 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.497 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.647159139, 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095198769999999995, 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 132 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.498 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:13.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2899999999999999e-07 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817683+0000", 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0379999999999998e-06 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817690+0000", 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3110000000000002e-06 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.500 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817699+0000", 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820828+0000", 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031296829999999999 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820873+0000", 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4869e-05 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.501 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820943+0000", 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9919999999999995e-05 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823683+0000", 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027395140000000002 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827161+0000", 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003477857 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.502 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827183+0000", 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2612999999999998e-05 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827201+0000", 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7842999999999998e-05 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.503 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.647091832, 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0094944009999999995, 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 133 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.504 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817749+0000", 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2399999999999998e-07 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.505 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817752+0000", 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1970000000000001e-06 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817760+0000", 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8080000000000005e-06 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817764+0000", 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9419999999999997e-06 2024-09-10T10:28:13.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821319+0000", 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035547740000000001 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821364+0000", 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5398000000000001e-05 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821434+0000", 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0382999999999993e-05 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823717+0000", 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022825089999999998 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827205+0000", 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034879009999999998 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.508 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827225+0000", 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0611999999999998e-05 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827243+0000", 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7353000000000001e-05 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:13.509 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.647038991, 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095863660000000007, 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 134 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:13.510 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817802+0000", 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8999999999999996e-07 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.511 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817871+0000", 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9572000000000004e-05 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817878+0000", 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2079999999999997e-06 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817882+0000", 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5889999999999996e-06 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821625+0000", 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037431069999999999 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821669+0000", 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3455000000000003e-05 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821740+0000", 2024-09-10T10:28:13.513 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1390999999999995e-05 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823826+0000", 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020863399999999999 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827343+0000", 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003516267 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827369+0000", 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.6044999999999999e-05 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827388+0000", 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8802000000000001e-05 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.515 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.646946296, 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095738179999999996, 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 135 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:13.517 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5999999999999999e-07 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817897+0000", 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0149999999999999e-06 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817904+0000", 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5089999999999999e-06 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.518 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817909+0000", 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3120000000000001e-06 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823862+0000", 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059524199999999999 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823911+0000", 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9323000000000002e-05 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823983+0000", 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1736000000000006e-05 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827101+0000", 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031181009999999999 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827392+0000", 2024-09-10T10:28:13.520 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00029090700000000003 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827426+0000", 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4127e-05 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827468+0000", 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1907999999999999e-05 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.521 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.64691928000001, 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.009594488, 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 136 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.522 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.523 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3700000000000001e-07 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817925+0000", 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4659999999999999e-06 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817931+0000", 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5710000000000003e-06 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.524 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817936+0000", 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7550000000000002e-06 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824178+0000", 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062415700000000001 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824225+0000", 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6966999999999999e-05 2024-09-10T10:28:13.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824296+0000", 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1284000000000004e-05 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827290+0000", 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002993697 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827473+0000", 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018376699999999999 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827497+0000", 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3445e-05 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827515+0000", 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8529000000000002e-05 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.527 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.646887591, 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096066220000000004, 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.528 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.529 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.529 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 137 2024-09-10T10:28:13.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.529 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.529 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.529 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:13.529 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3000000000000001e-07 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.530 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818004+0000", 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0902000000000002e-05 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818013+0000", 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5420000000000005e-06 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818019+0000", 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7939999999999997e-06 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824489+0000", 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0064692580000000003 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824532+0000", 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3631000000000003e-05 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824601+0000", 2024-09-10T10:28:13.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9028000000000004e-05 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827334+0000", 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027325409999999998 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827520+0000", 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018642499999999999 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.533 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827541+0000", 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1151000000000001e-05 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827559+0000", 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7819999999999999e-05 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.64680826999999, 2024-09-10T10:28:13.534 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010100303, 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 138 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818033+0000", 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.536 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818037+0000", 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0419999999999996e-06 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818045+0000", 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9759999999999995e-06 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818052+0000", 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5909999999999998e-06 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.537 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826594+0000", 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0085421660000000003 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826671+0000", 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7121999999999998e-05 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826795+0000", 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012315199999999999 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828081+0000", 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0012869139999999999 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828088+0000", 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9870000000000002e-06 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828112+0000", 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3405e-05 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828132+0000", 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0394000000000002e-05 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.646777899, 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01125819, 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.540 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 139 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.541 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818063+0000", 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7800000000000002e-07 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818067+0000", 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8500000000000004e-06 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818075+0000", 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818081+0000", 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1829999999999999e-06 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.543 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827594+0000", 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095129340000000007 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827669+0000", 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5239000000000002e-05 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827789+0000", 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000119757 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829195+0000", 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014064279999999999 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829232+0000", 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7058999999999999e-05 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829281+0000", 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8884000000000002e-05 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.545 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829321+0000", 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9527999999999998e-05 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.64674383000001, 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012263886, 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 140 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818097+0000", 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8999999999999997e-07 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818145+0000", 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8040000000000001e-05 2024-09-10T10:28:13.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818160+0000", 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4671999999999999e-05 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818169+0000", 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1900000000000001e-06 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828158+0000", 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099891440000000001 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828228+0000", 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0407000000000005e-05 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828343+0000", 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000114821 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829914+0000", 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001570866 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830312+0000", 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00039770499999999999 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830339+0000", 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.69e-05 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830360+0000", 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1651e-05 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.646656589, 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012742035000000001, 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.552 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 141 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:13.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7700000000000004e-07 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818188+0000", 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8659999999999999e-06 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818196+0000", 2024-09-10T10:28:13.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9480000000000008e-06 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818205+0000", 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8570000000000001e-06 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828883+0000", 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010678600999999999 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828927+0000", 2024-09-10T10:28:13.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3887999999999997e-05 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829001+0000", 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3442000000000006e-05 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830548+0000", 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001547194 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830884+0000", 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00033638100000000002 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830907+0000", 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3051999999999999e-05 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830926+0000", 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8229000000000001e-05 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.63350885200001, 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097270880000000001, 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 145 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831332+0000", 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8900000000000001e-07 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831379+0000", 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7134e-05 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831387+0000", 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7409999999999992e-06 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831391+0000", 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6190000000000002e-06 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833155+0000", 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017641969999999999 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833237+0000", 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1957000000000002e-05 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833356+0000", 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011843100000000001 2024-09-10T10:28:13.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.834068+0000", 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00071179000000000001 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841004+0000", 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069359749999999996 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841034+0000", 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.042e-05 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841058+0000", 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4535000000000001e-05 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:20 2.9 e26/18 2:92e99a1d:::612.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 64.589312587999999, 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074964979999999999, 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.563 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875534+0000", 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9109999999999998e-06 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875543+0000", 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8319999999999995e-06 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875549+0000", 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2380000000000002e-06 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875699+0000", 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014945900000000001 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875717+0000", 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8584e-05 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882982+0000", 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0072648039999999997 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883024+0000", 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2135000000000002e-05 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:27 2.9 e26/18 2:91bd965c:::501.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:13.567 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 64.589251954000005, 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074993129999999996, 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.568 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875589+0000", 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3999999999999997e-07 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875595+0000", 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9680000000000003e-06 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.569 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875602+0000", 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875608+0000", 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0430000000000004e-06 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875887+0000", 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000279578 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875904+0000", 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6439e-05 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883027+0000", 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0071234480000000001 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883088+0000", 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0424000000000003e-05 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.571 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 47.523172877, 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01193195, 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:13.572 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2399999999999999e-07 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.573 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941671+0000", 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8329999999999998e-06 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2919999999999998e-06 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941686+0000", 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944874+0000", 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031881639999999998 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945305+0000", 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00043084599999999997 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945438+0000", 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013344600000000001 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950653+0000", 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005214733 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953496+0000", 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028433989999999999 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953554+0000", 2024-09-10T10:28:13.576 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7933000000000003e-05 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953599+0000", 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5130999999999999e-05 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 47.523146930999999, 2024-09-10T10:28:13.577 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0093592050000000006, 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941694+0000", 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7899999999999999e-07 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941711+0000", 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7530999999999999e-05 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:13.579 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941720+0000", 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4279999999999996e-06 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941725+0000", 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2390000000000002e-06 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942690+0000", 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00096541400000000005 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942807+0000", 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000116777 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942967+0000", 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015997599999999999 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944708+0000", 2024-09-10T10:28:13.581 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001740492 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950956+0000", 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062481580000000002 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951016+0000", 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0299e-05 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951052+0000", 2024-09-10T10:28:13.582 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6412000000000003e-05 2024-09-10T10:28:13.583 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.583 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.583 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.583 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:13.583 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:13.583 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:13.584 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:13.583+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:13.585 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:13.685 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:13.684+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:13.694 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:13.694 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:13.703 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:13.754 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:28:13.785 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:13.784+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:13.879 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:13.879 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:13.879 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:13.879 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:13.879 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:13.886 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:13.885+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:13.890 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:13.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.931+0000 7f2d89f31640 1 Processor -- start 2024-09-10T10:28:13.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.931+0000 7f2d89f31640 1 -- start start 2024-09-10T10:28:13.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d89f31640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2d84154c10 0x7f2d8415b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:13.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d89f31640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2d840cca70 0x7f2d84073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:13.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d82ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2d840cca70 0x7f2d84073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:13.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d82ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2d840cca70 0x7f2d84073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57064/0 (socket says 172.21.15.120:57064) 2024-09-10T10:28:13.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d82ffd640 1 -- 172.21.15.120:0/2790357804 learned_addr learned my addr 172.21.15.120:0/2790357804 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:13.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d89f31640 1 --2- 172.21.15.120:0/2790357804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cafa0 0x7f2d84073880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:13.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d89f31640 1 -- 172.21.15.120:0/2790357804 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f2d8415de60 con 0x7f2d840cca70 2024-09-10T10:28:13.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d89f31640 1 -- 172.21.15.120:0/2790357804 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2d8415e0b0 con 0x7f2d84154c10 2024-09-10T10:28:13.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d89f31640 1 -- 172.21.15.120:0/2790357804 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8415e390 con 0x7f2d840cafa0 2024-09-10T10:28:13.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d837fe640 1 --2- 172.21.15.120:0/2790357804 >> v2:172.21.15.134:3300/0 conn(0x7f2d84154c10 0x7f2d8415b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:13.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d83fff640 1 --2- 172.21.15.120:0/2790357804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cafa0 0x7f2d84073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:13.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d837fe640 1 -- 172.21.15.120:0/2790357804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cafa0 msgr2=0x7f2d84073880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:13.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d837fe640 1 --2- 172.21.15.120:0/2790357804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cafa0 0x7f2d84073880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d837fe640 1 -- 172.21.15.120:0/2790357804 >> v2:172.21.15.120:3300/0 conn(0x7f2d840cca70 msgr2=0x7f2d84073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:13.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d837fe640 1 --2- 172.21.15.120:0/2790357804 >> v2:172.21.15.120:3300/0 conn(0x7f2d840cca70 0x7f2d84073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d837fe640 1 -- 172.21.15.120:0/2790357804 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8415e670 con 0x7f2d84154c10 2024-09-10T10:28:13.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.933+0000 7f2d82ffd640 1 --2- 172.21.15.120:0/2790357804 >> v2:172.21.15.120:3300/0 conn(0x7f2d840cca70 0x7f2d84073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:13.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.935+0000 7f2d837fe640 1 --2- 172.21.15.120:0/2790357804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d84154c10 0x7f2d8415b370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d8406df90 tx=0x7f2d7402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c2b5622d1b9703 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:13.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.935+0000 7f2d827fc640 1 -- 172.21.15.120:0/2790357804 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74031600 con 0x7f2d84154c10 2024-09-10T10:28:13.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.935+0000 7f2d827fc640 1 -- 172.21.15.120:0/2790357804 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d740317a0 con 0x7f2d84154c10 2024-09-10T10:28:13.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.935+0000 7f2d827fc640 1 -- 172.21.15.120:0/2790357804 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74031600 con 0x7f2d84154c10 2024-09-10T10:28:13.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.936+0000 7f2d89f31640 1 -- 172.21.15.120:0/2790357804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d84154c10 msgr2=0x7f2d8415b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:13.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.936+0000 7f2d89f31640 1 --2- 172.21.15.120:0/2790357804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d84154c10 0x7f2d8415b370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d8406df90 tx=0x7f2d7402a5d0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.937+0000 7f2d89f31640 1 -- 172.21.15.120:0/2790357804 shutdown_connections 2024-09-10T10:28:13.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.937+0000 7f2d89f31640 1 --2- 172.21.15.120:0/2790357804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cafa0 0x7f2d84073880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.937+0000 7f2d89f31640 1 --2- 172.21.15.120:0/2790357804 >> v2:172.21.15.120:3300/0 conn(0x7f2d840cca70 0x7f2d84073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.937+0000 7f2d89f31640 1 --2- 172.21.15.120:0/2790357804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d84154c10 0x7f2d8415b370 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.937+0000 7f2d89f31640 1 -- 172.21.15.120:0/2790357804 >> 172.21.15.120:0/2790357804 conn(0x7f2d8406ce80 msgr2=0x7f2d8406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:13.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.937+0000 7f2d89f31640 1 -- 172.21.15.120:0/2790357804 shutdown_connections 2024-09-10T10:28:13.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.937+0000 7f2d89f31640 1 -- 172.21.15.120:0/2790357804 wait complete. 2024-09-10T10:28:13.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.939+0000 7f2d89f31640 1 Processor -- start 2024-09-10T10:28:13.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.939+0000 7f2d89f31640 1 -- start start 2024-09-10T10:28:13.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d89f31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d840cafa0 0x7f2d840c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:13.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d89f31640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cca70 0x7f2d840c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:13.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d840cafa0 0x7f2d840c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:13.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d840cafa0 0x7f2d840c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57070/0 (socket says 172.21.15.120:57070) 2024-09-10T10:28:13.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d837fe640 1 -- 172.21.15.120:0/947384369 learned_addr learned my addr 172.21.15.120:0/947384369 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:13.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d89f31640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d84154c10 0x7f2d841f06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:13.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d89f31640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84154ff0 con 0x7f2d840cafa0 2024-09-10T10:28:13.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d82ffd640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cca70 0x7f2d840c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:13.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d89f31640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8415ec30 con 0x7f2d84154c10 2024-09-10T10:28:13.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d89f31640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2d840cb6f0 con 0x7f2d840cca70 2024-09-10T10:28:13.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.941+0000 7f2d83fff640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d84154c10 0x7f2d841f06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:13.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d82ffd640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d84154c10 msgr2=0x7f2d841f06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:13.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d82ffd640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d84154c10 0x7f2d841f06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d82ffd640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d840cafa0 msgr2=0x7f2d840c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:13.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d82ffd640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d840cafa0 0x7f2d840c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:13.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d82ffd640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d840c9880 con 0x7f2d840cca70 2024-09-10T10:28:13.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d82ffd640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cca70 0x7f2d840c7c10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009fd0 tx=0x7f2d7800ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:13.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78004030 con 0x7f2d840cca70 2024-09-10T10:28:13.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d780041d0 con 0x7f2d840cca70 2024-09-10T10:28:13.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800c9a0 con 0x7f2d840cca70 2024-09-10T10:28:13.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d89f31640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d84150dc0 con 0x7f2d840cca70 2024-09-10T10:28:13.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.942+0000 7f2d89f31640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d840c9af0 con 0x7f2d840cca70 2024-09-10T10:28:13.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.944+0000 7f2d89f31640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f2d8415b370 con 0x7f2d840cca70 2024-09-10T10:28:13.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.944+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d7800cb40 con 0x7f2d840cca70 2024-09-10T10:28:13.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.945+0000 7f2d6bfff640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2d7006be60 0x7f2d7006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:13.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.945+0000 7f2d837fe640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2d7006be60 0x7f2d7006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:13.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.946+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 6801+0+0 (secure 0 0 0) 0x7f2d7808b4e0 con 0x7f2d840cca70 2024-09-10T10:28:13.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.946+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v46) ==== 54+0+16 (secure 0 0 0) 0x7f2d78017020 con 0x7f2d840cca70 2024-09-10T10:28:13.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.947+0000 7f2d837fe640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2d7006be60 0x7f2d7006e320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009bd0 tx=0x7f2d7402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:13.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.947+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f2d5c001650 0x7f2d5c003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:13.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.947+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d5c006bb0 con 0x7f2d5c001650 2024-09-10T10:28:13.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.947+0000 7f2d83fff640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f2d5c001650 0x7f2d5c003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:13.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.947+0000 7f2d83fff640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f2d5c001650 0x7f2d5c003b10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:13.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.948+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d5c006bb0 con 0x7f2d5c001650 2024-09-10T10:28:13.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.976+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2d5c005c60 con 0x7f2d5c001650 2024-09-10T10:28:13.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.978+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2d5c005c60 con 0x7f2d5c001650 2024-09-10T10:28:13.980 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:13.980 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:13.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.979+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f2d5c004190 0x7f2d5c004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:13.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.979+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f2d5c004c00 con 0x7f2d5c004190 2024-09-10T10:28:13.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.979+0000 7f2d837fe640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f2d5c004190 0x7f2d5c004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:13.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.979+0000 7f2d837fe640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f2d5c004190 0x7f2d5c004640 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:13.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:13.980+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d5c004c00 con 0x7f2d5c004190 2024-09-10T10:28:13.986 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:13.986+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.009+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2d5c0051a0 con 0x7f2d5c004190 2024-09-10T10:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.011+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2d5c0051a0 con 0x7f2d5c004190 2024-09-10T10:28:14.012 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.012+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f2d5c009370 0x7f2d5c00b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.012+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f2d5c00e6c0 con 0x7f2d5c009370 2024-09-10T10:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.012+0000 7f2d82ffd640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f2d5c009370 0x7f2d5c00b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.012+0000 7f2d82ffd640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f2d5c009370 0x7f2d5c00b760 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.013+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d5c00e6c0 con 0x7f2d5c009370 2024-09-10T10:28:14.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.035+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2d5c00bfc0 con 0x7f2d5c009370 2024-09-10T10:28:14.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.037+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2d5c00bfc0 con 0x7f2d5c009370 2024-09-10T10:28:14.038 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:14.039 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:14.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.037+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f2d5c00ca40 0x7f2d5c00cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:14.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.037+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f2d5c00d6b0 con 0x7f2d5c00ca40 2024-09-10T10:28:14.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.037+0000 7f2d83fff640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f2d5c00ca40 0x7f2d5c00cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:14.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.038+0000 7f2d83fff640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f2d5c00ca40 0x7f2d5c00cea0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:14.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.038+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d5c00d6b0 con 0x7f2d5c00ca40 2024-09-10T10:28:14.042 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:14.042 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:14.051 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:14.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.053+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2d5c00dbc0 con 0x7f2d5c00ca40 2024-09-10T10:28:14.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.054+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2d5c00dbc0 con 0x7f2d5c00ca40 2024-09-10T10:28:14.056 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:14.056 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:14.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.055+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f2d5c010880 0x7f2d5c012c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:14.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.055+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f2d5c00e3a0 con 0x7f2d5c010880 2024-09-10T10:28:14.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.055+0000 7f2d837fe640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f2d5c010880 0x7f2d5c012c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:14.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.055+0000 7f2d837fe640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f2d5c010880 0x7f2d5c012c70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:14.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.056+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d5c00e3a0 con 0x7f2d5c010880 2024-09-10T10:28:14.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.070+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2d5c000950 con 0x7f2d5c010880 2024-09-10T10:28:14.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.072+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2d5c000950 con 0x7f2d5c010880 2024-09-10T10:28:14.073 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:14.074 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:14.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.072+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f2d5c0136c0 0x7f2d5c013b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:14.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.072+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f2d5c018cb0 con 0x7f2d5c0136c0 2024-09-10T10:28:14.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.073+0000 7f2d82ffd640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f2d5c0136c0 0x7f2d5c013b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:14.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.073+0000 7f2d82ffd640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f2d5c0136c0 0x7f2d5c013b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:14.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.074+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d5c018cb0 con 0x7f2d5c0136c0 2024-09-10T10:28:14.088 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:14.087+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:14.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.088+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2d5c014100 con 0x7f2d5c0136c0 2024-09-10T10:28:14.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.090+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2d5c014100 con 0x7f2d5c0136c0 2024-09-10T10:28:14.091 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:14.092 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:14.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.090+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f2d5c0147d0 0x7f2d5c014bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:14.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.090+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f2d5c015370 con 0x7f2d5c0147d0 2024-09-10T10:28:14.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.090+0000 7f2d83fff640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f2d5c0147d0 0x7f2d5c014bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:14.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.091+0000 7f2d83fff640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f2d5c0147d0 0x7f2d5c014bf0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:14.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.092+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d5c015370 con 0x7f2d5c0147d0 2024-09-10T10:28:14.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.120+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2d5c015630 con 0x7f2d5c0147d0 2024-09-10T10:28:14.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.122+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2d5c015630 con 0x7f2d5c0147d0 2024-09-10T10:28:14.123 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.123+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2d5c01a050 0x7f2d5c01c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.123+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f2d5c01f210 con 0x7f2d5c01a050 2024-09-10T10:28:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.123+0000 7f2d837fe640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2d5c01a050 0x7f2d5c01c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.123+0000 7f2d837fe640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2d5c01a050 0x7f2d5c01c440 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:14.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.124+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d5c01f210 con 0x7f2d5c01a050 2024-09-10T10:28:14.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.150+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2d5c01c980 con 0x7f2d5c01a050 2024-09-10T10:28:14.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.153+0000 7f2d6bfff640 1 -- 172.21.15.120:0/947384369 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2d5c01c980 con 0x7f2d5c01a050 2024-09-10T10:28:14.154 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:14.156 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.153+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f2d5c001650 msgr2=0x7f2d5c003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.153+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f2d5c001650 0x7f2d5c003b10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.153+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f2d5c004190 msgr2=0x7f2d5c004640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.153+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f2d5c004190 0x7f2d5c004640 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.153+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f2d5c009370 msgr2=0x7f2d5c00b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.153+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f2d5c009370 0x7f2d5c00b760 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.153+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f2d5c00ca40 msgr2=0x7f2d5c00cea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.153+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f2d5c00ca40 0x7f2d5c00cea0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f2d5c010880 msgr2=0x7f2d5c012c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f2d5c010880 0x7f2d5c012c70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f2d5c0136c0 msgr2=0x7f2d5c013b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f2d5c0136c0 0x7f2d5c013b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f2d5c0147d0 msgr2=0x7f2d5c014bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d83fff640 1 -- 172.21.15.120:0/947384369 reap_dead start 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d83fff640 1 -- 172.21.15.120:0/947384369 reap_dead start 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f2d5c0147d0 0x7f2d5c014bf0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2d5c01a050 msgr2=0x7f2d5c01c440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d83fff640 1 -- 172.21.15.120:0/947384369 reap_dead start 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2d5c01a050 0x7f2d5c01c440 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2d7006be60 msgr2=0x7f2d7006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2d7006be60 0x7f2d7006e320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009bd0 tx=0x7f2d7402f000 comp rx=0 tx=0).stop 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cca70 msgr2=0x7f2d840c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cca70 0x7f2d840c7c10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009fd0 tx=0x7f2d7800ee10 comp rx=0 tx=0).stop 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 shutdown_connections 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f2d7006be60 0x7f2d7006e320 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009bd0 tx=0x7f2d7402f000 comp rx=0 tx=0).stop 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2d5c01a050 0x7f2d5c01c440 crc :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 --2- 172.21.15.120:0/947384369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d840cca70 0x7f2d840c7c10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.154+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 >> 172.21.15.120:0/947384369 conn(0x7f2d8406ce80 msgr2=0x7f2d840cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.155+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 shutdown_connections 2024-09-10T10:28:14.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:14.155+0000 7f2d69ffb640 1 -- 172.21.15.120:0/947384369 wait complete. 2024-09-10T10:28:14.185 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:14.185 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:14.189 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:14.188+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:14.198 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:14.290 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:14.289+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:14.345 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:14.345 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:14.354 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:14.390 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:14.390+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:14.459 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:14.459 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:14.468 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:14.492 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:14.491+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:14.572 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:14.572 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:14.581 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:14.593 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:14.592+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:14.686 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:14.686 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:14.693 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:14.692+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:14.695 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:14.794 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:14.792+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:14.832 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:14.832 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:14.842 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:14.926 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:14.924+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:14.995 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:14.993+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:15.036 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:15.036 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:15.048 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:15.096 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:15.096+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:15.193 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:28:15.193 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:28:15.193 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-10T10:28:15.193 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-10T10:28:15.193 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-10T10:28:15.193 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-10T10:28:15.194 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:15.194 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:15.197 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:15.195+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:15.204 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:15.215 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-10T10:28:15.298 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:15.296+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:15.339 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:15.339 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-10T10:28:15.339 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:15.392 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:15.392 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:15.399 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:15.397+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:15.403 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:15.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.402+0000 7f35a3a8d640 1 Processor -- start 2024-09-10T10:28:15.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.403+0000 7f35a3a8d640 1 -- start start 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.404+0000 7f35a3a8d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f359c0c6270 0x7f359c06bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.404+0000 7f35a3a8d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f359c0c3790 0x7f359c0c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.404+0000 7f35a3a8d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c06c150 0x7f359c0c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.404+0000 7f35a3a8d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f359c0c75b0 con 0x7f359c0c3790 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.404+0000 7f35a3a8d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f359c15edf0 con 0x7f359c0c6270 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.404+0000 7f35a3a8d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f359c0c7800 con 0x7f359c06c150 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.404+0000 7f35a1028640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f359c0c3790 0x7f359c0c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.404+0000 7f35a1028640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f359c0c3790 0x7f359c0c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57080/0 (socket says 172.21.15.120:57080) 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.404+0000 7f35a1028640 1 -- 172.21.15.120:0/3024965968 learned_addr learned my addr 172.21.15.120:0/3024965968 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.404+0000 7f35a1829640 1 --2- 172.21.15.120:0/3024965968 >> v2:172.21.15.134:3300/0 conn(0x7f359c0c6270 0x7f359c06bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:15.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.405+0000 7f35a202a640 1 --2- 172.21.15.120:0/3024965968 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c06c150 0x7f359c0c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:15.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.405+0000 7f35a1028640 1 -- 172.21.15.120:0/3024965968 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c06c150 msgr2=0x7f359c0c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:15.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.405+0000 7f35a1028640 1 --2- 172.21.15.120:0/3024965968 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c06c150 0x7f359c0c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:15.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.405+0000 7f35a1028640 1 -- 172.21.15.120:0/3024965968 >> v2:172.21.15.134:3300/0 conn(0x7f359c0c6270 msgr2=0x7f359c06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:15.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.405+0000 7f35a1028640 1 --2- 172.21.15.120:0/3024965968 >> v2:172.21.15.134:3300/0 conn(0x7f359c0c6270 0x7f359c06bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:15.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.405+0000 7f35a1028640 1 -- 172.21.15.120:0/3024965968 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c06d2a0 con 0x7f359c0c3790 2024-09-10T10:28:15.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.405+0000 7f35a1829640 1 --2- 172.21.15.120:0/3024965968 >> v2:172.21.15.134:3300/0 conn(0x7f359c0c6270 0x7f359c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:15.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.405+0000 7f35a1028640 1 --2- 172.21.15.120:0/3024965968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c3790 0x7f359c0c7ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f359c071960 tx=0x7f358c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86986b1ee6e4fbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:15.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.406+0000 7f35a0827640 1 -- 172.21.15.120:0/3024965968 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c037070 con 0x7f359c0c3790 2024-09-10T10:28:15.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.406+0000 7f35a0827640 1 -- 172.21.15.120:0/3024965968 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358c032440 con 0x7f359c0c3790 2024-09-10T10:28:15.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.406+0000 7f35a0827640 1 -- 172.21.15.120:0/3024965968 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c032810 con 0x7f359c0c3790 2024-09-10T10:28:15.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.406+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3024965968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c3790 msgr2=0x7f359c0c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:15.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.406+0000 7f35a3a8d640 1 --2- 172.21.15.120:0/3024965968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c3790 0x7f359c0c7ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f359c071960 tx=0x7f358c02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:28:15.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.406+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3024965968 shutdown_connections 2024-09-10T10:28:15.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.406+0000 7f35a3a8d640 1 --2- 172.21.15.120:0/3024965968 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c06c150 0x7f359c0c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:15.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.406+0000 7f35a3a8d640 1 --2- 172.21.15.120:0/3024965968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c3790 0x7f359c0c7ea0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:15.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.406+0000 7f35a3a8d640 1 --2- 172.21.15.120:0/3024965968 >> v2:172.21.15.134:3300/0 conn(0x7f359c0c6270 0x7f359c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:15.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.406+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3024965968 >> 172.21.15.120:0/3024965968 conn(0x7f359c0ccbb0 msgr2=0x7f359c0ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:15.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.407+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3024965968 shutdown_connections 2024-09-10T10:28:15.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.407+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3024965968 wait complete. 2024-09-10T10:28:15.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.408+0000 7f35a3a8d640 1 Processor -- start 2024-09-10T10:28:15.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.408+0000 7f35a3a8d640 1 -- start start 2024-09-10T10:28:15.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.409+0000 7f35a3a8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c06c150 0x7f359c144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:15.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a3a8d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c0c3790 0x7f359c1f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a3a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c6270 0x7f359c0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a3a8d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f359c0c7ae0 con 0x7f359c0c6270 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a3a8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f359c0c6650 con 0x7f359c06c150 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a3a8d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f359c067e70 con 0x7f359c0c3790 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a202a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c6270 0x7f359c0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a202a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c6270 0x7f359c0c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57086/0 (socket says 172.21.15.120:57086) 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a202a640 1 -- 172.21.15.120:0/3367237244 learned_addr learned my addr 172.21.15.120:0/3367237244 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a1028640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c0c3790 0x7f359c1f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a1829640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c06c150 0x7f359c144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a202a640 1 -- 172.21.15.120:0/3367237244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c0c3790 msgr2=0x7f359c1f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a202a640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c0c3790 0x7f359c1f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:15.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a202a640 1 -- 172.21.15.120:0/3367237244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c06c150 msgr2=0x7f359c144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:15.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a202a640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c06c150 0x7f359c144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:15.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a202a640 1 -- 172.21.15.120:0/3367237244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c144ed0 con 0x7f359c0c6270 2024-09-10T10:28:15.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35a202a640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c6270 0x7f359c0c7fd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3594007d50 tx=0x7f359400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:15.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.410+0000 7f35927fc640 1 -- 172.21.15.120:0/3367237244 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594014420 con 0x7f359c0c6270 2024-09-10T10:28:15.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.411+0000 7f35927fc640 1 -- 172.21.15.120:0/3367237244 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3594019070 con 0x7f359c0c6270 2024-09-10T10:28:15.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.411+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3367237244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c0c8b90 con 0x7f359c0c6270 2024-09-10T10:28:15.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.411+0000 7f35927fc640 1 -- 172.21.15.120:0/3367237244 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594014800 con 0x7f359c0c6270 2024-09-10T10:28:15.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.411+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3367237244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c1548d0 con 0x7f359c0c6270 2024-09-10T10:28:15.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.413+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3367237244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359c15ea40 con 0x7f359c0c6270 2024-09-10T10:28:15.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.413+0000 7f35927fc640 1 -- 172.21.15.120:0/3367237244 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35940149a0 con 0x7f359c0c6270 2024-09-10T10:28:15.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.413+0000 7f35927fc640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f358006be60 0x7f358006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:15.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.413+0000 7f35927fc640 1 -- 172.21.15.120:0/3367237244 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6801+0+0 (secure 0 0 0) 0x7f359408ba00 con 0x7f359c0c6270 2024-09-10T10:28:15.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.414+0000 7f35a1829640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f358006be60 0x7f358006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.414+0000 7f35a1829640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f358006be60 0x7f358006e320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3598009560 tx=0x7f3598008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:15.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.418+0000 7f35927fc640 1 -- 172.21.15.120:0/3367237244 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f359c144ed0 con 0x7f359c0c6270 2024-09-10T10:28:15.499 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:15.498+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:15.517 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:15.517 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:15.526 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:15.612 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:15.611+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:15.672 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:15.672 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:15.681 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:15.700 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:15.698+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:15.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:15.781+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3367237244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f359c0729e0 con 0x7f359c0c6270 2024-09-10T10:28:15.801 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:15.799+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:15.824 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:15.825 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:15.833 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:15.902 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:15.900+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:15.936 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:15.936 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:15.945 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:16.003 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:16.002+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:16.048 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:16.048 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:16.056 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:16.103 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:16.101+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:16.171 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:28:16.198 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:16.198 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:16.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.201+0000 7f35927fc640 1 -- 172.21.15.120:0/3367237244 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v47) ==== 97+0+0 (secure 0 0 0) 0x7f3594059ed0 con 0x7f359c0c6270 2024-09-10T10:28:16.203 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-10T10:28:16.204 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:16.202+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:16.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3367237244 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f358006be60 msgr2=0x7f358006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f358006be60 0x7f358006e320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3598009560 tx=0x7f3598008040 comp rx=0 tx=0).stop 2024-09-10T10:28:16.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3367237244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c6270 msgr2=0x7f359c0c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c6270 0x7f359c0c7fd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3594007d50 tx=0x7f359400e4b0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3367237244 shutdown_connections 2024-09-10T10:28:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f358006be60 0x7f358006e320 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f359c0c6270 0x7f359c0c7fd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359c0c3790 0x7f359c1f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 --2- 172.21.15.120:0/3367237244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f359c06c150 0x7f359c144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3367237244 >> 172.21.15.120:0/3367237244 conn(0x7f359c0ccbb0 msgr2=0x7f359c0c9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.205+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3367237244 shutdown_connections 2024-09-10T10:28:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.206+0000 7f35a3a8d640 1 -- 172.21.15.120:0/3367237244 wait complete. 2024-09-10T10:28:16.210 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:16.304 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:16.303+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:16.313 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:16.313 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:16.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.319+0000 7f553db86640 1 Processor -- start 2024-09-10T10:28:16.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.320+0000 7f553db86640 1 -- start start 2024-09-10T10:28:16.323 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:16.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f553db86640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f55380cca70 0x7f553815b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f553db86640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f55380cafa0 0x7f55380cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f553db86640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f553815e0b0 con 0x7f5538154c10 2024-09-10T10:28:16.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f553db86640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f553815de60 con 0x7f55380cca70 2024-09-10T10:28:16.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f553db86640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f553815e390 con 0x7f55380cafa0 2024-09-10T10:28:16.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f5536ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f55380cca70 0x7f553815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f55377fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f5538154c10 0x7f5538073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:60800/0 (socket says 172.21.15.120:60800) 2024-09-10T10:28:16.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f55377fe640 1 -- 172.21.15.120:0/3039392769 learned_addr learned my addr 172.21.15.120:0/3039392769 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:16.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f5537fff640 1 --2- 172.21.15.120:0/3039392769 >> v2:172.21.15.134:3300/0 conn(0x7f55380cafa0 0x7f55380cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f55367fc640 1 -- 172.21.15.120:0/3039392769 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3470660260 0 0) 0x7f553815e0b0 con 0x7f5538154c10 2024-09-10T10:28:16.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.321+0000 7f55367fc640 1 -- 172.21.15.120:0/3039392769 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5520003520 con 0x7f5538154c10 2024-09-10T10:28:16.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.322+0000 7f5536ffd640 1 -- 172.21.15.120:0/3039392769 >> v2:172.21.15.134:3300/0 conn(0x7f55380cafa0 msgr2=0x7f55380cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:16.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.322+0000 7f5536ffd640 1 --2- 172.21.15.120:0/3039392769 >> v2:172.21.15.134:3300/0 conn(0x7f55380cafa0 0x7f55380cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.322+0000 7f5536ffd640 1 -- 172.21.15.120:0/3039392769 >> v1:172.21.15.134:6789/0 conn(0x7f5538154c10 legacy=0x7f5538073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.322+0000 7f5536ffd640 1 -- 172.21.15.120:0/3039392769 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553815e670 con 0x7f55380cca70 2024-09-10T10:28:16.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.323+0000 7f5536ffd640 1 --2- 172.21.15.120:0/3039392769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55380cca70 0x7f553815b370 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f553815dfe0 tx=0x7f552c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f1ee1a42c82777 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.323+0000 7f55367fc640 1 -- 172.21.15.120:0/3039392769 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c037070 con 0x7f55380cca70 2024-09-10T10:28:16.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.323+0000 7f55367fc640 1 -- 172.21.15.120:0/3039392769 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552c032440 con 0x7f55380cca70 2024-09-10T10:28:16.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.323+0000 7f55367fc640 1 -- 172.21.15.120:0/3039392769 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c032830 con 0x7f55380cca70 2024-09-10T10:28:16.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.324+0000 7f553db86640 1 -- 172.21.15.120:0/3039392769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55380cca70 msgr2=0x7f553815b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.324+0000 7f553db86640 1 --2- 172.21.15.120:0/3039392769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55380cca70 0x7f553815b370 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f553815dfe0 tx=0x7f552c02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.324+0000 7f553db86640 1 -- 172.21.15.120:0/3039392769 shutdown_connections 2024-09-10T10:28:16.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.324+0000 7f553db86640 1 --2- 172.21.15.120:0/3039392769 >> v2:172.21.15.134:3300/0 conn(0x7f55380cafa0 0x7f55380cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.324+0000 7f553db86640 1 --2- 172.21.15.120:0/3039392769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55380cca70 0x7f553815b370 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.324+0000 7f553db86640 1 -- 172.21.15.120:0/3039392769 >> 172.21.15.120:0/3039392769 conn(0x7f553806ce80 msgr2=0x7f553806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:16.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.324+0000 7f553db86640 1 -- 172.21.15.120:0/3039392769 shutdown_connections 2024-09-10T10:28:16.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.324+0000 7f553db86640 1 -- 172.21.15.120:0/3039392769 wait complete. 2024-09-10T10:28:16.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.326+0000 7f553db86640 1 Processor -- start 2024-09-10T10:28:16.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.326+0000 7f553db86640 1 -- start start 2024-09-10T10:28:16.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.327+0000 7f553db86640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55380cafa0 0x7f55381f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.327+0000 7f553db86640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55380cca70 0x7f55380ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.327+0000 7f553db86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5538154c10 0x7f553814eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f553db86640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5538154ff0 con 0x7f5538154c10 2024-09-10T10:28:16.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f553db86640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f553815ec30 con 0x7f55380cca70 2024-09-10T10:28:16.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f553db86640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5538073a50 con 0x7f55380cafa0 2024-09-10T10:28:16.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f5537fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5538154c10 0x7f553814eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f5537fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5538154c10 0x7f553814eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57102/0 (socket says 172.21.15.120:57102) 2024-09-10T10:28:16.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f55377fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55380cafa0 0x7f55381f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f5537fff640 1 -- 172.21.15.120:0/3344591010 learned_addr learned my addr 172.21.15.120:0/3344591010 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:16.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f5536ffd640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55380cca70 0x7f55380ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f5537fff640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55380cafa0 msgr2=0x7f55381f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f5537fff640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55380cafa0 0x7f55381f0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f5537fff640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55380cca70 msgr2=0x7f55380ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f5537fff640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55380cca70 0x7f55380ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f5537fff640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5538150dc0 con 0x7f5538154c10 2024-09-10T10:28:16.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.328+0000 7f5536ffd640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55380cca70 0x7f55380ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:16.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.329+0000 7f5537fff640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5538154c10 0x7f553814eeb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5530009dd0 tx=0x7f553000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.329+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530018020 con 0x7f5538154c10 2024-09-10T10:28:16.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.329+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5530014480 con 0x7f5538154c10 2024-09-10T10:28:16.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.329+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55381f1270 con 0x7f5538154c10 2024-09-10T10:28:16.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.329+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530014870 con 0x7f5538154c10 2024-09-10T10:28:16.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.329+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5538151340 con 0x7f5538154c10 2024-09-10T10:28:16.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.330+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5538151580 con 0x7f5538154c10 2024-09-10T10:28:16.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.331+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5530005d10 con 0x7f5538154c10 2024-09-10T10:28:16.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.332+0000 7f551ffff640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f552406be30 0x7f552406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.332+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6801+0+0 (secure 0 0 0) 0x7f553008bbc0 con 0x7f5538154c10 2024-09-10T10:28:16.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.332+0000 7f55377fe640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f552406be30 0x7f552406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.332+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v47) ==== 54+0+16 (secure 0 0 0) 0x7f5538151580 con 0x7f5538154c10 2024-09-10T10:28:16.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.333+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f553814f3f0 0x7f55380cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.333+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f553815b370 con 0x7f553814f3f0 2024-09-10T10:28:16.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.333+0000 7f5536ffd640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f553814f3f0 0x7f55380cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.333+0000 7f55377fe640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f552406be30 0x7f552406e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5528012940 tx=0x7f55280128d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.333+0000 7f5536ffd640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f553814f3f0 0x7f55380cefd0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.334+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f553815b370 con 0x7f553814f3f0 2024-09-10T10:28:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.362+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f55380cb950 con 0x7f553814f3f0 2024-09-10T10:28:16.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.365+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f55380cb950 con 0x7f553814f3f0 2024-09-10T10:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:16.367 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:16.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.365+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f55381f0830 0x7f553814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.365+0000 7f55377fe640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f55381f0830 0x7f553814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.366+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5538073590 con 0x7f55381f0830 2024-09-10T10:28:16.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.366+0000 7f55377fe640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f55381f0830 0x7f553814def0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.367+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5538073590 con 0x7f55381f0830 2024-09-10T10:28:16.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.392+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5538150380 con 0x7f55381f0830 2024-09-10T10:28:16.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.394+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5538150380 con 0x7f55381f0830 2024-09-10T10:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:16.397 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:16.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.395+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f55381f3890 0x7f55381f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.395+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f553814e940 con 0x7f55381f3890 2024-09-10T10:28:16.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.395+0000 7f5536ffd640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f55381f3890 0x7f55381f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.395+0000 7f5536ffd640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f55381f3890 0x7f55381f3c90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.396+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f553814e940 con 0x7f55381f3890 2024-09-10T10:28:16.405 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:16.403+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:16.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.422+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f55381f44a0 con 0x7f55381f3890 2024-09-10T10:28:16.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.424+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f55381f44a0 con 0x7f55381f3890 2024-09-10T10:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:16.427 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:16.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.424+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f5538062f00 0x7f553815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.424+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f55381f46b0 con 0x7f5538062f00 2024-09-10T10:28:16.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.424+0000 7f5537fff640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f5538062f00 0x7f553815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.425+0000 7f5537fff640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f5538062f00 0x7f553815ba50 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.425+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5530078e50 con 0x7f5538062f00 2024-09-10T10:28:16.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.452+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f553815bf90 con 0x7f5538062f00 2024-09-10T10:28:16.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.454+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f553815bf90 con 0x7f5538062f00 2024-09-10T10:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.455+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f553815c6b0 0x7f553802c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.455+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f55380635d0 con 0x7f553815c6b0 2024-09-10T10:28:16.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.455+0000 7f55377fe640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f553815c6b0 0x7f553802c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.455+0000 7f55377fe640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f553815c6b0 0x7f553802c640 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.456+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55380635d0 con 0x7f553815c6b0 2024-09-10T10:28:16.466 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:16.466 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:16.476 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.482+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f553815bf90 con 0x7f553815c6b0 2024-09-10T10:28:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.484+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f553815bf90 con 0x7f553815c6b0 2024-09-10T10:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.484+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f553802d090 0x7f553802d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.484+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f553802db20 con 0x7f553802d090 2024-09-10T10:28:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.484+0000 7f5536ffd640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f553802d090 0x7f553802d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.485+0000 7f5536ffd640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f553802d090 0x7f553802d4d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.486+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f553802db20 con 0x7f553802d090 2024-09-10T10:28:16.506 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:16.504+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:16.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.512+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f553802db20 con 0x7f553802d090 2024-09-10T10:28:16.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.514+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f553802db20 con 0x7f553802d090 2024-09-10T10:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:16.517 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:16.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.514+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f5538203110 0x7f5538205500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.514+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f5538208430 con 0x7f5538203110 2024-09-10T10:28:16.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.515+0000 7f5537fff640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f5538203110 0x7f5538205500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.515+0000 7f5537fff640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f5538203110 0x7f5538205500 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.516+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5538208430 con 0x7f5538203110 2024-09-10T10:28:16.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.542+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5538208400 con 0x7f5538203110 2024-09-10T10:28:16.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.544+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5538208400 con 0x7f5538203110 2024-09-10T10:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:16.547 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:16.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.544+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f5538205f50 0x7f55382063d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:16.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.544+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f553820b800 con 0x7f5538205f50 2024-09-10T10:28:16.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.545+0000 7f55377fe640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f5538205f50 0x7f55382063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:16.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.545+0000 7f55377fe640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f5538205f50 0x7f55382063d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:16.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.546+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f553820b800 con 0x7f5538205f50 2024-09-10T10:28:16.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.572+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5538206a20 con 0x7f5538205f50 2024-09-10T10:28:16.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.574+0000 7f551ffff640 1 -- 172.21.15.120:0/3344591010 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5538206a20 con 0x7f5538205f50 2024-09-10T10:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:16.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.574+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f553814f3f0 msgr2=0x7f55380cefd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.574+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f553814f3f0 0x7f55380cefd0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f55381f0830 msgr2=0x7f553814def0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f55381f0830 0x7f553814def0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f55381f3890 msgr2=0x7f55381f3c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f55381f3890 0x7f55381f3c90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f5538062f00 msgr2=0x7f553815ba50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f5538062f00 0x7f553815ba50 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f5537fff640 1 -- 172.21.15.120:0/3344591010 reap_dead start 2024-09-10T10:28:16.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f553815c6b0 msgr2=0x7f553802c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f553815c6b0 0x7f553802c640 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f5537fff640 1 -- 172.21.15.120:0/3344591010 reap_dead start 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f553802d090 msgr2=0x7f553802d4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f553802d090 0x7f553802d4d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f5538203110 msgr2=0x7f5538205500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f5538203110 0x7f5538205500 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f5538205f50 msgr2=0x7f55382063d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f5538205f50 0x7f55382063d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f552406be30 msgr2=0x7f552406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f552406be30 0x7f552406e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5528012940 tx=0x7f55280128d0 comp rx=0 tx=0).stop 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5538154c10 msgr2=0x7f553814eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 --2- 172.21.15.120:0/3344591010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5538154c10 0x7f553814eeb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5530009dd0 tx=0x7f553000c490 comp rx=0 tx=0).stop 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f5537fff640 1 -- 172.21.15.120:0/3344591010 reap_dead start 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 shutdown_connections 2024-09-10T10:28:16.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 >> 172.21.15.120:0/3344591010 conn(0x7f553806ce80 msgr2=0x7f5538159200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:16.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 shutdown_connections 2024-09-10T10:28:16.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:16.575+0000 7f553db86640 1 -- 172.21.15.120:0/3344591010 wait complete. 2024-09-10T10:28:16.582 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:16.582 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:16.582 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:16.582 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 89.800667805000003, 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:16.583 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.584 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.585 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.586 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:28:16.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 89.800337304999999, 2024-09-10T10:28:16.588 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.589 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:28:16.590 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:28:16.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.592 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.592 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.592 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:28:16.592 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:28:16.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.592 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.593 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.594 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 89.800290762000003, 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.595 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.596 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.597 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.598 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.708848549999999, 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:28:16.600 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:28:16.601 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:28:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:28:16.603 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.708767206999994, 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:28:16.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:28:16.606 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:28:16.607 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:28:16.608 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.708740652000003, 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:28:16.609 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.610 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:28:16.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:28:16.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:28:16.613 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.708598003000006, 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.616 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.617 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.618 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.619 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.705951069999998, 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.620 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:28:16.621 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.622 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.623 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:16.624 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.705100329999993, 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:16.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:28:16.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:28:16.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.632948646999999, 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:28:16.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.632899510999998, 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:28:16.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:28:16.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:16.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.632788032999997, 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:16.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:28:16.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:28:16.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 56.659387514000002, 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:28:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 56.659357343000003, 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 56.659300850999998, 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:28:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:28:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 56.659199682999997, 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 50.638932586000003, 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.670 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 50.634867366999998, 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:16.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.674 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 50.634849496999998, 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:28:16.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.680 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.680 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:28:16.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:28:16.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.680 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.680 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:28:16.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 50.634803259000002, 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:16.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:16.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:28:16.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:16.685 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:16.686 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:16.687 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:16.688 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:16.686+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:16.787 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:16.786+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:16.834 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:16.834 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:16.835 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:16.835 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:16.843 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:16.889 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:16.887+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:16.946 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:16.946 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:16.946 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:16.946 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:16.947 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:16.955 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:16.990 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:16.988+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:17.060 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:17.060 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.96201308099999, 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.17492211699999999, 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.061 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 1 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096318+0000", 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.657e-06 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096326+0000", 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0560000000000005e-06 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096339+0000", 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2965999999999999e-05 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096349+0000", 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0784e-05 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096432+0000", 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2657999999999994e-05 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op must wait for map", 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096441+0000", 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4590000000000004e-06 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096935+0000", 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049331899999999998 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for peered", 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096944+0000", 2024-09-10T10:28:17.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9539999999999993e-06 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267505+0000", 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.170561622 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267700+0000", 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019420400000000001 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267923+0000", 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000223451 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.270575+0000", 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026522030000000001 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271131+0000", 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055599000000000002 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271191+0000", 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9947000000000001e-05 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271238+0000", 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6847000000000001e-05 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.573178837, 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078035889999999997, 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 115 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.069 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:17.070 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1300000000000001e-07 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485209+0000", 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8628999999999997e-05 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485214+0000", 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0849999999999996e-06 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485218+0000", 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3840000000000001e-06 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490185+0000", 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0049675270000000002 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490231+0000", 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5488000000000002e-05 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490307+0000", 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6038000000000006e-05 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491263+0000", 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095625099999999998 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492908+0000", 2024-09-10T10:28:17.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001645355 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492934+0000", 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5431000000000001e-05 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492954+0000", 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0088000000000001e-05 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.573104348, 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077741099999999999, 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 116 2024-09-10T10:28:17.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1100000000000002e-07 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485227+0000", 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1890000000000001e-06 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485231+0000", 2024-09-10T10:28:17.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4010000000000001e-06 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485233+0000", 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.728e-06 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490628+0000", 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0053952109999999996 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490674+0000", 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5946999999999999e-05 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490750+0000", 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5227000000000003e-05 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491443+0000", 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069346800000000004 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.079 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492959+0000", 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015154470000000001 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492983+0000", 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.406e-05 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492999+0000", 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6121e-05 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.080 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.57309222000001, 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078059490000000004, 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.081 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 117 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.23e-07 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485240+0000", 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780000000000001e-06 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.083 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485243+0000", 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8430000000000003e-06 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485247+0000", 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2279999999999998e-06 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490995+0000", 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057480730000000002 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491043+0000", 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7936000000000003e-05 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491113+0000", 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0872999999999997e-05 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.085 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491636+0000", 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00052242300000000005 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493002+0000", 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001365808 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493025+0000", 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2932999999999999e-05 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493043+0000", 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8031000000000001e-05 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.240737078, 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095639920000000003, 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.087 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 131 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.088 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1199999999999998e-07 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.089 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817663+0000", 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1099000000000005e-05 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817670+0000", 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.8539999999999999e-06 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817674+0000", 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.574e-06 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.091 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818465+0000", 2024-09-10T10:28:17.091 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000791445 2024-09-10T10:28:17.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.092 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.092 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818509+0000", 2024-09-10T10:28:17.092 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3569000000000002e-05 2024-09-10T10:28:17.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.092 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.092 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.092 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818580+0000", 2024-09-10T10:28:17.092 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1503999999999998e-05 2024-09-10T10:28:17.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821292+0000", 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027120529999999999 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827108+0000", 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0058160759999999999 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827134+0000", 2024-09-10T10:28:17.093 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5859000000000001e-05 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827156+0000", 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1546999999999999e-05 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.24064787, 2024-09-10T10:28:17.094 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095198769999999995, 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 132 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.095 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2899999999999999e-07 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817683+0000", 2024-09-10T10:28:17.096 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0379999999999998e-06 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817690+0000", 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3110000000000002e-06 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817699+0000", 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.097 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820828+0000", 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031296829999999999 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820873+0000", 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4869e-05 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820943+0000", 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9919999999999995e-05 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.098 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823683+0000", 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027395140000000002 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827161+0000", 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003477857 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827183+0000", 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2612999999999998e-05 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827201+0000", 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7842999999999998e-05 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.24058056299999, 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0094944009999999995, 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.100 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 133 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.101 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817749+0000", 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2399999999999998e-07 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817752+0000", 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1970000000000001e-06 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817760+0000", 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8080000000000005e-06 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817764+0000", 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9419999999999997e-06 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821319+0000", 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035547740000000001 2024-09-10T10:28:17.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821364+0000", 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5398000000000001e-05 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821434+0000", 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0382999999999993e-05 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823717+0000", 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022825089999999998 2024-09-10T10:28:17.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827205+0000", 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034879009999999998 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827225+0000", 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0611999999999998e-05 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827243+0000", 2024-09-10T10:28:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7353000000000001e-05 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.240527722, 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095863660000000007, 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.106 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 134 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817802+0000", 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8999999999999996e-07 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817871+0000", 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9572000000000004e-05 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817878+0000", 2024-09-10T10:28:17.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2079999999999997e-06 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817882+0000", 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5889999999999996e-06 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821625+0000", 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037431069999999999 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821669+0000", 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3455000000000003e-05 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821740+0000", 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1390999999999995e-05 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823826+0000", 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020863399999999999 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.110 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827343+0000", 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003516267 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827369+0000", 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.6044999999999999e-05 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827388+0000", 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8802000000000001e-05 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.111 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.240435027, 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095738179999999996, 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 135 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.112 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:17.113 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5999999999999999e-07 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817897+0000", 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0149999999999999e-06 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817904+0000", 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5089999999999999e-06 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.114 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817909+0000", 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3120000000000001e-06 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823862+0000", 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059524199999999999 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823911+0000", 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9323000000000002e-05 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823983+0000", 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1736000000000006e-05 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827101+0000", 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031181009999999999 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827392+0000", 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00029090700000000003 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827426+0000", 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4127e-05 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827468+0000", 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1907999999999999e-05 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.117 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.240408011, 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.009594488, 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 136 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3700000000000001e-07 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.119 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817925+0000", 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4659999999999999e-06 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817931+0000", 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5710000000000003e-06 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.120 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817936+0000", 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7550000000000002e-06 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824178+0000", 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062415700000000001 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824225+0000", 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6966999999999999e-05 2024-09-10T10:28:17.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824296+0000", 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1284000000000004e-05 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827290+0000", 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002993697 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827473+0000", 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018376699999999999 2024-09-10T10:28:17.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827497+0000", 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3445e-05 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827515+0000", 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8529000000000002e-05 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.240376322, 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096066220000000004, 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 137 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.124 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3000000000000001e-07 2024-09-10T10:28:17.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818004+0000", 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0902000000000002e-05 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818013+0000", 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5420000000000005e-06 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818019+0000", 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7939999999999997e-06 2024-09-10T10:28:17.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824489+0000", 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0064692580000000003 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824532+0000", 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3631000000000003e-05 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824601+0000", 2024-09-10T10:28:17.127 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9028000000000004e-05 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827334+0000", 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027325409999999998 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827520+0000", 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018642499999999999 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827541+0000", 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1151000000000001e-05 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827559+0000", 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7819999999999999e-05 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:17.129 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.240297001, 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010100303, 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 138 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818033+0000", 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818037+0000", 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0419999999999996e-06 2024-09-10T10:28:17.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818045+0000", 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9759999999999995e-06 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818052+0000", 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5909999999999998e-06 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826594+0000", 2024-09-10T10:28:17.132 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0085421660000000003 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826671+0000", 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7121999999999998e-05 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826795+0000", 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012315199999999999 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828081+0000", 2024-09-10T10:28:17.133 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0012869139999999999 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828088+0000", 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9870000000000002e-06 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828112+0000", 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3405e-05 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828132+0000", 2024-09-10T10:28:17.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0394000000000002e-05 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.24026662999999, 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01125819, 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.135 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 139 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.136 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818063+0000", 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7800000000000002e-07 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818067+0000", 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8500000000000004e-06 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818075+0000", 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818081+0000", 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1829999999999999e-06 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827594+0000", 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095129340000000007 2024-09-10T10:28:17.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827669+0000", 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5239000000000002e-05 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827789+0000", 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000119757 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829195+0000", 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014064279999999999 2024-09-10T10:28:17.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829232+0000", 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7058999999999999e-05 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829281+0000", 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8884000000000002e-05 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829321+0000", 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9527999999999998e-05 2024-09-10T10:28:17.140 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.240232561, 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012263886, 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.141 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 140 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.142 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818097+0000", 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8999999999999997e-07 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818145+0000", 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8040000000000001e-05 2024-09-10T10:28:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818160+0000", 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4671999999999999e-05 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818169+0000", 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1900000000000001e-06 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828158+0000", 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099891440000000001 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828228+0000", 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0407000000000005e-05 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828343+0000", 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000114821 2024-09-10T10:28:17.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829914+0000", 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001570866 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830312+0000", 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00039770499999999999 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830339+0000", 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.69e-05 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830360+0000", 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1651e-05 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.147 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.24014532, 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012742035000000001, 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 141 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:17.149 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7700000000000004e-07 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818188+0000", 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8659999999999999e-06 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818196+0000", 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9480000000000008e-06 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818205+0000", 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8570000000000001e-06 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828883+0000", 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010678600999999999 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828927+0000", 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3887999999999997e-05 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829001+0000", 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3442000000000006e-05 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830548+0000", 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001547194 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830884+0000", 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00033638100000000002 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830907+0000", 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3051999999999999e-05 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830926+0000", 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8229000000000001e-05 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.226997583, 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097270880000000001, 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 145 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.154 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.155 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831332+0000", 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8900000000000001e-07 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831379+0000", 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7134e-05 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831387+0000", 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7409999999999992e-06 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831391+0000", 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6190000000000002e-06 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833155+0000", 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017641969999999999 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833237+0000", 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1957000000000002e-05 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833356+0000", 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011843100000000001 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.834068+0000", 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00071179000000000001 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841004+0000", 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069359749999999996 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841034+0000", 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.042e-05 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841058+0000", 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4535000000000001e-05 2024-09-10T10:28:17.159 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:20 2.9 e26/18 2:92e99a1d:::612.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 68.182801319000006, 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074964979999999999, 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:17.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875534+0000", 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9109999999999998e-06 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875543+0000", 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8319999999999995e-06 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875549+0000", 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2380000000000002e-06 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875699+0000", 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014945900000000001 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875717+0000", 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8584e-05 2024-09-10T10:28:17.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882982+0000", 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0072648039999999997 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883024+0000", 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2135000000000002e-05 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:27 2.9 e26/18 2:91bd965c:::501.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 68.182740684999999, 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074993129999999996, 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875589+0000", 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3999999999999997e-07 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.166 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875595+0000", 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9680000000000003e-06 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875602+0000", 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875608+0000", 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0430000000000004e-06 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875887+0000", 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000279578 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875904+0000", 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6439e-05 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883027+0000", 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0071234480000000001 2024-09-10T10:28:17.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883088+0000", 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0424000000000003e-05 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 51.116661608000001, 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01193195, 2024-09-10T10:28:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2399999999999999e-07 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941671+0000", 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8329999999999998e-06 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2919999999999998e-06 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941686+0000", 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944874+0000", 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031881639999999998 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945305+0000", 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00043084599999999997 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945438+0000", 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013344600000000001 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950653+0000", 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005214733 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953496+0000", 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028433989999999999 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953554+0000", 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7933000000000003e-05 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953599+0000", 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5130999999999999e-05 2024-09-10T10:28:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 51.116635662, 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0093592050000000006, 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941694+0000", 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7899999999999999e-07 2024-09-10T10:28:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941711+0000", 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7530999999999999e-05 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941720+0000", 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4279999999999996e-06 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941725+0000", 2024-09-10T10:28:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2390000000000002e-06 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942690+0000", 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00096541400000000005 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942807+0000", 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000116777 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942967+0000", 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015997599999999999 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944708+0000", 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001740492 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950956+0000", 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062481580000000002 2024-09-10T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951016+0000", 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0299e-05 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951052+0000", 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6412000000000003e-05 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:17.181 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:17.182 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:17.181+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:17.182 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:17.282 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:17.280+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:17.299 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:17.384 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:17.383+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:17.459 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:17.484 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:17.483+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:1 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-09-10T10:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.297087037, 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028809700000000001, 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9999999999999999e-07 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268119+0000", 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1810000000000003e-06 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268129+0000", 2024-09-10T10:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9790000000000005e-06 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268137+0000", 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4800000000000001e-06 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268198+0000", 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0918000000000002e-05 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268226+0000", 2024-09-10T10:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7267000000000002e-05 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270943+0000", 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027177529999999998 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270996+0000", 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2191999999999998e-05 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:119 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'81, mlcod=16'78)", 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.07123608400001, 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029329880000000001, 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493966+0000", 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1699999999999998e-07 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494040+0000", 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3883999999999999e-05 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494048+0000", 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4880000000000007e-06 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494057+0000", 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1310000000000006e-06 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494095+0000", 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8470000000000003e-05 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494108+0000", 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2588999999999999e-05 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496869+0000", 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027610400000000002 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496898+0000", 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9869000000000002e-05 2024-09-10T10:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.070852448, 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005336365, 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494350+0000", 2024-09-10T10:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494416+0000", 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6413000000000003e-05 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494424+0000", 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.576e-06 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494430+0000", 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6579999999999996e-06 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494466+0000", 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5648999999999997e-05 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494478+0000", 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.234e-05 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499654+0000", 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051755209999999998 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499685+0000", 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.167e-05 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-10T10:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 105.07045763799999, 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049664970000000003, 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-10T10:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494819+0000", 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4690000000000005e-05 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494825+0000", 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7080000000000002e-06 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494828+0000", 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1109999999999999e-06 2024-09-10T10:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494869+0000", 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1146000000000002e-05 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494881+0000", 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2295e-05 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499687+0000", 2024-09-10T10:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048061459999999999 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499710+0000", 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-05 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.785101435000001, 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054008370000000003, 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780104+0000", 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3039999999999998e-06 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0738e-05 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780141+0000", 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6317999999999999e-05 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2199999999999992e-06 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780405+0000", 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025557599999999998 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780498+0000", 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2860000000000002e-05 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780591+0000", 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3094e-05 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024814469999999999 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785441+0000", 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023678280000000002 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785480+0000", 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9511999999999997e-05 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785501+0000", 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0939999999999999e-05 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.785026600999998, 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054780480000000001, 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780176+0000", 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6700000000000003e-07 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1289999999999998e-06 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780188+0000", 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.96e-06 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780194+0000", 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9989999999999997e-06 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780882+0000", 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068832400000000003 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780968+0000", 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5959999999999997e-05 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781098+0000", 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012994100000000001 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783078+0000", 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019803540000000001 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785553+0000", 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002474515 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785608+0000", 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5319999999999999e-05 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785653+0000", 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4879000000000002e-05 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.784995058000007, 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042025200000000004, 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780217+0000", 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780224+0000", 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2529999999999998e-06 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780230+0000", 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6110000000000001e-06 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780282+0000", 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2148000000000003e-05 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780394+0000", 2024-09-10T10:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111864 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780553+0000", 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015913799999999999 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783054+0000", 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025008809999999999 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784349+0000", 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012947480000000001 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784381+0000", 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2268999999999999e-05 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784409+0000", 2024-09-10T10:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8007000000000001e-05 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.784952705999999, 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054782659999999999, 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780253+0000", 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4199999999999999e-06 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780260+0000", 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5100000000000001e-06 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780267+0000", 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.443e-06 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781353+0000", 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010859769999999999 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781457+0000", 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104545 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781561+0000", 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010409200000000001 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783532+0000", 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019702280000000001 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785685+0000", 2024-09-10T10:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002153313 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785710+0000", 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5024999999999998e-05 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785727+0000", 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7175000000000001e-05 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.784926615000003, 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005104407, 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780278+0000", 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.959e-06 2024-09-10T10:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780285+0000", 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8040000000000004e-06 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780292+0000", 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4099999999999996e-06 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781345+0000", 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010533579999999999 2024-09-10T10:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781456+0000", 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110624 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781570+0000", 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011417999999999999 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783625+0000", 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020553780000000001 2024-09-10T10:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785303+0000", 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001677827 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785345+0000", 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2364999999999998e-05 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785379+0000", 2024-09-10T10:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4054000000000003e-05 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.784902786999993, 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0060500800000000002, 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780306+0000", 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780313+0000", 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-06 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1739999999999999e-06 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782543+0000", 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022253419999999999 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010785699999999999 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.786349+0000", 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036978829999999999 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.693200747000006, 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058824769999999997, 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872002+0000", 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5149999999999999e-06 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872008+0000", 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.66e-06 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872022+0000", 2024-09-10T10:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3932e-05 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872031+0000", 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5259999999999999e-06 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872111+0000", 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9412000000000005e-05 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872231+0000", 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119843 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872340+0000", 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109442 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.875579+0000", 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032383899999999998 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877781+0000", 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022022299999999999 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877835+0000", 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4419000000000002e-05 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877883+0000", 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8108000000000001e-05 2024-09-10T10:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.692111916000002, 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049968149999999999, 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7300000000000005e-07 2024-09-10T10:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.245e-06 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3898e-05 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873116+0000", 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8050000000000007e-06 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873184+0000", 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7809000000000001e-05 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873257+0000", 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2602999999999996e-05 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873374+0000", 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011744900000000001 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.876282+0000", 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029079459999999998 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878003+0000", 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001720886 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878054+0000", 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0596999999999997e-05 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878086+0000", 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2804e-05 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.617719544000003, 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029964739999999998, 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947484+0000", 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.857e-06 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947490+0000", 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9789999999999998e-06 2024-09-10T10:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947504+0000", 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3942e-05 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947512+0000", 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5620000000000004e-06 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947599+0000", 2024-09-10T10:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6525000000000003e-05 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947714+0000", 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114709 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947823+0000", 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109477 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950336+0000", 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025131509999999999 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950386+0000", 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9974e-05 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950440+0000", 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4011000000000002e-05 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950478+0000", 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8287000000000001e-05 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.616279145999997, 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043004360000000004, 2024-09-10T10:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948923+0000", 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8999999999999996e-07 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948928+0000", 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948940+0000", 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1759e-05 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948949+0000", 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5149999999999995e-06 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949028+0000", 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8182999999999993e-05 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949137+0000", 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109065 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949245+0000", 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108047 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950193+0000", 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094856200000000002 2024-09-10T10:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953173+0000", 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029801960000000001 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953200+0000", 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6707999999999999e-05 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953223+0000", 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2758000000000001e-05 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 63.616223634000001, 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046333320000000004, 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948979+0000", 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-07 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6490000000000001e-06 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948990+0000", 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2560000000000002e-06 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948996+0000", 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9739999999999999e-06 2024-09-10T10:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949496+0000", 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049934299999999997 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949588+0000", 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1762999999999995e-05 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949729+0000", 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014145299999999999 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950984+0000", 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012548419999999999 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953505+0000", 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025208129999999998 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953568+0000", 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3695000000000006e-05 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953611+0000", 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2913000000000001e-05 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.644172019000003, 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048620219999999997, 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4899999999999995e-07 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921034+0000", 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7050000000000001e-06 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921043+0000", 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1840000000000005e-06 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921049+0000", 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4459999999999998e-06 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921575+0000", 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052602300000000003 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921646+0000", 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0475000000000005e-05 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921769+0000", 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123347 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923077+0000", 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013078759999999999 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925794+0000", 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027172310000000001 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925853+0000", 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8739999999999997e-05 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925892+0000", 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8445999999999997e-05 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.644134985000001, 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032737339999999999, 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9919999999999999e-06 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921080+0000", 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0063e-05 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921086+0000", 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921142+0000", 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6174999999999997e-05 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013480499999999999 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921405+0000", 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012819200000000001 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923001+0000", 2024-09-10T10:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015962249999999999 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924243+0000", 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012425820000000001 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924300+0000", 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6901999999999998e-05 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924340+0000", 2024-09-10T10:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9918e-05 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(2.4 epoch 26/18 rep_tid 19 entries 26'1 (0'0) error 2:218afd09:::203.00000003:head by mds.3.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.641996540000001, 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030589089999999998, 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923206+0000", 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-09-10T10:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7249999999999997e-06 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923229+0000", 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5905000000000001e-05 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923241+0000", 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1990999999999999e-05 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923317+0000", 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5622000000000002e-05 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926264+0000", 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029470170000000001 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.621638539999999, 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050642550000000001, 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943564+0000", 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.13e-07 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943576+0000", 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2187e-05 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943585+0000", 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9279999999999999e-06 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943592+0000", 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0110000000000003e-06 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943681+0000", 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9487999999999994e-05 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943767+0000", 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5866999999999996e-05 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943882+0000", 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011443299999999999 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.947042+0000", 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031600629999999998 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948540+0000", 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001498451 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948581+0000", 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0839e-05 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948627+0000", 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6375000000000002e-05 2024-09-10T10:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.621570216999999, 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051386089999999997, 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943632+0000", 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1500000000000004e-07 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943643+0000", 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0954e-05 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943652+0000", 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4929999999999998e-06 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943657+0000", 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6509999999999998e-06 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944230+0000", 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057322999999999999 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944340+0000", 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010986 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948770+0000", 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044297060000000003 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:17.671 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:17.672 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:17.670+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:17.772 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:17.771+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:17.780 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:17.780 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:17.780 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:17.780 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:17.789 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:17.873 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:17.871+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:17.899 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:17.899 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:17.899 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:17.899 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:17.899 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:17.908 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:17.992 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:17.990+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:18.074 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:18.073+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:18.117 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.334539922000005, 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049304329999999997, 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780094+0000", 2024-09-10T10:28:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780107+0000", 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3703000000000001e-05 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780116+0000", 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5210000000000002e-06 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6340000000000007e-06 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001735838 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782000+0000", 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013984100000000001 2024-09-10T10:28:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782131+0000", 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000131025 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783737+0000", 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016054179999999999 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784937+0000", 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001200638 2024-09-10T10:28:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784987+0000", 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9591e-05 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785024+0000", 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6751000000000001e-05 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.334500466999998, 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052929559999999997, 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780136+0000", 2024-09-10T10:28:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9950000000000001e-06 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780144+0000", 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.174e-06 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4949999999999999e-06 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781163+0000", 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001013438 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781279+0000", 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011579200000000001 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781388+0000", 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108834 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783243+0000", 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00185491 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785355+0000", 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021117990000000001 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785386+0000", 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1612000000000002e-05 2024-09-10T10:28:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785426+0000", 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9474999999999997e-05 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:8 2.10 e24/18 2:09401a81:::601.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.332309237000004, 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005352224, 2024-09-10T10:28:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4400000000000001e-07 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782467+0000", 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142452 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782598+0000", 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013078300000000001 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782604+0000", 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.533e-06 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7073999999999997e-05 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782668+0000", 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6861999999999999e-05 2024-09-10T10:28:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787632+0000", 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049636510000000003 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787676+0000", 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4425000000000002e-05 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:9 2.10 e24/18 2:09401a81:::601.00000000:head v 24'4, mlcod=0'0)", 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.331672221999995, 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047438960000000001, 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782962+0000", 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4499999999999997e-07 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782967+0000", 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2000000000000002e-06 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782980+0000", 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3685e-05 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782987+0000", 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4690000000000002e-06 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783112+0000", 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012481300000000001 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783130+0000", 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7753000000000001e-05 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787678+0000", 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045487419999999997 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787705+0000", 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6588999999999999e-05 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:10 2.10 e24/18 2:0bd6d154:::602.00000000:head v 24'5, mlcod=0'0)", 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.331565287999993, 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00466629, 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1300000000000001e-07 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6970000000000001e-06 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783309+0000", 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000236209 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783319+0000", 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7180000000000008e-06 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783365+0000", 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6179e-05 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783383+0000", 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8382000000000001e-05 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787708+0000", 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043246059999999999 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787734+0000", 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6086000000000001e-05 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.242163094000006, 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054527359999999997, 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872472+0000", 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5409999999999999e-06 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872479+0000", 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9550000000000003e-06 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872494+0000", 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5452999999999999e-05 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872503+0000", 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1509999999999993e-06 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872571+0000", 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7747e-05 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872704+0000", 2024-09-10T10:28:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013269900000000001 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872804+0000", 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9934999999999998e-05 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877617+0000", 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048138069999999998 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877819+0000", 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020164799999999999 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877880+0000", 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1296000000000004e-05 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877923+0000", 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2503999999999998e-05 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.241141526999996, 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0098302110000000002, 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873493+0000", 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873498+0000", 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1619999999999999e-06 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873511+0000", 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3319000000000001e-05 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873520+0000", 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3409999999999992e-06 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873586+0000", 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6115e-05 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873660+0000", 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3103999999999997e-05 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873775+0000", 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115617 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878285+0000", 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045096049999999999 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883229+0000", 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049443999999999998 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883283+0000", 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4262999999999999e-05 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883322+0000", 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8417000000000002e-05 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.240652386999997, 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047062800000000002, 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873982+0000", 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2900000000000003e-07 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873986+0000", 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3939999999999998e-06 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873995+0000", 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0750000000000004e-06 2024-09-10T10:28:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874002+0000", 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874061+0000", 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8770999999999997e-05 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874178+0000", 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117108 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874290+0000", 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112503 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878269+0000", 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039784870000000002 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878593+0000", 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032385299999999998 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878644+0000", 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1790999999999999e-05 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878687+0000", 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2738999999999999e-05 2024-09-10T10:28:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.240052108, 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089185449999999999, 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874582+0000", 2024-09-10T10:28:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1699999999999997e-07 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874592+0000", 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0604999999999999e-05 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874602+0000", 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874610+0000", 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1919999999999999e-06 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874696+0000", 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6033999999999997e-05 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874809+0000", 2024-09-10T10:28:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113373 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874923+0000", 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114077 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878289+0000", 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033662380000000001 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883420+0000", 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051305760000000004 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883465+0000", 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5252e-05 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883500+0000", 2024-09-10T10:28:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4415000000000002e-05 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.166188786999996, 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073932399999999997, 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948446+0000", 2024-09-10T10:28:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3400000000000001e-06 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948451+0000", 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6219999999999997e-06 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948464+0000", 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2955e-05 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948472+0000", 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0730000000000005e-06 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948534+0000", 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1727e-05 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948649+0000", 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114991 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948757+0000", 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107935 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950179+0000", 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014219440000000001 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055817970000000003 2024-09-10T10:28:18.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955794+0000", 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3012000000000002e-05 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955838+0000", 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3844000000000002e-05 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.165649967999997, 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069680200000000001, 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948984+0000", 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6100000000000001e-07 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948993+0000", 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9809999999999992e-06 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949002+0000", 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8789999999999992e-06 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949008+0000", 2024-09-10T10:28:18.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2759999999999997e-06 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949057+0000", 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9159999999999997e-05 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949128+0000", 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1292999999999997e-05 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:18.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949261+0000", 2024-09-10T10:28:18.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132564 2024-09-10T10:28:18.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950220+0000", 2024-09-10T10:28:18.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00095953300000000004 2024-09-10T10:28:18.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955890+0000", 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056692790000000002 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955928+0000", 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8278000000000002e-05 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955951+0000", 2024-09-10T10:28:18.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3215999999999999e-05 2024-09-10T10:28:18.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.177 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.177 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.177 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:18.177 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.193673457999999, 2024-09-10T10:28:18.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050489669999999997, 2024-09-10T10:28:18.177 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:18.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5799999999999999e-07 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920964+0000", 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6720000000000002e-06 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920971+0000", 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5140000000000004e-06 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920977+0000", 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0100000000000001e-06 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921033+0000", 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5899000000000001e-05 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921172+0000", 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139011 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921288+0000", 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011605200000000001 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922851+0000", 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001562499 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925915+0000", 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003063727 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925969+0000", 2024-09-10T10:28:18.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4083999999999999e-05 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926009+0000", 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9941000000000003e-05 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.193066600999998, 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045836510000000002, 2024-09-10T10:28:18.183 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:18.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:18.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4499999999999996e-07 2024-09-10T10:28:18.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921572+0000", 2024-09-10T10:28:18.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3239999999999996e-06 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921581+0000", 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9190000000000005e-06 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921589+0000", 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0649999999999999e-06 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921635+0000", 2024-09-10T10:28:18.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6863999999999997e-05 2024-09-10T10:28:18.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921705+0000", 2024-09-10T10:28:18.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9046000000000003e-05 2024-09-10T10:28:18.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921818+0000", 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011313499999999999 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923033+0000", 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012153590000000001 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926055+0000", 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030223310000000001 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926104+0000", 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8421999999999998e-05 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926150+0000", 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6440999999999998e-05 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.192627657000003, 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051621749999999997, 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3600000000000003e-07 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922011+0000", 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4449999999999998e-06 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922019+0000", 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0930000000000003e-06 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922025+0000", 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5060000000000001e-06 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922076+0000", 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0671000000000003e-05 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922175+0000", 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9611000000000006e-05 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922278+0000", 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102565 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925217+0000", 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029384670000000002 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927074+0000", 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018579499999999999 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927129+0000", 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4428000000000001e-05 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927168+0000", 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8803000000000001e-05 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:18.195 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.192591987, 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059957930000000001, 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:18.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922042+0000", 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922045+0000", 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4209999999999999e-06 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922052+0000", 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1430000000000001e-06 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922059+0000", 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2139999999999999e-06 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922495+0000", 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043517499999999999 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922563+0000", 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8084999999999996e-05 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922679+0000", 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011609599999999999 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927208+0000", 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045292170000000003 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927952+0000", 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00074421700000000003 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928001+0000", 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9004e-05 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928037+0000", 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.578e-05 2024-09-10T10:28:18.200 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.192565227999999, 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045268130000000002, 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922078+0000", 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922085+0000", 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3710000000000002e-06 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922091+0000", 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0340000000000002e-06 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922144+0000", 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3106999999999998e-05 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922248+0000", 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010373400000000001 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:18.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922399+0000", 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015127200000000001 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925201+0000", 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028019189999999999 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926514+0000", 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001312279 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926561+0000", 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000002e-05 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926595+0000", 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3408000000000003e-05 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.206 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.191930380000002, 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005322116, 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999996e-07 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922707+0000", 2024-09-10T10:28:18.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.766e-06 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922717+0000", 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7229999999999999e-06 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922724+0000", 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8410000000000002e-06 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922792+0000", 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8557000000000001e-05 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922912+0000", 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119354 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928025+0000", 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051131919999999999 2024-09-10T10:28:18.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.191454227000001, 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049271260000000004, 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923180+0000", 2024-09-10T10:28:18.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923183+0000", 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4000000000000001e-06 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923196+0000", 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3127e-05 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923203+0000", 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.939e-06 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923270+0000", 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7087000000000003e-05 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923359+0000", 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9054999999999994e-05 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928106+0000", 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047469239999999996 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.190993585999998, 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045860270000000003, 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:18.215 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:18.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2500000000000005e-07 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923644+0000", 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8240000000000001e-06 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923659+0000", 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5358000000000001e-05 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923666+0000", 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9659999999999999e-06 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923725+0000", 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8674000000000001e-05 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923850+0000", 2024-09-10T10:28:18.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000124797 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928226+0000", 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043757830000000003 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 52.173471151000001, 2024-09-10T10:28:18.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005230859, 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:18.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941164+0000", 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.863e-06 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941172+0000", 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2360000000000004e-06 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941190+0000", 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7812000000000002e-05 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941200+0000", 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7599999999999997e-06 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941350+0000", 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015070400000000001 2024-09-10T10:28:18.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941501+0000", 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015044200000000001 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000167076 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943487+0000", 2024-09-10T10:28:18.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018190260000000001 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946297+0000", 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028102840000000001 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946349+0000", 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1649e-05 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946393+0000", 2024-09-10T10:28:18.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4007000000000001e-05 2024-09-10T10:28:18.225 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.225 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.225 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.225 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.225 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.225 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:18.227 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:18.227 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-10T10:28:18.228 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:18.226+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:18.327 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:18.326+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:18.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.422+0000 7fec36c1e640 1 Processor -- start 2024-09-10T10:28:18.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.422+0000 7fec36c1e640 1 -- start start 2024-09-10T10:28:18.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.423+0000 7fec36c1e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fec300cb2f0 0x7fec3015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.423+0000 7fec2ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fec300cb2f0 0x7fec3015aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.423+0000 7fec349ba640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fec300cc0c0 0x7fec3015e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:39302/0 (socket says 172.21.15.120:39302) 2024-09-10T10:28:18.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.423+0000 7fec351bb640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fec300cc4a0 0x7fec30165430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:47380/0 (socket says 172.21.15.120:47380) 2024-09-10T10:28:18.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.423+0000 7fec2ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fec300cb2f0 0x7fec3015aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:50146/0 (socket says 172.21.15.120:50146) 2024-09-10T10:28:18.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec349ba640 1 -- 172.21.15.120:0/4277009756 learned_addr learned my addr 172.21.15.120:0/4277009756 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:18.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec36c1e640 1 -- 172.21.15.120:0/4277009756 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec3015b1d0 con 0x7fec300cc4a0 2024-09-10T10:28:18.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec36c1e640 1 -- 172.21.15.120:0/4277009756 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec3015b4b0 con 0x7fec300cc0c0 2024-09-10T10:28:18.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec2f7fe640 1 -- 172.21.15.120:0/4277009756 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918119560 0 0) 0x7fec3015b1d0 con 0x7fec300cc4a0 2024-09-10T10:28:18.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec36c1e640 1 -- 172.21.15.120:0/4277009756 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fec300cbd00 con 0x7fec300cb2f0 2024-09-10T10:28:18.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec2f7fe640 1 -- 172.21.15.120:0/4277009756 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec20003520 con 0x7fec300cc4a0 2024-09-10T10:28:18.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec2f7fe640 1 -- 172.21.15.120:0/4277009756 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1513142116 0 0) 0x7fec3015b4b0 con 0x7fec300cc0c0 2024-09-10T10:28:18.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec2f7fe640 1 -- 172.21.15.120:0/4277009756 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec3015b1d0 con 0x7fec300cc0c0 2024-09-10T10:28:18.427 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:18.428 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:18.428 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:18.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec2ffff640 1 -- 172.21.15.120:0/4277009756 >> v1:172.21.15.134:6789/0 conn(0x7fec300cc0c0 legacy=0x7fec3015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec2ffff640 1 -- 172.21.15.120:0/4277009756 >> v1:172.21.15.120:6789/0 conn(0x7fec300cc4a0 legacy=0x7fec30165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec2f7fe640 1 -- 172.21.15.120:0/4277009756 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3928842500 0 0) 0x7fec20003520 con 0x7fec300cc4a0 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.424+0000 7fec2ffff640 1 -- 172.21.15.120:0/4277009756 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3015b790 con 0x7fec300cb2f0 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.425+0000 7fec2ffff640 1 --2- 172.21.15.120:0/4277009756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 0x7fec3015aab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0097a0 tx=0x7fec1c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4db4272fa65db8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.426+0000 7fec2f7fe640 1 -- 172.21.15.120:0/4277009756 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c037070 con 0x7fec300cb2f0 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.426+0000 7fec2f7fe640 1 -- 172.21.15.120:0/4277009756 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1c032440 con 0x7fec300cb2f0 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.426+0000 7fec2f7fe640 1 -- 172.21.15.120:0/4277009756 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c032810 con 0x7fec300cb2f0 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.427+0000 7fec36c1e640 1 -- 172.21.15.120:0/4277009756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 msgr2=0x7fec3015aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.427+0000 7fec36c1e640 1 --2- 172.21.15.120:0/4277009756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 0x7fec3015aab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0097a0 tx=0x7fec1c02a230 comp rx=0 tx=0).stop 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.427+0000 7fec36c1e640 1 -- 172.21.15.120:0/4277009756 shutdown_connections 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.427+0000 7fec36c1e640 1 --2- 172.21.15.120:0/4277009756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 0x7fec3015aab0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.427+0000 7fec36c1e640 1 -- 172.21.15.120:0/4277009756 >> 172.21.15.120:0/4277009756 conn(0x7fec3006d560 msgr2=0x7fec3006d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.427+0000 7fec36c1e640 1 -- 172.21.15.120:0/4277009756 shutdown_connections 2024-09-10T10:28:18.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.427+0000 7fec36c1e640 1 -- 172.21.15.120:0/4277009756 wait complete. 2024-09-10T10:28:18.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.429+0000 7fec36c1e640 1 Processor -- start 2024-09-10T10:28:18.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.429+0000 7fec36c1e640 1 -- start start 2024-09-10T10:28:18.432 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:18.431+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:18.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.430+0000 7fec36c1e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 0x7fec301f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.430+0000 7fec36c1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec300cc0c0 0x7fec3014c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.430+0000 7fec36c1e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec300cc4a0 0x7fec3015d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.430+0000 7fec2ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec300cc0c0 0x7fec3014c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec2ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec300cc0c0 0x7fec3014c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53840/0 (socket says 172.21.15.120:53840) 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec349ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 0x7fec301f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec349ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 0x7fec301f4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:50162/0 (socket says 172.21.15.120:50162) 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec2ffff640 1 -- 172.21.15.120:0/3010819121 learned_addr learned my addr 172.21.15.120:0/3010819121 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec3015e540 con 0x7fec300cc0c0 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec3015ba70 con 0x7fec300cb2f0 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec300cc8b0 con 0x7fec300cc4a0 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec351bb640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec300cc4a0 0x7fec3015d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec349ba640 1 -- 172.21.15.120:0/3010819121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec300cc4a0 msgr2=0x7fec3015d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec349ba640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec300cc4a0 0x7fec3015d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec349ba640 1 -- 172.21.15.120:0/3010819121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec300cc0c0 msgr2=0x7fec3014c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec349ba640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec300cc0c0 0x7fec3014c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec349ba640 1 -- 172.21.15.120:0/3010819121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3014cfd0 con 0x7fec300cb2f0 2024-09-10T10:28:18.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.431+0000 7fec2ffff640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec300cc0c0 0x7fec3014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:18.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.432+0000 7fec349ba640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 0x7fec301f4770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fec2400c3e0 tx=0x7fec2400c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.432+0000 7fec2d7fa640 1 -- 172.21.15.120:0/3010819121 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec240074e0 con 0x7fec300cb2f0 2024-09-10T10:28:18.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.432+0000 7fec2d7fa640 1 -- 172.21.15.120:0/3010819121 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec240047e0 con 0x7fec300cb2f0 2024-09-10T10:28:18.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.432+0000 7fec2d7fa640 1 -- 172.21.15.120:0/3010819121 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24004c50 con 0x7fec300cb2f0 2024-09-10T10:28:18.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.432+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3014cbc0 con 0x7fec300cb2f0 2024-09-10T10:28:18.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.432+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3014f6d0 con 0x7fec300cb2f0 2024-09-10T10:28:18.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.434+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec300d10f0 con 0x7fec300cb2f0 2024-09-10T10:28:18.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.434+0000 7fec2d7fa640 1 -- 172.21.15.120:0/3010819121 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec24011070 con 0x7fec300cb2f0 2024-09-10T10:28:18.438 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:18.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.435+0000 7fec2d7fa640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec1406be30 0x7fec1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.435+0000 7fec2d7fa640 1 -- 172.21.15.120:0/3010819121 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6801+0+0 (secure 0 0 0) 0x7fec2408ae00 con 0x7fec300cb2f0 2024-09-10T10:28:18.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.435+0000 7fec2ffff640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec1406be30 0x7fec1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.435+0000 7fec2ffff640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec1406be30 0x7fec1406e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fec1c000c00 tx=0x7fec1c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.439+0000 7fec2d7fa640 1 -- 172.21.15.120:0/3010819121 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fec3014cfd0 con 0x7fec300cb2f0 2024-09-10T10:28:18.532 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:18.530+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:18.554 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:18.554 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:18.554 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:18.554 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:18.554 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:18.563 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:18.634 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:18.633+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:18.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.640+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fec300c5560 con 0x7fec300cb2f0 2024-09-10T10:28:18.688 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:28:18.713 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:18.713 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:18.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:18.713 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:18.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.931384412, 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078690670000000004, 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2200000000000004e-07 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780048+0000", 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2427999999999999e-05 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780057+0000", 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1090000000000004e-06 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780063+0000", 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9780000000000002e-06 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011213480000000001 2024-09-10T10:28:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781281+0000", 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6728e-05 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781382+0000", 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101455 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784820+0000", 2024-09-10T10:28:18.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034376200000000002 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787779+0000", 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002959373 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787852+0000", 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3135999999999999e-05 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787894+0000", 2024-09-10T10:28:18.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1369999999999999e-05 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.931336870999999, 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079289789999999992, 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780073+0000", 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780075+0000", 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5129999999999999e-06 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780082+0000", 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9410000000000001e-06 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780087+0000", 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1880000000000001e-06 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781620+0000", 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001532346 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781698+0000", 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8232999999999994e-05 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781816+0000", 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011825800000000001 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784824+0000", 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030076349999999998 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787937+0000", 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031134829999999998 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787969+0000", 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1745999999999997e-05 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788001+0000", 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2132000000000003e-05 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.931306656000004, 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079888619999999994, 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.724 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780103+0000", 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-09-10T10:28:18.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780106+0000", 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1379999999999999e-06 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780115+0000", 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4299999999999995e-06 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780122+0000", 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3380000000000001e-06 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782041+0000", 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019193700000000001 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782149+0000", 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107493 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782246+0000", 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999996e-05 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785205+0000", 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002958535 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788041+0000", 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028359650000000002 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788069+0000", 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8123000000000002e-05 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788091+0000", 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2506000000000001e-05 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.931278114999998, 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080531070000000003, 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780134+0000", 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0570000000000001e-06 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780143+0000", 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-06 2024-09-10T10:28:18.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780156+0000", 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3179e-05 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023278280000000001 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782555+0000", 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0494000000000006e-05 2024-09-10T10:28:18.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782675+0000", 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120463 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785209+0000", 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002533455 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788127+0000", 2024-09-10T10:28:18.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029181260000000001 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788155+0000", 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8495000000000001e-05 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788184+0000", 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8736999999999999e-05 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.931245598000004, 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008097524, 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780164+0000", 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.13e-07 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9780000000000001e-06 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780174+0000", 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7510000000000002e-06 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780179+0000", 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0869999999999997e-06 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782890+0000", 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027113509999999999 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782985+0000", 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5152e-05 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783090+0000", 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104947 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785222+0000", 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021320269999999999 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788216+0000", 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029940629999999999 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788242+0000", 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5259999999999999e-05 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788261+0000", 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9395e-05 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.739 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.931223227000004, 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081619529999999996, 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:18.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780189+0000", 2024-09-10T10:28:18.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.926e-06 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780197+0000", 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2039999999999997e-06 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780202+0000", 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783326+0000", 2024-09-10T10:28:18.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003123855 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783402+0000", 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6548e-05 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783558+0000", 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015539199999999999 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785225+0000", 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001666989 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788290+0000", 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030647890000000001 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788316+0000", 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6662e-05 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788348+0000", 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1417000000000001e-05 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:3 2.10 2:0f65c9b8:::60a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.839629715000001, 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066829990000000002, 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.745 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871781+0000", 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2129999999999998e-06 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871790+0000", 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3380000000000005e-06 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871805+0000", 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5353999999999999e-05 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871817+0000", 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1399999999999999e-05 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871932+0000", 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114982 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872069+0000", 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013713199999999999 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872208+0000", 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013976900000000001 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874946+0000", 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027378789999999999 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878355+0000", 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034089139999999999 2024-09-10T10:28:18.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878413+0000", 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7540000000000001e-05 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878462+0000", 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9478000000000003e-05 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.764186582999997, 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067332750000000004, 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947223+0000", 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7299999999999995e-07 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947227+0000", 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.895e-06 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947235+0000", 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947246+0000", 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0738e-05 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948505+0000", 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001259402 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948578+0000", 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2788999999999997e-05 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948718+0000", 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013953699999999999 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953391+0000", 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046736670000000003 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953881+0000", 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048956900000000003 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953923+0000", 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2153000000000001e-05 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953956+0000", 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2526e-05 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.755 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.764127794999993, 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066654160000000004, 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947282+0000", 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947285+0000", 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7689999999999998e-06 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947292+0000", 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6200000000000001e-06 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947304+0000", 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1702000000000001e-05 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947363+0000", 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9426000000000003e-05 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947485+0000", 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012227599999999999 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947588+0000", 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102641 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950690+0000", 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003102264 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953856+0000", 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031656200000000001 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953907+0000", 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1430000000000001e-05 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953947+0000", 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9164e-05 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.764090621999998, 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071148679999999999, 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.761 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947319+0000", 2024-09-10T10:28:18.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947322+0000", 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.523e-06 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947331+0000", 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3240000000000003e-06 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947336+0000", 2024-09-10T10:28:18.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7189999999999998e-06 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947994+0000", 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065770599999999998 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948080+0000", 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5649000000000006e-05 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111783 2024-09-10T10:28:18.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953965+0000", 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057732089999999996 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954343+0000", 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037846199999999998 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954395+0000", 2024-09-10T10:28:18.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.126e-05 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954433+0000", 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8693999999999999e-05 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.763960425999997, 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067554449999999997, 2024-09-10T10:28:18.766 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6899999999999996e-07 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947454+0000", 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947463+0000", 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0310000000000004e-06 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947477+0000", 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4328000000000001e-05 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3869999999999998e-05 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947631+0000", 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9874e-05 2024-09-10T10:28:18.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947750+0000", 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118483 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953644+0000", 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058946889999999998 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954111+0000", 2024-09-10T10:28:18.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00046712099999999998 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954162+0000", 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000003e-05 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954204+0000", 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1718000000000003e-05 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.763573387999998, 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064551169999999998, 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3e-07 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947841+0000", 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3460000000000004e-06 2024-09-10T10:28:18.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947856+0000", 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947864+0000", 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4939999999999997e-06 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947997+0000", 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013305399999999999 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948076+0000", 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9146000000000004e-05 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115586 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953707+0000", 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055153470000000003 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954215+0000", 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050850400000000001 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954258+0000", 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2932000000000002e-05 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954291+0000", 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2322e-05 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.776 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.763534067999998, 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088919089999999999, 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7199999999999999e-07 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947879+0000", 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8380000000000004e-06 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947887+0000", 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5410000000000003e-06 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947893+0000", 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6930000000000004e-06 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948405+0000", 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051236000000000001 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948502+0000", 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6916999999999994e-05 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948647+0000", 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000145265 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954479+0000", 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058316970000000003 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956680+0000", 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002201494 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956728+0000", 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8161999999999997e-05 2024-09-10T10:28:18.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956767+0000", 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.837e-05 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.762971406999995, 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083279530000000008, 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.782 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8999999999999995e-07 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948443+0000", 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2189999999999998e-06 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948450+0000", 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8879999999999997e-06 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948457+0000", 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0750000000000003e-06 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948501+0000", 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948594+0000", 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3009000000000003e-05 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948696+0000", 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101461 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953756+0000", 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050607129999999997 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956690+0000", 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029334109999999999 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956737+0000", 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7080999999999999e-05 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956766+0000", 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8778000000000001e-05 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.787 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.762431733, 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0077788589999999999, 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948986+0000", 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948998+0000", 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.19e-05 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949007+0000", 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.081e-06 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949062+0000", 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754999999999997e-05 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949172+0000", 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110128 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949282+0000", 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109289 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955941+0000", 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066594990000000001 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956654+0000", 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071279399999999999 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956706+0000", 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2027000000000002e-05 2024-09-10T10:28:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956756+0000", 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0012999999999997e-05 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.761893638999993, 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073200349999999999, 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.793 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949516+0000", 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999995e-07 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949522+0000", 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3980000000000004e-06 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949534+0000", 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1562e-05 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949541+0000", 2024-09-10T10:28:18.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1380000000000001e-06 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950700+0000", 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011593459999999999 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950786+0000", 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5556999999999994e-05 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950898+0000", 2024-09-10T10:28:18.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111743 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954326+0000", 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034280510000000001 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956774+0000", 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024477430000000001 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956815+0000", 2024-09-10T10:28:18.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1752999999999998e-05 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956835+0000", 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0032999999999999e-05 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.761851338, 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073303700000000001, 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.798 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949571+0000", 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2298e-05 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949579+0000", 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5909999999999996e-06 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949585+0000", 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7370000000000003e-06 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953802+0000", 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042170929999999999 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953915+0000", 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011256699999999999 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956888+0000", 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029734639999999999 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:24 2.3 e26/18 2:c216c0d1:::mds2_inotable:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 64.761033337000001, 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0063254399999999999, 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950380+0000", 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2590000000000002e-06 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950387+0000", 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950392+0000", 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-10T10:28:18.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950438+0000", 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5592e-05 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950457+0000", 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8569000000000001e-05 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956657+0000", 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062003409999999998 2024-09-10T10:28:18.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956701+0000", 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4140000000000001e-05 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:29 2.10 2:0e4da237:::203.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.789372534000002, 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064228799999999997, 2024-09-10T10:28:18.806 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922037+0000", 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922040+0000", 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.694e-06 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922054+0000", 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906e-05 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922065+0000", 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922125+0000", 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.982e-05 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922233+0000", 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108175 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928459+0000", 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062264570000000003 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 52.770108903999997, 2024-09-10T10:28:18.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081214100000000008, 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941301+0000", 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.062e-06 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941306+0000", 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0919999999999997e-06 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941316+0000", 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5640000000000002e-06 2024-09-10T10:28:18.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941323+0000", 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941405+0000", 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1650000000000006e-05 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941549+0000", 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014447599999999999 2024-09-10T10:28:18.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012929 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943549+0000", 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018704049999999999 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949343+0000", 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057943300000000003 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949378+0000", 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4474999999999998e-05 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949421+0000", 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3955999999999997e-05 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:18.815 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:18.817 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:18.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.788+0000 7f6b62b52640 1 Processor -- start 2024-09-10T10:28:18.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.788+0000 7f6b62b52640 1 -- start start 2024-09-10T10:28:18.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.788+0000 7f6b62b52640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b5c154c10 0x7f6b5c15b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b62b52640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6b5c0cafa0 0x7f6b5c0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b62b52640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b5c15e0b0 con 0x7f6b5c0cca70 2024-09-10T10:28:18.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b62b52640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f6b5c15de60 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b62b52640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6b5c15e390 con 0x7f6b5c154c10 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b5bfff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f6b5c0cca70 0x7f6b5c073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:47386/0 (socket says 172.21.15.120:47386) 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b5bfff640 1 -- 172.21.15.120:0/3397051212 learned_addr learned my addr 172.21.15.120:0/3397051212 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b610ef640 1 --2- 172.21.15.120:0/3397051212 >> v2:172.21.15.120:3300/0 conn(0x7f6b5c0cafa0 0x7f6b5c0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b608ee640 1 --2- 172.21.15.120:0/3397051212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b5c154c10 0x7f6b5c15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b610ef640 1 -- 172.21.15.120:0/3397051212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b5c154c10 msgr2=0x7f6b5c15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b610ef640 1 --2- 172.21.15.120:0/3397051212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b5c154c10 0x7f6b5c15b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b610ef640 1 -- 172.21.15.120:0/3397051212 >> v1:172.21.15.120:6789/0 conn(0x7f6b5c0cca70 legacy=0x7f6b5c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b5b7fe640 1 -- 172.21.15.120:0/3397051212 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3594637928 0 0) 0x7f6b5c15e0b0 con 0x7f6b5c0cca70 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.789+0000 7f6b610ef640 1 -- 172.21.15.120:0/3397051212 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c15e670 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.790+0000 7f6b610ef640 1 --2- 172.21.15.120:0/3397051212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c0cafa0 0x7f6b5c0cb6f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009d90 tx=0x7f6b5402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4819ac5c7aa078cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.790+0000 7f6b5b7fe640 1 -- 172.21.15.120:0/3397051212 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54007740 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.790+0000 7f6b5b7fe640 1 -- 172.21.15.120:0/3397051212 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b540078e0 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.790+0000 7f6b5b7fe640 1 -- 172.21.15.120:0/3397051212 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54007cb0 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.791+0000 7f6b62b52640 1 -- 172.21.15.120:0/3397051212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c0cafa0 msgr2=0x7f6b5c0cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.791+0000 7f6b62b52640 1 --2- 172.21.15.120:0/3397051212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c0cafa0 0x7f6b5c0cb6f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009d90 tx=0x7f6b5402a660 comp rx=0 tx=0).stop 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.792+0000 7f6b62b52640 1 -- 172.21.15.120:0/3397051212 shutdown_connections 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.792+0000 7f6b62b52640 1 --2- 172.21.15.120:0/3397051212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c0cafa0 0x7f6b5c0cb6f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.792+0000 7f6b62b52640 1 --2- 172.21.15.120:0/3397051212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b5c154c10 0x7f6b5c15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.792+0000 7f6b62b52640 1 -- 172.21.15.120:0/3397051212 >> 172.21.15.120:0/3397051212 conn(0x7f6b5c06ce80 msgr2=0x7f6b5c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.792+0000 7f6b62b52640 1 -- 172.21.15.120:0/3397051212 shutdown_connections 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.792+0000 7f6b62b52640 1 -- 172.21.15.120:0/3397051212 wait complete. 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.793+0000 7f6b62b52640 1 Processor -- start 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.793+0000 7f6b62b52640 1 -- start start 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b62b52640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c0cafa0 0x7f6b5c15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b62b52640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b5c0cca70 0x7f6b5c1f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b62b52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b5c154c10 0x7f6b5c14df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b62b52640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c154ff0 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b608ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c0cafa0 0x7f6b5c15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b608ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c0cafa0 0x7f6b5c15ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53856/0 (socket says 172.21.15.120:53856) 2024-09-10T10:28:18.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b62b52640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c15ec30 con 0x7f6b5c154c10 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b608ee640 1 -- 172.21.15.120:0/2099838757 learned_addr learned my addr 172.21.15.120:0/2099838757 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b62b52640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6b5c073a50 con 0x7f6b5c0cca70 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b5bfff640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b5c0cca70 0x7f6b5c1f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b610ef640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b5c154c10 0x7f6b5c14df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b608ee640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b5c0cca70 msgr2=0x7f6b5c1f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b608ee640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b5c0cca70 0x7f6b5c1f3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b608ee640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b5c154c10 msgr2=0x7f6b5c14df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b608ee640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b5c154c10 0x7f6b5c14df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.795+0000 7f6b608ee640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c1f0a30 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.796+0000 7f6b608ee640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c0cafa0 0x7f6b5c15ccf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000d830 tx=0x7f6b5000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.796+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5000b6c0 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.796+0000 7f6b62b52640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5c1f0cd0 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.796+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b50019070 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.796+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5000b6c0 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.796+0000 7f6b62b52640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5c0ca7e0 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.797+0000 7f6b62b52640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f6b5c15b370 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.798+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b5000b860 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.798+0000 7f6b597fa640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6b4006c060 0x7f6b4006e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.798+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6801+0+0 (secure 0 0 0) 0x7f6b5008b210 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.798+0000 7f6b5bfff640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6b4006c060 0x7f6b4006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.798+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v47) ==== 54+0+16 (secure 0 0 0) 0x7f6b5008daa0 con 0x7f6b5c0cafa0 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.799+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f6b34001650 0x7f6b34003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.799+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b34006bb0 con 0x7f6b34001650 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.799+0000 7f6b610ef640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f6b34001650 0x7f6b34003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.799+0000 7f6b5bfff640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6b4006c060 0x7f6b4006e520 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c153020 tx=0x7f6b48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.799+0000 7f6b610ef640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f6b34001650 0x7f6b34003b10 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.800+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b34006bb0 con 0x7f6b34001650 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.814+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6b34005c00 con 0x7f6b34001650 2024-09-10T10:28:18.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.816+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6b34005c00 con 0x7f6b34001650 2024-09-10T10:28:18.826 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:18.827 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:18.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.816+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f6b34004190 0x7f6b34004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.816+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f6b34004b80 con 0x7f6b34004190 2024-09-10T10:28:18.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.817+0000 7f6b608ee640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f6b34004190 0x7f6b34004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.817+0000 7f6b608ee640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f6b34004190 0x7f6b34004640 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.818+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b34004b80 con 0x7f6b34004190 2024-09-10T10:28:18.827 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:18.816+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:18.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.834+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6b34005090 con 0x7f6b34004190 2024-09-10T10:28:18.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.837+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6b34005090 con 0x7f6b34004190 2024-09-10T10:28:18.839 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:18.839 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:18.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.837+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f6b34009340 0x7f6b3400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.837+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f6b3400e5b0 con 0x7f6b34009340 2024-09-10T10:28:18.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.837+0000 7f6b5bfff640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f6b34009340 0x7f6b3400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.838+0000 7f6b5bfff640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f6b34009340 0x7f6b3400b730 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.839+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b3400e5b0 con 0x7f6b34009340 2024-09-10T10:28:18.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.860+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6b3400c000 con 0x7f6b34009340 2024-09-10T10:28:18.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.861+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6b3400c000 con 0x7f6b34009340 2024-09-10T10:28:18.864 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:18.864 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:18.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.862+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f6b3400c1b0 0x7f6b3400c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.862+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f6b3400cb50 con 0x7f6b3400c1b0 2024-09-10T10:28:18.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.862+0000 7f6b610ef640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f6b3400c1b0 0x7f6b3400c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.862+0000 7f6b610ef640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f6b3400c1b0 0x7f6b3400c610 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.863+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b3400cb50 con 0x7f6b3400c1b0 2024-09-10T10:28:18.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.877+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6b3400d060 con 0x7f6b3400c1b0 2024-09-10T10:28:18.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.880+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6b3400d060 con 0x7f6b3400c1b0 2024-09-10T10:28:18.882 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:18.883 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:18.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.880+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f6b3400d420 0x7f6b34012810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.880+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f6b34015720 con 0x7f6b3400d420 2024-09-10T10:28:18.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.881+0000 7f6b608ee640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f6b3400d420 0x7f6b34012810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.881+0000 7f6b608ee640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f6b3400d420 0x7f6b34012810 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.882+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b34015720 con 0x7f6b3400d420 2024-09-10T10:28:18.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.896+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6b34012d50 con 0x7f6b3400d420 2024-09-10T10:28:18.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.897+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6b34012d50 con 0x7f6b3400d420 2024-09-10T10:28:18.900 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:18.900 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:18.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.898+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f6b34013470 0x7f6b34013890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.898+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f6b34018af0 con 0x7f6b34013470 2024-09-10T10:28:18.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.898+0000 7f6b5bfff640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f6b34013470 0x7f6b34013890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.899+0000 7f6b5bfff640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f6b34013470 0x7f6b34013890 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.899+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b34018af0 con 0x7f6b34013470 2024-09-10T10:28:18.918 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:18.917+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.926+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6b34017970 con 0x7f6b34013470 2024-09-10T10:28:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.928+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6b34017970 con 0x7f6b34013470 2024-09-10T10:28:18.931 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.929+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f6b34017ff0 0x7f6b34018430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.929+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f6b3401beb0 con 0x7f6b34017ff0 2024-09-10T10:28:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.929+0000 7f6b610ef640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f6b34017ff0 0x7f6b34018430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.930+0000 7f6b610ef640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f6b34017ff0 0x7f6b34018430 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.931+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b3401beb0 con 0x7f6b34017ff0 2024-09-10T10:28:18.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.957+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6b34013dd0 con 0x7f6b34017ff0 2024-09-10T10:28:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.959+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6b34013dd0 con 0x7f6b34017ff0 2024-09-10T10:28:18.962 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.960+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f6b34014450 0x7f6b340148b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.960+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f6b34014f20 con 0x7f6b34014450 2024-09-10T10:28:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.960+0000 7f6b608ee640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f6b34014450 0x7f6b340148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:18.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.961+0000 7f6b608ee640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f6b34014450 0x7f6b340148b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:18.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.961+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b34014f20 con 0x7f6b34014450 2024-09-10T10:28:18.981 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:18.981 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:18.991 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:18.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.988+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6b340156f0 con 0x7f6b34014450 2024-09-10T10:28:18.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.990+0000 7f6b597fa640 1 -- 172.21.15.120:0/2099838757 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6b340156f0 con 0x7f6b34014450 2024-09-10T10:28:18.992 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:18.994 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:18.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.990+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f6b34001650 msgr2=0x7f6b34003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.990+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f6b34001650 0x7f6b34003b10 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.990+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f6b34004190 msgr2=0x7f6b34004640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.990+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f6b34004190 0x7f6b34004640 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.990+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f6b34009340 msgr2=0x7f6b3400b730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.990+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f6b34009340 0x7f6b3400b730 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f6b3400c1b0 msgr2=0x7f6b3400c610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f6b3400c1b0 0x7f6b3400c610 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f6b3400d420 msgr2=0x7f6b34012810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f6b3400d420 0x7f6b34012810 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f6b34013470 msgr2=0x7f6b34013890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f6b34013470 0x7f6b34013890 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f6b34017ff0 msgr2=0x7f6b34018430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f6b34017ff0 0x7f6b34018430 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f6b34014450 msgr2=0x7f6b340148b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f6b34014450 0x7f6b340148b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b610ef640 1 -- 172.21.15.120:0/2099838757 reap_dead start 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b610ef640 1 -- 172.21.15.120:0/2099838757 reap_dead start 2024-09-10T10:28:18.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6b4006c060 msgr2=0x7f6b4006e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f6b4006c060 0x7f6b4006e520 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c153020 tx=0x7f6b48009040 comp rx=0 tx=0).stop 2024-09-10T10:28:18.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c0cafa0 msgr2=0x7f6b5c15ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:18.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 --2- 172.21.15.120:0/2099838757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c0cafa0 0x7f6b5c15ccf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000d830 tx=0x7f6b5000dcf0 comp rx=0 tx=0).stop 2024-09-10T10:28:18.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b610ef640 1 -- 172.21.15.120:0/2099838757 reap_dead start 2024-09-10T10:28:18.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b610ef640 1 -- 172.21.15.120:0/2099838757 reap_dead start 2024-09-10T10:28:18.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b610ef640 1 -- 172.21.15.120:0/2099838757 reap_dead start 2024-09-10T10:28:18.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 shutdown_connections 2024-09-10T10:28:18.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 >> 172.21.15.120:0/2099838757 conn(0x7f6b5c06ce80 msgr2=0x7f6b5c153700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:18.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 shutdown_connections 2024-09-10T10:28:18.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:18.991+0000 7f6b46ffd640 1 -- 172.21.15.120:0/2099838757 wait complete. 2024-09-10T10:28:19.049 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:19.047+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:19.105 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:19.105 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:19.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:19.119 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:19.118+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:19.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.214+0000 7fec2d7fa640 1 -- 172.21.15.120:0/3010819121 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v48) ==== 91+0+0 (secure 0 0 0) 0x7fec24058d00 con 0x7fec300cb2f0 2024-09-10T10:28:19.217 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-10T10:28:19.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec1406be30 msgr2=0x7fec1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:19.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec1406be30 0x7fec1406e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fec1c000c00 tx=0x7fec1c02e000 comp rx=0 tx=0).stop 2024-09-10T10:28:19.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 msgr2=0x7fec301f4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:19.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 0x7fec301f4770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fec2400c3e0 tx=0x7fec2400c8a0 comp rx=0 tx=0).stop 2024-09-10T10:28:19.220 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:19.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 shutdown_connections 2024-09-10T10:28:19.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec1406be30 0x7fec1406e2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:19.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec300cc4a0 0x7fec3015d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:19.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec300cc0c0 0x7fec3014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:19.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 --2- 172.21.15.120:0/3010819121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec300cb2f0 0x7fec301f4770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:19.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 >> 172.21.15.120:0/3010819121 conn(0x7fec3006d560 msgr2=0x7fec300cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:19.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 shutdown_connections 2024-09-10T10:28:19.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:19.218+0000 7fec36c1e640 1 -- 172.21.15.120:0/3010819121 wait complete. 2024-09-10T10:28:19.268 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:19.268 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:19.278 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:19.323 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:19.322+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:19.396 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:19.396 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:19.405 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:19.446 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:19.445+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:19.521 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:19.520+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:19.550 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:19.550 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:19.558 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:19.622 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:19.621+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:19.662 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:19.663 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:19.671 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:19.756 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:19.755+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:19.816 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:19.817 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:19.823 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:19.822+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:19.826 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:19.924 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:19.923+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:19.970 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:19.970 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:19.979 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:20.063 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:20.062+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:20.123 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:20.123 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:20.125 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:20.124+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:20.132 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:20.226 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:20.226+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:20.278 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:20.278 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:20.287 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:20.327 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:20.326+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:20.354 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:28:20.354 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:28:20.354 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-10T10:28:20.354 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-10T10:28:20.428 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:20.426+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:20.429 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:20.430 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:20.439 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:20.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.508+0000 7f770ec77640 1 Processor -- start 2024-09-10T10:28:20.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.508+0000 7f770ec77640 1 -- start start 2024-09-10T10:28:20.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f770ec77640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f77080cb2f0 0x7f770815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:20.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f770ec77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 0x7f7708165430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:20.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f770ec77640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f770815b1d0 con 0x7f77080cc0c0 2024-09-10T10:28:20.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f770ec77640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f77080cbd00 con 0x7f77080cb2f0 2024-09-10T10:28:20.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f770ec77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f770815ba70 con 0x7f77080cc4a0 2024-09-10T10:28:20.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f7707fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f77080cb2f0 0x7f770815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:20.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f7707fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f77080cb2f0 0x7f770815aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53868/0 (socket says 172.21.15.120:53868) 2024-09-10T10:28:20.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f7707fff640 1 -- 172.21.15.120:0/3161162024 learned_addr learned my addr 172.21.15.120:0/3161162024 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:20.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f770d214640 1 --2- 172.21.15.120:0/3161162024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 0x7f7708165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:20.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f7707fff640 1 -- 172.21.15.120:0/3161162024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 msgr2=0x7f7708165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:20.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f7707fff640 1 --2- 172.21.15.120:0/3161162024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 0x7f7708165430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:20.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f7707fff640 1 -- 172.21.15.120:0/3161162024 >> v1:172.21.15.120:6789/0 conn(0x7f77080cc0c0 legacy=0x7f770815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:20.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f7707fff640 1 -- 172.21.15.120:0/3161162024 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770815b4b0 con 0x7f77080cb2f0 2024-09-10T10:28:20.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.510+0000 7f770d214640 1 --2- 172.21.15.120:0/3161162024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 0x7f7708165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:20.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.511+0000 7f7707fff640 1 --2- 172.21.15.120:0/3161162024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 0x7f770815aab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009870 tx=0x7f76f402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaed0166b03c97a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:20.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.511+0000 7f77077fe640 1 -- 172.21.15.120:0/3161162024 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4037070 con 0x7f77080cb2f0 2024-09-10T10:28:20.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.512+0000 7f77077fe640 1 -- 172.21.15.120:0/3161162024 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f4032440 con 0x7f77080cb2f0 2024-09-10T10:28:20.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.512+0000 7f77077fe640 1 -- 172.21.15.120:0/3161162024 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4032810 con 0x7f77080cb2f0 2024-09-10T10:28:20.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.512+0000 7f770ec77640 1 -- 172.21.15.120:0/3161162024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 msgr2=0x7f770815aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:20.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.512+0000 7f770ec77640 1 --2- 172.21.15.120:0/3161162024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 0x7f770815aab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009870 tx=0x7f76f402a250 comp rx=0 tx=0).stop 2024-09-10T10:28:20.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.513+0000 7f770ec77640 1 -- 172.21.15.120:0/3161162024 shutdown_connections 2024-09-10T10:28:20.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.513+0000 7f770ec77640 1 --2- 172.21.15.120:0/3161162024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 0x7f7708165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:20.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.513+0000 7f770ec77640 1 --2- 172.21.15.120:0/3161162024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 0x7f770815aab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:20.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.513+0000 7f770ec77640 1 -- 172.21.15.120:0/3161162024 >> 172.21.15.120:0/3161162024 conn(0x7f770806d560 msgr2=0x7f770806d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:20.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.513+0000 7f770ec77640 1 -- 172.21.15.120:0/3161162024 shutdown_connections 2024-09-10T10:28:20.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.513+0000 7f770ec77640 1 -- 172.21.15.120:0/3161162024 wait complete. 2024-09-10T10:28:20.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.514+0000 7f770ec77640 1 Processor -- start 2024-09-10T10:28:20.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.514+0000 7f770ec77640 1 -- start start 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ec77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 0x7f770814ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ec77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77080cc0c0 0x7f77081f1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ca13640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 0x7f770814ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ca13640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 0x7f770814ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53884/0 (socket says 172.21.15.120:53884) 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ca13640 1 -- 172.21.15.120:0/2360449905 learned_addr learned my addr 172.21.15.120:0/2360449905 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ec77640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 0x7f770814d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f770815e1d0 con 0x7f77080cb2f0 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f770815e540 con 0x7f77080cc0c0 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f77080cc8f0 con 0x7f77080cc4a0 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f7707fff640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77080cc0c0 0x7f77081f1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770d214640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 0x7f770814d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ca13640 1 -- 172.21.15.120:0/2360449905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 msgr2=0x7f770814d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:20.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ca13640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 0x7f770814d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:20.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ca13640 1 -- 172.21.15.120:0/2360449905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77080cc0c0 msgr2=0x7f77081f1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:20.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.516+0000 7f770ca13640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77080cc0c0 0x7f77081f1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:20.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.517+0000 7f770ca13640 1 -- 172.21.15.120:0/2360449905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77081f3c90 con 0x7f77080cb2f0 2024-09-10T10:28:20.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.517+0000 7f770ca13640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 0x7f770814ca80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00c430 tx=0x7f76fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:20.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.517+0000 7f77057fa640 1 -- 172.21.15.120:0/2360449905 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0052d0 con 0x7f77080cb2f0 2024-09-10T10:28:20.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.517+0000 7f77057fa640 1 -- 172.21.15.120:0/2360449905 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76fc005490 con 0x7f77080cb2f0 2024-09-10T10:28:20.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.517+0000 7f77057fa640 1 -- 172.21.15.120:0/2360449905 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc007260 con 0x7f77080cb2f0 2024-09-10T10:28:20.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.517+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77080d0070 con 0x7f77080cb2f0 2024-09-10T10:28:20.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.517+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77080d0350 con 0x7f77080cb2f0 2024-09-10T10:28:20.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.518+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7708068ef0 con 0x7f77080cb2f0 2024-09-10T10:28:20.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.519+0000 7f77057fa640 1 -- 172.21.15.120:0/2360449905 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76fc007400 con 0x7f77080cb2f0 2024-09-10T10:28:20.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.520+0000 7f77057fa640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f76ec06be80 0x7f76ec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:20.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.520+0000 7f77057fa640 1 -- 172.21.15.120:0/2360449905 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6801+0+0 (secure 0 0 0) 0x7f76fc08a640 con 0x7f77080cb2f0 2024-09-10T10:28:20.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.520+0000 7f7707fff640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f76ec06be80 0x7f76ec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:20.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.521+0000 7f7707fff640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f76ec06be80 0x7f76ec06e340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f76f4034000 tx=0x7f76f402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:20.529 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:20.527+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:20.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.528+0000 7f77057fa640 1 -- 172.21.15.120:0/2360449905 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f76fc0586c0 con 0x7f77080cb2f0 2024-09-10T10:28:20.541 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:20.541 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:20.550 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:20.630 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:20.628+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:20.652 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:20.652 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:20.661 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:20.731 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:20.729+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:20.764 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:20.765 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:20.773 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:20.831 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:20.830+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:20.876 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:20.876 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:20.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.876+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f77080cbea0 con 0x7f77080cb2f0 2024-09-10T10:28:20.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.878+0000 7f77057fa640 1 -- 172.21.15.120:0/2360449905 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v48) ==== 81+0+39222 (secure 0 0 0) 0x7f76fc05c9c0 con 0x7f77080cb2f0 2024-09-10T10:28:20.880 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 48, 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-10T10:26:18.018818+0000", 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-10T10:28:19.205191+0000", 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-10T10:26:29.078139+0000", 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-10T10:26:20.091201+0000", 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5804032, 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "nodeep-scrub", 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-10T10:28:20.881 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 4, 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "luminous", 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.882 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:30.118649+0000", 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:20.883 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-10T10:28:20.884 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:20.885 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:20.886 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:20.887 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.888 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_metadata", 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:38.936606+0000", 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:20.889 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "20", 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:20.890 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:20.891 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:20.892 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_bias": 4, 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 16, 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_priority": 5 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: "metadata": "cephfs" 2024-09-10T10:28:20.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-10T10:28:20.894 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data", 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:40.094780+0000", 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:20.895 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 64, 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 64, 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 64, 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 64, 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 64, 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:20.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:20.897 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.898 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.05000000074505806 2024-09-10T10:28:20.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:28:20.900 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 4, 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data_ec", 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:43.814005+0000", 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 5, 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,ec_overwrites", 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 3, 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 4, 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 3, 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 1, 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:20.901 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:20.902 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "26", 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:20.903 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:20.904 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "cephfs_data_ec", 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 8192, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:20.905 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e3b9c3c3-cf7d-4b33-913e-60e928439d8e", 2024-09-10T10:28:20.906 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 44, 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:20.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.908 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.909 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-10T10:28:20.910 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/3664783490", 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/3664783490", 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/3664783490", 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/3664783490", 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:20.911 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "62c20a9d-e6ab-446d-998c-9ee46e46aa90", 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:20.912 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-10T10:28:20.913 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.914 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.915 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.916 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/3972761964", 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/3972761964", 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/3972761964", 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/3972761964", 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1528f8e0-5e13-4ec7-9d98-0d94b7a505c3", 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:20.918 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.919 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:20.920 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:20.921 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:20.922 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2916410859", 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2916410859", 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2916410859", 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2916410859", 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "74942f78-c1fc-4886-abf0-950722c5edc6", 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:20.923 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 44, 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:20.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-10T10:28:20.925 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.927 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1224278950", 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1224278950", 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1224278950", 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1224278950", 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e71319d4-5318-4441-84a8-7665ae037596", 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 0.9353485107421875, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:20.929 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6816", 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6817", 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:20.930 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6818", 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6819", 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:20.931 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6822", 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6823", 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.932 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6820", 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6821", 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6817/1874635282", 2024-09-10T10:28:20.933 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6819/1874635282", 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6823/1874635282", 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6821/1874635282", 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "29e193cd-37af-43e5-bd0c-2f85ce0631e8", 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:20.934 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 27, 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6808", 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6809", 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6810", 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:20.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6811", 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6814", 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6815", 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6812", 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.938 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6813", 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6809/457724087", 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6811/457724087", 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6815/457724087", 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6813/457724087", 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "55523ec2-cf12-44b3-aeea-f1cedabc996a", 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 44, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.940 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6824", 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6825", 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.941 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6826", 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6827", 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6830", 2024-09-10T10:28:20.942 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6831", 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6828", 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6829", 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6825/50133702", 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6827/50133702", 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6831/50133702", 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6829/50133702", 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:20.944 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "097b1149-3e3c-4e5a-890b-d4cc4361d51b", 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 35, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:20.945 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6800", 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6801", 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:20.946 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6802", 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6803", 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6806", 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6807", 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.948 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6804", 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6805", 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6801/251960875", 2024-09-10T10:28:20.949 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6803/251960875", 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6807/251960875", 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6805/251960875", 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:20.950 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.143056+0000", 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.423968+0000", 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.951 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.640076+0000", 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:20.952 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.536160+0000", 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:20.953 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.146033+0000", 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.426011+0000", 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.598992+0000", 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:28:20.955 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.182929+0000", 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [ 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.34", 2024-09-10T10:28:20.956 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: 3, 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: 1 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [ 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "2.1d", 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 6, 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 4 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.16", 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 1 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.1d", 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.958 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.28", 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.959 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.36", 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.4", 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 6 2024-09-10T10:28:20.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 2, 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 1 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.d", 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 5 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.1b", 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-10T10:28:20.962 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_data_ec": { 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-device-class": "", 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-num-failure-domains": "0", 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-osds-per-failure-domain": "0", 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-root": "default", 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "jerasure-per-chunk-alignment": "false", 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "2", 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van", 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "w": "8" 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.963 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [], 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [], 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-10T10:28:20.964 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-10T10:28:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-10T10:28:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-10T10:28:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-10T10:28:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-10T10:28:20.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:20.965 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:20.967 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:20.965+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:20.967 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f76ec06be80 msgr2=0x7f76ec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f76ec06be80 0x7f76ec06e340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f76f4034000 tx=0x7f76f402e000 comp rx=0 tx=0).stop 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 msgr2=0x7f770814ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 0x7f770814ca80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00c430 tx=0x7f76fc00c8f0 comp rx=0 tx=0).stop 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 shutdown_connections 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f76ec06be80 0x7f76ec06e340 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77080cc4a0 0x7f770814d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77080cc0c0 0x7f77081f1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 --2- 172.21.15.120:0/2360449905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77080cb2f0 0x7f770814ca80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 >> 172.21.15.120:0/2360449905 conn(0x7f770806d560 msgr2=0x7f770815a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 shutdown_connections 2024-09-10T10:28:20.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:20.881+0000 7f770ec77640 1 -- 172.21.15.120:0/2360449905 wait complete. 2024-09-10T10:28:20.972 INFO:tasks.thrashosds.thrasher:j is {'epoch': 48, 'fsid': '7d0d0830-4370-41c3-9acc-b67b07024f68', 'created': '2024-09-10T10:26:18.018818+0000', 'modified': '2024-09-10T10:28:19.205191+0000', 'last_up_change': '2024-09-10T10:26:29.078139+0000', 'last_in_change': '2024-09-10T10:26:20.091201+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:26:30.118649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:26:38.936606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:26:40.094780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T10:26:43.814005+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'e3b9c3c3-cf7d-4b33-913e-60e928439d8e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 3664783490}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 3664783490}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 3664783490}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 3664783490}]}, 'public_addr': '172.21.15.120:6809/3664783490', 'cluster_addr': '172.21.15.120:6811/3664783490', 'heartbeat_back_addr': '172.21.15.120:6815/3664783490', 'heartbeat_front_addr': '172.21.15.120:6813/3664783490', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '62c20a9d-e6ab-446d-998c-9ee46e46aa90', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 3972761964}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 3972761964}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 3972761964}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 3972761964}]}, 'public_addr': '172.21.15.120:6801/3972761964', 'cluster_addr': '172.21.15.120:6803/3972761964', 'heartbeat_back_addr': '172.21.15.120:6807/3972761964', 'heartbeat_front_addr': '172.21.15.120:6805/3972761964', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1528f8e0-5e13-4ec7-9d98-0d94b7a505c3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2916410859}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2916410859}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2916410859}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2916410859}]}, 'public_addr': '172.21.15.120:6817/2916410859', 'cluster_addr': '172.21.15.120:6819/2916410859', 'heartbeat_back_addr': '172.21.15.120:6823/2916410859', 'heartbeat_front_addr': '172.21.15.120:6821/2916410859', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '74942f78-c1fc-4886-abf0-950722c5edc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1224278950}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1224278950}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1224278950}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1224278950}]}, 'public_addr': '172.21.15.120:6825/1224278950', 'cluster_addr': '172.21.15.120:6827/1224278950', 'heartbeat_back_addr': '172.21.15.120:6831/1224278950', 'heartbeat_front_addr': '172.21.15.120:6829/1224278950', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e71319d4-5318-4441-84a8-7665ae037596', 'up': 1, 'in': 1, 'weight': 0.9353485107421875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6816', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6817', 'nonce': 1874635282}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6818', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6819', 'nonce': 1874635282}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6822', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6823', 'nonce': 1874635282}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6820', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6821', 'nonce': 1874635282}]}, 'public_addr': '172.21.15.134:6817/1874635282', 'cluster_addr': '172.21.15.134:6819/1874635282', 'heartbeat_back_addr': '172.21.15.134:6823/1874635282', 'heartbeat_front_addr': '172.21.15.134:6821/1874635282', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '29e193cd-37af-43e5-bd0c-2f85ce0631e8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6808', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6809', 'nonce': 457724087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6810', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6811', 'nonce': 457724087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6814', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6815', 'nonce': 457724087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6812', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6813', 'nonce': 457724087}]}, 'public_addr': '172.21.15.134:6809/457724087', 'cluster_addr': '172.21.15.134:6811/457724087', 'heartbeat_back_addr': '172.21.15.134:6815/457724087', 'heartbeat_front_addr': '172.21.15.134:6813/457724087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '55523ec2-cf12-44b3-aeea-f1cedabc996a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6824', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6825', 'nonce': 50133702}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6826', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6827', 'nonce': 50133702}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6830', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6831', 'nonce': 50133702}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6828', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6829', 'nonce': 50133702}]}, 'public_addr': '172.21.15.134:6825/50133702', 'cluster_addr': '172.21.15.134:6827/50133702', 'heartbeat_back_addr': '172.21.15.134:6831/50133702', 'heartbeat_front_addr': '172.21.15.134:6829/50133702', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '097b1149-3e3c-4e5a-890b-d4cc4361d51b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6800', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6801', 'nonce': 251960875}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6802', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6803', 'nonce': 251960875}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6806', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6807', 'nonce': 251960875}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6804', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6805', 'nonce': 251960875}]}, 'public_addr': '172.21.15.134:6801/251960875', 'cluster_addr': '172.21.15.134:6803/251960875', 'heartbeat_back_addr': '172.21.15.134:6807/251960875', 'heartbeat_front_addr': '172.21.15.134:6805/251960875', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.143056+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.423968+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.640076+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.536160+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.146033+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.426011+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.598992+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.182929+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.34', 'osds': [3, 1]}], 'pg_upmap_items': [{'pgid': '2.1d', 'mappings': [{'from': 6, 'to': 4}]}, {'pgid': '3.16', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.1d', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.28', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.36', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '4.4', 'mappings': [{'from': 3, 'to': 6}, {'from': 2, 'to': 1}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'cephfs_data_ec': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '2', 'plugin': 'jerasure', 'technique': 'reed_sol_van', 'w': '8'}, 'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-10T10:28:20.973 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:28:21.008 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:28:21.067 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:21.065+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:21.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.156+0000 7ff12e0e6640 1 Processor -- start 2024-09-10T10:28:21.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.156+0000 7ff12e0e6640 1 -- start start 2024-09-10T10:28:21.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.158+0000 7ff12e0e6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128162d50 0x7ff12816a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.158+0000 7ff12e0e6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff128159320 con 0x7ff128158f40 2024-09-10T10:28:21.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.158+0000 7ff12e0e6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1280ca3d0 con 0x7ff1280c9a20 2024-09-10T10:28:21.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.158+0000 7ff12e0e6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff1280ca1a0 con 0x7ff128162d50 2024-09-10T10:28:21.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.158+0000 7ff1277fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff128158f40 0x7ff12806d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:47408/0 (socket says 172.21.15.120:47408) 2024-09-10T10:28:21.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.158+0000 7ff1277fe640 1 -- 172.21.15.120:0/2621869604 learned_addr learned my addr 172.21.15.120:0/2621869604 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:21.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.160+0000 7ff127fff640 1 --2- 172.21.15.120:0/2621869604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128162d50 0x7ff12816a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.160+0000 7ff1267fc640 1 -- 172.21.15.120:0/2621869604 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3716614060 0 0) 0x7ff128159320 con 0x7ff128158f40 2024-09-10T10:28:21.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.160+0000 7ff1267fc640 1 -- 172.21.15.120:0/2621869604 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff110003520 con 0x7ff128158f40 2024-09-10T10:28:21.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.160+0000 7ff127fff640 1 -- 172.21.15.120:0/2621869604 >> v1:172.21.15.134:6789/0 conn(0x7ff1280c9a20 legacy=0x7ff128162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.160+0000 7ff127fff640 1 -- 172.21.15.120:0/2621869604 >> v1:172.21.15.120:6789/0 conn(0x7ff128158f40 legacy=0x7ff12806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.160+0000 7ff127fff640 1 -- 172.21.15.120:0/2621869604 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1280ca6b0 con 0x7ff128162d50 2024-09-10T10:28:21.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.162+0000 7ff127fff640 1 --2- 172.21.15.120:0/2621869604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128162d50 0x7ff12816a600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff120009dc0 tx=0x7ff12002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ce2ed57d9482f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.162+0000 7ff1267fc640 1 -- 172.21.15.120:0/2621869604 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120035020 con 0x7ff128162d50 2024-09-10T10:28:21.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.162+0000 7ff1267fc640 1 -- 172.21.15.120:0/2621869604 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff120007540 con 0x7ff128162d50 2024-09-10T10:28:21.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.162+0000 7ff1267fc640 1 -- 172.21.15.120:0/2621869604 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118001670 con 0x7ff128162d50 2024-09-10T10:28:21.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.163+0000 7f5ef60a6640 1 Processor -- start 2024-09-10T10:28:21.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.163+0000 7ff12e0e6640 1 -- 172.21.15.120:0/2621869604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128162d50 msgr2=0x7ff12816a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.163+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/2621869604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128162d50 0x7ff12816a600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff120009dc0 tx=0x7ff12002a640 comp rx=0 tx=0).stop 2024-09-10T10:28:21.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.163+0000 7ff12e0e6640 1 -- 172.21.15.120:0/2621869604 shutdown_connections 2024-09-10T10:28:21.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.163+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/2621869604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128162d50 0x7ff12816a600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.163+0000 7ff12e0e6640 1 -- 172.21.15.120:0/2621869604 >> 172.21.15.120:0/2621869604 conn(0x7ff12806bc10 msgr2=0x7ff12806bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:21.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.163+0000 7f5ef60a6640 1 -- start start 2024-09-10T10:28:21.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.164+0000 7f5ef60a6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5ef00cc780 0x7f5ef015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.164+0000 7f5ef60a6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ef00ccf60 0x7f5ef0072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.164+0000 7f5ef60a6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef0162790 con 0x7f5ef00ccb60 2024-09-10T10:28:21.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.164+0000 7f5ef60a6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef0072a60 con 0x7f5ef00cc780 2024-09-10T10:28:21.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.164+0000 7f5ef60a6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef0162a50 con 0x7f5ef00ccf60 2024-09-10T10:28:21.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.163+0000 7ff12e0e6640 1 -- 172.21.15.120:0/2621869604 shutdown_connections 2024-09-10T10:28:21.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.163+0000 7ff12e0e6640 1 -- 172.21.15.120:0/2621869604 wait complete. 2024-09-10T10:28:21.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7f5eeffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ef00ccf60 0x7f5ef0072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7f5eef7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5ef00cc780 0x7f5ef015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7f5eeeffd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f5ef00ccb60 0x7f5ef0158be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:47424/0 (socket says 172.21.15.120:47424) 2024-09-10T10:28:21.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7f5eeeffd640 1 -- 172.21.15.120:0/2332293427 learned_addr learned my addr 172.21.15.120:0/2332293427 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:21.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7f5eee7fc640 1 -- 172.21.15.120:0/2332293427 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2351865325 0 0) 0x7f5ef0162790 con 0x7f5ef00ccb60 2024-09-10T10:28:21.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7f5eee7fc640 1 -- 172.21.15.120:0/2332293427 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ed8003520 con 0x7f5ef00ccb60 2024-09-10T10:28:21.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7f5eef7fe640 1 -- 172.21.15.120:0/2332293427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ef00ccf60 msgr2=0x7f5ef0072ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:21.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7f5eef7fe640 1 --2- 172.21.15.120:0/2332293427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ef00ccf60 0x7f5ef0072ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7f5eef7fe640 1 -- 172.21.15.120:0/2332293427 >> v1:172.21.15.120:6789/0 conn(0x7f5ef00ccb60 legacy=0x7f5ef0158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7f5eef7fe640 1 -- 172.21.15.120:0/2332293427 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef0162d30 con 0x7f5ef00cc780 2024-09-10T10:28:21.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7ff12e0e6640 1 Processor -- start 2024-09-10T10:28:21.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.165+0000 7ff12e0e6640 1 -- start start 2024-09-10T10:28:21.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.166+0000 7ff12e0e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1280c9a20 0x7ff1281f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.166+0000 7f5eef7fe640 1 --2- 172.21.15.120:0/2332293427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00cc780 0x7f5ef015ed40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009a00 tx=0x7f5ee002a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b62a7e3066f6703 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.166+0000 7ff12e0e6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128158f40 0x7ff128200f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff12e0e6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff128162d50 0x7ff1282014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff12e0e6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1280cbd20 con 0x7ff128162d50 2024-09-10T10:28:21.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff12e0e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1280cac70 con 0x7ff1280c9a20 2024-09-10T10:28:21.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff12e0e6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff1280c8df0 con 0x7ff128158f40 2024-09-10T10:28:21.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7f5eee7fc640 1 -- 172.21.15.120:0/2332293427 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0036070 con 0x7f5ef00cc780 2024-09-10T10:28:21.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7f5eee7fc640 1 -- 172.21.15.120:0/2332293427 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee0031600 con 0x7f5ef00cc780 2024-09-10T10:28:21.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7f5eee7fc640 1 -- 172.21.15.120:0/2332293427 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00319d0 con 0x7f5ef00cc780 2024-09-10T10:28:21.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff1277fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1280c9a20 0x7ff1281f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff126ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128158f40 0x7ff128200f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff1277fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1280c9a20 0x7ff1281f30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:50188/0 (socket says 172.21.15.120:50188) 2024-09-10T10:28:21.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff126ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128158f40 0x7ff128200f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:41578/0 (socket says 172.21.15.120:41578) 2024-09-10T10:28:21.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff1277fe640 1 -- 172.21.15.120:0/3424263445 learned_addr learned my addr 172.21.15.120:0/3424263445 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:21.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff127fff640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff128162d50 0x7ff1282014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff1277fe640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128158f40 msgr2=0x7ff128200f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff1277fe640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff128158f40 0x7ff128200f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff1277fe640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff128162d50 msgr2=0x7ff1282014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff1277fe640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff128162d50 0x7ff1282014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff1277fe640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1281596e0 con 0x7ff1280c9a20 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff1277fe640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1280c9a20 0x7ff1281f30c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff11800c800 tx=0x7ff11800ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff127fff640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff128162d50 0x7ff1282014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.167+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118018070 con 0x7ff1280c9a20 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.168+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1180074e0 con 0x7ff1280c9a20 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.168+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118004a00 con 0x7ff1280c9a20 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.168+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1281599a0 con 0x7ff1280c9a20 2024-09-10T10:28:21.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.168+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff128159f60 con 0x7ff1280c9a20 2024-09-10T10:28:21.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.168+0000 7f5ef60a6640 1 -- 172.21.15.120:0/2332293427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00cc780 msgr2=0x7f5ef015ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.168+0000 7f5ef60a6640 1 --2- 172.21.15.120:0/2332293427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00cc780 0x7f5ef015ed40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009a00 tx=0x7f5ee002a400 comp rx=0 tx=0).stop 2024-09-10T10:28:21.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.168+0000 7f5ef60a6640 1 -- 172.21.15.120:0/2332293427 shutdown_connections 2024-09-10T10:28:21.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.168+0000 7f5ef60a6640 1 --2- 172.21.15.120:0/2332293427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ef00ccf60 0x7f5ef0072ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.168+0000 7f5ef60a6640 1 --2- 172.21.15.120:0/2332293427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00cc780 0x7f5ef015ed40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.168+0000 7f5ef60a6640 1 -- 172.21.15.120:0/2332293427 >> 172.21.15.120:0/2332293427 conn(0x7f5ef006bc10 msgr2=0x7f5ef006bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:21.177 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:21.177 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:21.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.169+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ff1280c9440 con 0x7ff1280c9a20 2024-09-10T10:28:21.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.170+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff11801b070 con 0x7ff1280c9a20 2024-09-10T10:28:21.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.169+0000 7f5ef60a6640 1 -- 172.21.15.120:0/2332293427 shutdown_connections 2024-09-10T10:28:21.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.169+0000 7f5ef60a6640 1 -- 172.21.15.120:0/2332293427 wait complete. 2024-09-10T10:28:21.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.170+0000 7ff10ffff640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff11406bf00 0x7ff11406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.170+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6801+0+0 (secure 0 0 0) 0x7ff11808a8d0 con 0x7ff1280c9a20 2024-09-10T10:28:21.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.171+0000 7ff126ffd640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff11406bf00 0x7ff11406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.171+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v48) ==== 54+0+16 (secure 0 0 0) 0x7ff11808f220 con 0x7ff1280c9a20 2024-09-10T10:28:21.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.171+0000 7f5ef60a6640 1 Processor -- start 2024-09-10T10:28:21.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.171+0000 7f5ef60a6640 1 -- start start 2024-09-10T10:28:21.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.171+0000 7ff126ffd640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff11406bf00 0x7ff11406e3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff128152c60 tx=0x7ff11c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.171+0000 7ff10dffb640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ff128158500 0x7ff128156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.171+0000 7ff10dffb640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff128156500 con 0x7ff128158500 2024-09-10T10:28:21.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.172+0000 7ff127fff640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ff128158500 0x7ff128156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.172+0000 7ff127fff640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ff128158500 0x7ff128156760 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.172+0000 7f5ef60a6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ef00cc780 0x7f5ef0152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.172+0000 7f5ef60a6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef00ccb60 0x7f5ef0070120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5ef60a6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00ccf60 0x7f5ef01610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5ef60a6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01592f0 con 0x7f5ef00ccf60 2024-09-10T10:28:21.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5ef60a6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef00c8cc0 con 0x7f5ef00ccb60 2024-09-10T10:28:21.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5ef60a6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef0159470 con 0x7f5ef00cc780 2024-09-10T10:28:21.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5eeffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00ccf60 0x7f5ef01610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5eeffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00ccf60 0x7f5ef01610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53908/0 (socket says 172.21.15.120:53908) 2024-09-10T10:28:21.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5eeffff640 1 -- 172.21.15.120:0/1872211229 learned_addr learned my addr 172.21.15.120:0/1872211229 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:21.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5eeeffd640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef00ccb60 0x7f5ef0070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff128156500 con 0x7ff128158500 2024-09-10T10:28:21.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5eef7fe640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ef00cc780 0x7f5ef0152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5eeeffd640 1 -- 172.21.15.120:0/1872211229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ef00cc780 msgr2=0x7f5ef0152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5eeeffd640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ef00cc780 0x7f5ef0152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.173+0000 7f5eeeffd640 1 -- 172.21.15.120:0/1872211229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00ccf60 msgr2=0x7f5ef01610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.174+0000 7f5eeeffd640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00ccf60 0x7f5ef01610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.174+0000 7f5eeeffd640 1 -- 172.21.15.120:0/1872211229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef00cc340 con 0x7f5ef00ccb60 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.174+0000 7f5eeffff640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00ccf60 0x7f5ef01610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.174+0000 7f5eeeffd640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef00ccb60 0x7f5ef0070120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400c410 tx=0x7f5ee400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.174+0000 7f5ed7fff640 1 -- 172.21.15.120:0/1872211229 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40074e0 con 0x7f5ef00ccb60 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.174+0000 7f5ed7fff640 1 -- 172.21.15.120:0/1872211229 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee40047e0 con 0x7f5ef00ccb60 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.174+0000 7f5ed7fff640 1 -- 172.21.15.120:0/1872211229 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4004bd0 con 0x7f5ef00ccb60 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.175+0000 7f5ef60a6640 1 -- 172.21.15.120:0/1872211229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef00cc060 con 0x7f5ef00ccb60 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.175+0000 7f5ef60a6640 1 -- 172.21.15.120:0/1872211229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef00c4770 con 0x7f5ef00ccb60 2024-09-10T10:28:21.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.176+0000 7f5ef60a6640 1 -- 172.21.15.120:0/1872211229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef0162330 con 0x7f5ef00ccb60 2024-09-10T10:28:21.184 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:21.176+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:21.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.178+0000 7f5ed7fff640 1 -- 172.21.15.120:0/1872211229 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ee4011070 con 0x7f5ef00ccb60 2024-09-10T10:28:21.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.178+0000 7f5ed7fff640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5edc06be30 0x7f5edc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.179+0000 7f5eef7fe640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5edc06be30 0x7f5edc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.180+0000 7f5ed7fff640 1 -- 172.21.15.120:0/1872211229 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6801+0+0 (secure 0 0 0) 0x7f5ee408ad20 con 0x7f5ef00ccb60 2024-09-10T10:28:21.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.180+0000 7f5eef7fe640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5edc06be30 0x7f5edc06e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0033000 tx=0x7f5ee0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.186 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:21.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.183+0000 7f5ed7fff640 1 -- 172.21.15.120:0/1872211229 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ee408f070 con 0x7f5ef00ccb60 2024-09-10T10:28:21.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.201+0000 7ff10dffb640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff1280c5380 con 0x7ff128158500 2024-09-10T10:28:21.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.204+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1280c5380 con 0x7ff128158500 2024-09-10T10:28:21.206 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:21.206 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:21.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.204+0000 7ff10dffb640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ff128156ca0 0x7ff128157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.204+0000 7ff10dffb640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ff1281575c0 con 0x7ff128156ca0 2024-09-10T10:28:21.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.204+0000 7ff1277fe640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ff128156ca0 0x7ff128157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.205+0000 7ff1277fe640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ff128156ca0 0x7ff128157080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.206+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1281575c0 con 0x7ff128156ca0 2024-09-10T10:28:21.237 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-10T10:28:21.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.237+0000 7ff10dffb640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff1281626e0 con 0x7ff128156ca0 2024-09-10T10:28:21.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.240+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1281626e0 con 0x7ff128156ca0 2024-09-10T10:28:21.242 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:21.243 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:21.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.240+0000 7ff10dffb640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ff12814e370 0x7ff12814e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.240+0000 7ff10dffb640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ff12814ec90 con 0x7ff12814e370 2024-09-10T10:28:21.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.241+0000 7ff126ffd640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ff12814e370 0x7ff12814e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.241+0000 7ff126ffd640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ff12814e370 0x7ff12814e750 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.242+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff12814ec90 con 0x7ff12814e370 2024-09-10T10:28:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.270+0000 7ff10dffb640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff128062900 con 0x7ff12814e370 2024-09-10T10:28:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.272+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff128062900 con 0x7ff12814e370 2024-09-10T10:28:21.275 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.273+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ff1281f4190 0x7ff128063020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.273+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ff12815a310 con 0x7ff1281f4190 2024-09-10T10:28:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.273+0000 7ff127fff640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ff1281f4190 0x7ff128063020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.274+0000 7ff127fff640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ff1281f4190 0x7ff128063020 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.275+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff12815a310 con 0x7ff1281f4190 2024-09-10T10:28:21.279 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:21.277+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.301+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff12815a310 con 0x7ff1281f4190 2024-09-10T10:28:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.304+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff12815a310 con 0x7ff1281f4190 2024-09-10T10:28:21.306 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.304+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff1281f21e0 0x7ff1281f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.304+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ff1281f2b80 con 0x7ff1281f21e0 2024-09-10T10:28:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.305+0000 7ff1277fe640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff1281f21e0 0x7ff1281f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.305+0000 7ff1277fe640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff1281f21e0 0x7ff1281f2640 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.306+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1281f2b80 con 0x7ff1281f21e0 2024-09-10T10:28:21.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.331+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff1281f2b80 con 0x7ff1281f21e0 2024-09-10T10:28:21.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.333+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff1281f2b80 con 0x7ff1281f21e0 2024-09-10T10:28:21.336 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:21.336 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:21.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.334+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ff12802cb50 0x7ff12802cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.334+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ff12802d600 con 0x7ff12802cb50 2024-09-10T10:28:21.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.334+0000 7ff126ffd640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ff12802cb50 0x7ff12802cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.335+0000 7ff126ffd640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ff12802cb50 0x7ff12802cfb0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.336+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff12802d600 con 0x7ff12802cb50 2024-09-10T10:28:21.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.362+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff12802d600 con 0x7ff12802cb50 2024-09-10T10:28:21.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.364+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff12802d600 con 0x7ff12802cb50 2024-09-10T10:28:21.366 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:21.367 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:21.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.365+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff12802d8c0 0x7ff128217c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.365+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ff12821ab80 con 0x7ff12802d8c0 2024-09-10T10:28:21.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.365+0000 7ff127fff640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff12802d8c0 0x7ff128217c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.365+0000 7ff127fff640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff12802d8c0 0x7ff128217c80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.366+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff12821ab80 con 0x7ff12802d8c0 2024-09-10T10:28:21.372 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:21.372 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:21.372 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:21.372 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:21.372 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:21.381 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:21.380+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:21.385 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:21.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.391+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff128000950 con 0x7ff12802d8c0 2024-09-10T10:28:21.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.392+0000 7f5c77f4b640 1 Processor -- start 2024-09-10T10:28:21.395 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:21.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.393+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff128000950 con 0x7ff12802d8c0 2024-09-10T10:28:21.396 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:21.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.393+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1282186d0 0x7ff128218b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.393+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7ff12821dfa0 con 0x7ff1282186d0 2024-09-10T10:28:21.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.394+0000 7ff1277fe640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1282186d0 0x7ff128218b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.394+0000 7f5c77f4b640 1 -- start start 2024-09-10T10:28:21.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.394+0000 7ff1277fe640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1282186d0 0x7ff128218b30 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.395+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff12821dfa0 con 0x7ff1282186d0 2024-09-10T10:28:21.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.395+0000 7f5c77f4b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5c70162430 0x7f5c7006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.395+0000 7f5c77f4b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5c700ccc30 0x7f5c7006c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.395+0000 7f5c77f4b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c7015fc40 con 0x7f5c70162ed0 2024-09-10T10:28:21.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.395+0000 7f5c77f4b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f5c7006c8c0 con 0x7f5c700ccc30 2024-09-10T10:28:21.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.395+0000 7f5c77f4b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5c7006e660 con 0x7f5c70162430 2024-09-10T10:28:21.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.395+0000 7f5c75ce7640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f5c70162ed0 0x7f5c700cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:47426/0 (socket says 172.21.15.120:47426) 2024-09-10T10:28:21.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.395+0000 7f5c75ce7640 1 -- 172.21.15.120:0/2752842780 learned_addr learned my addr 172.21.15.120:0/2752842780 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:21.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.396+0000 7f5c754e6640 1 --2- 172.21.15.120:0/2752842780 >> v2:172.21.15.134:3300/0 conn(0x7f5c70162430 0x7f5c7006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.396+0000 7f5c764e8640 1 --2- 172.21.15.120:0/2752842780 >> v2:172.21.15.120:3300/0 conn(0x7f5c700ccc30 0x7f5c7006c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.396+0000 7f5c74ce5640 1 -- 172.21.15.120:0/2752842780 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930791233 0 0) 0x7f5c7015fc40 con 0x7f5c70162ed0 2024-09-10T10:28:21.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.396+0000 7f5c74ce5640 1 -- 172.21.15.120:0/2752842780 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c5c003520 con 0x7f5c70162ed0 2024-09-10T10:28:21.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.396+0000 7f5c754e6640 1 -- 172.21.15.120:0/2752842780 >> v2:172.21.15.120:3300/0 conn(0x7f5c700ccc30 msgr2=0x7f5c7006c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:21.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.396+0000 7f5c754e6640 1 --2- 172.21.15.120:0/2752842780 >> v2:172.21.15.120:3300/0 conn(0x7f5c700ccc30 0x7f5c7006c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.396+0000 7f5c754e6640 1 -- 172.21.15.120:0/2752842780 >> v1:172.21.15.120:6789/0 conn(0x7f5c70162ed0 legacy=0x7f5c700cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.396+0000 7f5c754e6640 1 -- 172.21.15.120:0/2752842780 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7006e390 con 0x7f5c70162430 2024-09-10T10:28:21.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.397+0000 7f5c754e6640 1 --2- 172.21.15.120:0/2752842780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c70162430 0x7f5c7006bc10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c700b2790 tx=0x7f5c6002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26dcd0b33aa097e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.398+0000 7f5c74ce5640 1 -- 172.21.15.120:0/2752842780 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60037070 con 0x7f5c70162430 2024-09-10T10:28:21.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.398+0000 7f5c74ce5640 1 -- 172.21.15.120:0/2752842780 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c60032440 con 0x7f5c70162430 2024-09-10T10:28:21.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.398+0000 7f5c74ce5640 1 -- 172.21.15.120:0/2752842780 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60037070 con 0x7f5c70162430 2024-09-10T10:28:21.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.399+0000 7f5c77f4b640 1 -- 172.21.15.120:0/2752842780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c70162430 msgr2=0x7f5c7006bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.399+0000 7f5c77f4b640 1 --2- 172.21.15.120:0/2752842780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c70162430 0x7f5c7006bc10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c700b2790 tx=0x7f5c6002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.399+0000 7f5c77f4b640 1 -- 172.21.15.120:0/2752842780 shutdown_connections 2024-09-10T10:28:21.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.399+0000 7f5c77f4b640 1 --2- 172.21.15.120:0/2752842780 >> v2:172.21.15.120:3300/0 conn(0x7f5c700ccc30 0x7f5c7006c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.399+0000 7f5c77f4b640 1 --2- 172.21.15.120:0/2752842780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c70162430 0x7f5c7006bc10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.399+0000 7f5c77f4b640 1 -- 172.21.15.120:0/2752842780 >> 172.21.15.120:0/2752842780 conn(0x7f5c700703e0 msgr2=0x7f5c700707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:21.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.400+0000 7f5c77f4b640 1 -- 172.21.15.120:0/2752842780 shutdown_connections 2024-09-10T10:28:21.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.400+0000 7f5c77f4b640 1 -- 172.21.15.120:0/2752842780 wait complete. 2024-09-10T10:28:21.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.402+0000 7f5c77f4b640 1 Processor -- start 2024-09-10T10:28:21.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.402+0000 7f5c77f4b640 1 -- start start 2024-09-10T10:28:21.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.403+0000 7f5c77f4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c700ccc30 0x7f5c70154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.404+0000 7f5c77f4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c70162430 0x7f5c700c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.404+0000 7f5c75ce7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c700ccc30 0x7f5c70154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.404+0000 7f5c75ce7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c700ccc30 0x7f5c70154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:50222/0 (socket says 172.21.15.120:50222) 2024-09-10T10:28:21.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.404+0000 7f5c75ce7640 1 -- 172.21.15.120:0/859189187 learned_addr learned my addr 172.21.15.120:0/859189187 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:21.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.404+0000 7f5c754e6640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c70162430 0x7f5c700c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.404+0000 7f5c77f4b640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5c70162ed0 0x7f5c7014f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.404+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7006e0b0 con 0x7f5c70162430 2024-09-10T10:28:21.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.404+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7006ddd0 con 0x7f5c700ccc30 2024-09-10T10:28:21.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.404+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5c701632b0 con 0x7f5c70162ed0 2024-09-10T10:28:21.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.405+0000 7f5c764e8640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5c70162ed0 0x7f5c7014f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.405+0000 7f5c75ce7640 1 -- 172.21.15.120:0/859189187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5c70162ed0 msgr2=0x7f5c7014f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.405+0000 7f5c75ce7640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5c70162ed0 0x7f5c7014f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.405+0000 7f5c75ce7640 1 -- 172.21.15.120:0/859189187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c70162430 msgr2=0x7f5c700c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.405+0000 7f5c75ce7640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c70162430 0x7f5c700c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.405+0000 7f5c75ce7640 1 -- 172.21.15.120:0/859189187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c70152fe0 con 0x7f5c700ccc30 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.405+0000 7f5c75ce7640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c700ccc30 0x7f5c70154940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c430 tx=0x7f5c6c00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.406+0000 7f5c667fc640 1 -- 172.21.15.120:0/859189187 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c017070 con 0x7f5c700ccc30 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.406+0000 7f5c667fc640 1 -- 172.21.15.120:0/859189187 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c005ce0 con 0x7f5c700ccc30 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.406+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c701f8d10 con 0x7f5c700ccc30 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.406+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c700c95a0 con 0x7f5c700ccc30 2024-09-10T10:28:21.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.407+0000 7f5c667fc640 1 -- 172.21.15.120:0/859189187 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0072d0 con 0x7f5c700ccc30 2024-09-10T10:28:21.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.407+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c70161d10 con 0x7f5c700ccc30 2024-09-10T10:28:21.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.409+0000 7f5c667fc640 1 -- 172.21.15.120:0/859189187 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c6c01a070 con 0x7f5c700ccc30 2024-09-10T10:28:21.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.409+0000 7f5c667fc640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5c5406be80 0x7f5c5406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.410+0000 7f5c667fc640 1 -- 172.21.15.120:0/859189187 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6801+0+0 (secure 0 0 0) 0x7f5c6c08aa00 con 0x7f5c700ccc30 2024-09-10T10:28:21.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.410+0000 7f5c754e6640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5c5406be80 0x7f5c5406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.411+0000 7f5c754e6640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5c5406be80 0x7f5c5406e340 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c60034000 tx=0x7f5c6002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.412+0000 7f5c667fc640 1 -- 172.21.15.120:0/859189187 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5c70161d10 con 0x7f5c700ccc30 2024-09-10T10:28:21.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.421+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff128219180 con 0x7ff1282186d0 2024-09-10T10:28:21.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.423+0000 7ff10ffff640 1 -- 172.21.15.120:0/3424263445 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff128219180 con 0x7ff1282186d0 2024-09-10T10:28:21.426 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ff128158500 msgr2=0x7ff128156760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7ff128158500 0x7ff128156760 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ff128156ca0 msgr2=0x7ff128157080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7ff128156ca0 0x7ff128157080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ff12814e370 msgr2=0x7ff12814e750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7ff12814e370 0x7ff12814e750 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ff1281f4190 msgr2=0x7ff128063020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7ff1281f4190 0x7ff128063020 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff1281f21e0 msgr2=0x7ff1281f2640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7ff1281f21e0 0x7ff1281f2640 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ff12802cb50 msgr2=0x7ff12802cfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7ff12802cb50 0x7ff12802cfb0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff12802d8c0 msgr2=0x7ff128217c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff12802d8c0 0x7ff128217c80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.424+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1282186d0 msgr2=0x7ff128218b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7ff1282186d0 0x7ff128218b30 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff127fff640 1 -- 172.21.15.120:0/3424263445 reap_dead start 2024-09-10T10:28:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff127fff640 1 -- 172.21.15.120:0/3424263445 reap_dead start 2024-09-10T10:28:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff127fff640 1 -- 172.21.15.120:0/3424263445 reap_dead start 2024-09-10T10:28:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff127fff640 1 -- 172.21.15.120:0/3424263445 reap_dead start 2024-09-10T10:28:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff127fff640 1 -- 172.21.15.120:0/3424263445 reap_dead start 2024-09-10T10:28:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff11406bf00 msgr2=0x7ff11406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff11406bf00 0x7ff11406e3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff128152c60 tx=0x7ff11c009040 comp rx=0 tx=0).stop 2024-09-10T10:28:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1280c9a20 msgr2=0x7ff1281f30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1280c9a20 0x7ff1281f30c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff11800c800 tx=0x7ff11800ccc0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 shutdown_connections 2024-09-10T10:28:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff11406bf00 0x7ff11406e3c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 --2- 172.21.15.120:0/3424263445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1280c9a20 0x7ff1281f30c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 >> 172.21.15.120:0/3424263445 conn(0x7ff12806bc10 msgr2=0x7ff12815d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 shutdown_connections 2024-09-10T10:28:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.425+0000 7ff12e0e6640 1 -- 172.21.15.120:0/3424263445 wait complete. 2024-09-10T10:28:21.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.474+0000 7f5ef60a6640 1 -- 172.21.15.120:0/1872211229 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ef006cc20 con 0x7f5edc06be30 2024-09-10T10:28:21.481 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:21.479+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:21.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.482+0000 7f5ed7fff640 1 -- 172.21.15.120:0/1872211229 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332754 (secure 0 0 0) 0x7f5ef006cc20 con 0x7f5edc06be30 2024-09-10T10:28:21.487 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:28:21.489 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 -- 172.21.15.120:0/1872211229 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5edc06be30 msgr2=0x7f5edc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5edc06be30 0x7f5edc06e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0033000 tx=0x7f5ee0008030 comp rx=0 tx=0).stop 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 -- 172.21.15.120:0/1872211229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef00ccb60 msgr2=0x7f5ef0070120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef00ccb60 0x7f5ef0070120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400c410 tx=0x7f5ee400c8d0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 -- 172.21.15.120:0/1872211229 shutdown_connections 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5edc06be30 0x7f5edc06e2f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ef00ccf60 0x7f5ef01610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef00ccb60 0x7f5ef0070120 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 --2- 172.21.15.120:0/1872211229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ef00cc780 0x7f5ef0152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 -- 172.21.15.120:0/1872211229 >> 172.21.15.120:0/1872211229 conn(0x7f5ef006bc10 msgr2=0x7f5ef0157180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 -- 172.21.15.120:0/1872211229 shutdown_connections 2024-09-10T10:28:21.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.486+0000 7f5ef60a6640 1 -- 172.21.15.120:0/1872211229 wait complete. 2024-09-10T10:28:21.501 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-10T10:28:20.110469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604953,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":383,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":588,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220808,"kb_used_data":4584,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753912,"statfs":{"total":773094113280,"available":772868005888,"internally_reserved":0,"allocated":4694016,"data_stored":3278458,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1730,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.913206"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783629+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:28:16.783629+0000","last_peered":"2024-09-10T10:28:16.783629+0000","last_clean":"2024-09-10T10:28:16.783629+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:28:16.783629+0000","last_undegraded":"2024-09-10T10:28:16.783629+0000","last_fullsized":"2024-09-10T10:28:16.783629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.211823+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:28:16.211823+0000","last_peered":"2024-09-10T10:28:16.211823+0000","last_clean":"2024-09-10T10:28:16.211823+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:28:16.211823+0000","last_undegraded":"2024-09-10T10:28:16.211823+0000","last_fullsized":"2024-09-10T10:28:16.211823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.210503+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:28:16.210503+0000","last_peered":"2024-09-10T10:28:16.210503+0000","last_clean":"2024-09-10T10:28:16.210503+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:28:16.210503+0000","last_undegraded":"2024-09-10T10:28:16.210503+0000","last_fullsized":"2024-09-10T10:28:16.210503+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":68,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.211940+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:28:16.211940+0000","last_peered":"2024-09-10T10:28:16.211940+0000","last_clean":"2024-09-10T10:28:16.211940+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:28:16.211940+0000","last_undegraded":"2024-09-10T10:28:16.211940+0000","last_fullsized":"2024-09-10T10:28:16.211940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783541+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:28:16.783541+0000","last_peered":"2024-09-10T10:28:16.783541+0000","last_clean":"2024-09-10T10:28:16.783541+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:28:16.783541+0000","last_undegraded":"2024-09-10T10:28:16.783541+0000","last_fullsized":"2024-09-10T10:28:16.783541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.790619+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:28:16.790619+0000","last_peered":"2024-09-10T10:28:16.790619+0000","last_clean":"2024-09-10T10:28:16.790619+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:28:16.790619+0000","last_undegraded":"2024-09-10T10:28:16.790619+0000","last_fullsized":"2024-09-10T10:28:16.790619+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"28'12","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206711+0000","last_change":"2024-09-10T10:27:51.060242+0000","last_active":"2024-09-10T10:28:16.206711+0000","last_peered":"2024-09-10T10:28:16.206711+0000","last_clean":"2024-09-10T10:28:16.206711+0000","last_became_active":"2024-09-10T10:27:51.049148+0000","last_became_peered":"2024-09-10T10:27:51.049148+0000","last_unstale":"2024-09-10T10:28:16.206711+0000","last_undegraded":"2024-09-10T10:28:16.206711+0000","last_fullsized":"2024-09-10T10:28:16.206711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.604144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206374+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:28:16.206374+0000","last_peered":"2024-09-10T10:28:16.206374+0000","last_clean":"2024-09-10T10:28:16.206374+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:28:16.206374+0000","last_undegraded":"2024-09-10T10:28:16.206374+0000","last_fullsized":"2024-09-10T10:28:16.206374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.790752+0000","last_change":"2024-09-10T10:27:23.955254+0000","last_active":"2024-09-10T10:28:16.790752+0000","last_peered":"2024-09-10T10:28:16.790752+0000","last_clean":"2024-09-10T10:28:16.790752+0000","last_became_active":"2024-09-10T10:27:23.954737+0000","last_became_peered":"2024-09-10T10:27:23.954737+0000","last_unstale":"2024-09-10T10:28:16.790752+0000","last_undegraded":"2024-09-10T10:28:16.790752+0000","last_fullsized":"2024-09-10T10:28:16.790752+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206424+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:28:16.206424+0000","last_peered":"2024-09-10T10:28:16.206424+0000","last_clean":"2024-09-10T10:28:16.206424+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:28:16.206424+0000","last_undegraded":"2024-09-10T10:28:16.206424+0000","last_fullsized":"2024-09-10T10:28:16.206424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":48,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.272687+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:28:16.272687+0000","last_peered":"2024-09-10T10:28:16.272687+0000","last_clean":"2024-09-10T10:28:16.272687+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:28:16.272687+0000","last_undegraded":"2024-09-10T10:28:16.272687+0000","last_fullsized":"2024-09-10T10:28:16.272687+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372557+0000","last_change":"2024-09-10T10:28:11.179443+0000","last_active":"2024-09-10T10:28:17.372557+0000","last_peered":"2024-09-10T10:28:17.372557+0000","last_clean":"2024-09-10T10:28:17.372557+0000","last_became_active":"2024-09-10T10:28:11.179067+0000","last_became_peered":"2024-09-10T10:28:11.179067+0000","last_unstale":"2024-09-10T10:28:17.372557+0000","last_undegraded":"2024-09-10T10:28:17.372557+0000","last_fullsized":"2024-09-10T10:28:17.372557+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:43.114500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0,5],"acting":[6,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372674+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:28:17.372674+0000","last_peered":"2024-09-10T10:28:17.372674+0000","last_clean":"2024-09-10T10:28:17.372674+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:28:17.372674+0000","last_undegraded":"2024-09-10T10:28:17.372674+0000","last_fullsized":"2024-09-10T10:28:17.372674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'4","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206197+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:28:16.206197+0000","last_peered":"2024-09-10T10:28:16.206197+0000","last_clean":"2024-09-10T10:28:16.206197+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:28:16.206197+0000","last_undegraded":"2024-09-10T10:28:16.206197+0000","last_fullsized":"2024-09-10T10:28:16.206197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.791102+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:28:16.791102+0000","last_peered":"2024-09-10T10:28:16.791102+0000","last_clean":"2024-09-10T10:28:16.791102+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:28:16.791102+0000","last_undegraded":"2024-09-10T10:28:16.791102+0000","last_fullsized":"2024-09-10T10:28:16.791102+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.211183+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:28:16.211183+0000","last_peered":"2024-09-10T10:28:16.211183+0000","last_clean":"2024-09-10T10:28:16.211183+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:28:16.211183+0000","last_undegraded":"2024-09-10T10:28:16.211183+0000","last_fullsized":"2024-09-10T10:28:16.211183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'4","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.373740+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:28:17.373740+0000","last_peered":"2024-09-10T10:28:17.373740+0000","last_clean":"2024-09-10T10:28:17.373740+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:28:17.373740+0000","last_undegraded":"2024-09-10T10:28:17.373740+0000","last_fullsized":"2024-09-10T10:28:17.373740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.790949+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:28:16.790949+0000","last_peered":"2024-09-10T10:28:16.790949+0000","last_clean":"2024-09-10T10:28:16.790949+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:28:16.790949+0000","last_undegraded":"2024-09-10T10:28:16.790949+0000","last_fullsized":"2024-09-10T10:28:16.790949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.210706+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:28:16.210706+0000","last_peered":"2024-09-10T10:28:16.210706+0000","last_clean":"2024-09-10T10:28:16.210706+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:28:16.210706+0000","last_undegraded":"2024-09-10T10:28:16.210706+0000","last_fullsized":"2024-09-10T10:28:16.210706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.373164+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:28:17.373164+0000","last_peered":"2024-09-10T10:28:17.373164+0000","last_clean":"2024-09-10T10:28:17.373164+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:28:17.373164+0000","last_undegraded":"2024-09-10T10:28:17.373164+0000","last_fullsized":"2024-09-10T10:28:17.373164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.374612+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:28:17.374612+0000","last_peered":"2024-09-10T10:28:17.374612+0000","last_clean":"2024-09-10T10:28:17.374612+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:28:17.374612+0000","last_undegraded":"2024-09-10T10:28:17.374612+0000","last_fullsized":"2024-09-10T10:28:17.374612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.790761+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:28:16.790761+0000","last_peered":"2024-09-10T10:28:16.790761+0000","last_clean":"2024-09-10T10:28:16.790761+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:28:16.790761+0000","last_undegraded":"2024-09-10T10:28:16.790761+0000","last_fullsized":"2024-09-10T10:28:16.790761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":67,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205360+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:28:16.205360+0000","last_peered":"2024-09-10T10:28:16.205360+0000","last_clean":"2024-09-10T10:28:16.205360+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:28:16.205360+0000","last_undegraded":"2024-09-10T10:28:16.205360+0000","last_fullsized":"2024-09-10T10:28:16.205360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.791129+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:28:16.791129+0000","last_peered":"2024-09-10T10:28:16.791129+0000","last_clean":"2024-09-10T10:28:16.791129+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:28:16.791129+0000","last_undegraded":"2024-09-10T10:28:16.791129+0000","last_fullsized":"2024-09-10T10:28:16.791129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783445+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:28:16.783445+0000","last_peered":"2024-09-10T10:28:16.783445+0000","last_clean":"2024-09-10T10:28:16.783445+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:28:16.783445+0000","last_undegraded":"2024-09-10T10:28:16.783445+0000","last_fullsized":"2024-09-10T10:28:16.783445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'7","reported_seq":73,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372286+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:28:17.372286+0000","last_peered":"2024-09-10T10:28:17.372286+0000","last_clean":"2024-09-10T10:28:17.372286+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:28:17.372286+0000","last_undegraded":"2024-09-10T10:28:17.372286+0000","last_fullsized":"2024-09-10T10:28:17.372286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.790565+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:28:16.790565+0000","last_peered":"2024-09-10T10:28:16.790565+0000","last_clean":"2024-09-10T10:28:16.790565+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:28:16.790565+0000","last_undegraded":"2024-09-10T10:28:16.790565+0000","last_fullsized":"2024-09-10T10:28:16.790565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.212152+0000","last_change":"2024-09-10T10:26:41.795085+0000","last_active":"2024-09-10T10:28:16.212152+0000","last_peered":"2024-09-10T10:28:16.212152+0000","last_clean":"2024-09-10T10:28:16.212152+0000","last_became_active":"2024-09-10T10:26:41.794819+0000","last_became_peered":"2024-09-10T10:26:41.794819+0000","last_unstale":"2024-09-10T10:28:16.212152+0000","last_undegraded":"2024-09-10T10:28:16.212152+0000","last_fullsized":"2024-09-10T10:28:16.212152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"47'10","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.374707+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:28:17.374707+0000","last_peered":"2024-09-10T10:28:17.374707+0000","last_clean":"2024-09-10T10:28:17.374707+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:28:17.374707+0000","last_undegraded":"2024-09-10T10:28:17.374707+0000","last_fullsized":"2024-09-10T10:28:17.374707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.210438+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:28:16.210438+0000","last_peered":"2024-09-10T10:28:16.210438+0000","last_clean":"2024-09-10T10:28:16.210438+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:28:16.210438+0000","last_undegraded":"2024-09-10T10:28:16.210438+0000","last_fullsized":"2024-09-10T10:28:16.210438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.373330+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:28:17.373330+0000","last_peered":"2024-09-10T10:28:17.373330+0000","last_clean":"2024-09-10T10:28:17.373330+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:28:17.373330+0000","last_undegraded":"2024-09-10T10:28:17.373330+0000","last_fullsized":"2024-09-10T10:28:17.373330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206801+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:28:16.206801+0000","last_peered":"2024-09-10T10:28:16.206801+0000","last_clean":"2024-09-10T10:28:16.206801+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:28:16.206801+0000","last_undegraded":"2024-09-10T10:28:16.206801+0000","last_fullsized":"2024-09-10T10:28:16.206801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.782870+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:28:16.782870+0000","last_peered":"2024-09-10T10:28:16.782870+0000","last_clean":"2024-09-10T10:28:16.782870+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:28:16.782870+0000","last_undegraded":"2024-09-10T10:28:16.782870+0000","last_fullsized":"2024-09-10T10:28:16.782870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206334+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:28:16.206334+0000","last_peered":"2024-09-10T10:28:16.206334+0000","last_clean":"2024-09-10T10:28:16.206334+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:28:16.206334+0000","last_undegraded":"2024-09-10T10:28:16.206334+0000","last_fullsized":"2024-09-10T10:28:16.206334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372583+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:28:17.372583+0000","last_peered":"2024-09-10T10:28:17.372583+0000","last_clean":"2024-09-10T10:28:17.372583+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:28:17.372583+0000","last_undegraded":"2024-09-10T10:28:17.372583+0000","last_fullsized":"2024-09-10T10:28:17.372583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.374021+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:28:17.374021+0000","last_peered":"2024-09-10T10:28:17.374021+0000","last_clean":"2024-09-10T10:28:17.374021+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:28:17.374021+0000","last_undegraded":"2024-09-10T10:28:17.374021+0000","last_fullsized":"2024-09-10T10:28:17.374021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.790971+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:28:16.790971+0000","last_peered":"2024-09-10T10:28:16.790971+0000","last_clean":"2024-09-10T10:28:16.790971+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:28:16.790971+0000","last_undegraded":"2024-09-10T10:28:16.790971+0000","last_fullsized":"2024-09-10T10:28:16.790971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206057+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:28:16.206057+0000","last_peered":"2024-09-10T10:28:16.206057+0000","last_clean":"2024-09-10T10:28:16.206057+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:28:16.206057+0000","last_undegraded":"2024-09-10T10:28:16.206057+0000","last_fullsized":"2024-09-10T10:28:16.206057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.373240+0000","last_change":"2024-09-10T10:26:45.814008+0000","last_active":"2024-09-10T10:28:17.373240+0000","last_peered":"2024-09-10T10:28:17.373240+0000","last_clean":"2024-09-10T10:28:17.373240+0000","last_became_active":"2024-09-10T10:26:45.813627+0000","last_became_peered":"2024-09-10T10:26:45.813627+0000","last_unstale":"2024-09-10T10:28:17.373240+0000","last_undegraded":"2024-09-10T10:28:17.373240+0000","last_fullsized":"2024-09-10T10:28:17.373240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.373058+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:28:17.373058+0000","last_peered":"2024-09-10T10:28:17.373058+0000","last_clean":"2024-09-10T10:28:17.373058+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:28:17.373058+0000","last_undegraded":"2024-09-10T10:28:17.373058+0000","last_fullsized":"2024-09-10T10:28:17.373058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"28'5","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.271955+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:28:16.271955+0000","last_peered":"2024-09-10T10:28:16.271955+0000","last_clean":"2024-09-10T10:28:16.271955+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:28:16.271955+0000","last_undegraded":"2024-09-10T10:28:16.271955+0000","last_fullsized":"2024-09-10T10:28:16.271955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.374712+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:28:17.374712+0000","last_peered":"2024-09-10T10:28:17.374712+0000","last_clean":"2024-09-10T10:28:17.374712+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:28:17.374712+0000","last_undegraded":"2024-09-10T10:28:17.374712+0000","last_fullsized":"2024-09-10T10:28:17.374712+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":68,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.782593+0000","last_change":"2024-09-10T10:28:11.179252+0000","last_active":"2024-09-10T10:28:16.782593+0000","last_peered":"2024-09-10T10:28:16.782593+0000","last_clean":"2024-09-10T10:28:16.782593+0000","last_became_active":"2024-09-10T10:28:11.178949+0000","last_became_peered":"2024-09-10T10:28:11.178949+0000","last_unstale":"2024-09-10T10:28:16.782593+0000","last_undegraded":"2024-09-10T10:28:16.782593+0000","last_fullsized":"2024-09-10T10:28:16.782593+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:12.656803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205465+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:28:16.205465+0000","last_peered":"2024-09-10T10:28:16.205465+0000","last_clean":"2024-09-10T10:28:16.205465+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:28:16.205465+0000","last_undegraded":"2024-09-10T10:28:16.205465+0000","last_fullsized":"2024-09-10T10:28:16.205465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783254+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:28:16.783254+0000","last_peered":"2024-09-10T10:28:16.783254+0000","last_clean":"2024-09-10T10:28:16.783254+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:28:16.783254+0000","last_undegraded":"2024-09-10T10:28:16.783254+0000","last_fullsized":"2024-09-10T10:28:16.783254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.211038+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:28:16.211038+0000","last_peered":"2024-09-10T10:28:16.211038+0000","last_clean":"2024-09-10T10:28:16.211038+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:28:16.211038+0000","last_undegraded":"2024-09-10T10:28:16.211038+0000","last_fullsized":"2024-09-10T10:28:16.211038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206216+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:28:16.206216+0000","last_peered":"2024-09-10T10:28:16.206216+0000","last_clean":"2024-09-10T10:28:16.206216+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:28:16.206216+0000","last_undegraded":"2024-09-10T10:28:16.206216+0000","last_fullsized":"2024-09-10T10:28:16.206216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":221,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205561+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:28:16.205561+0000","last_peered":"2024-09-10T10:28:16.205561+0000","last_clean":"2024-09-10T10:28:16.205561+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:28:16.205561+0000","last_undegraded":"2024-09-10T10:28:16.205561+0000","last_fullsized":"2024-09-10T10:28:16.205561+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372766+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:28:17.372766+0000","last_peered":"2024-09-10T10:28:17.372766+0000","last_clean":"2024-09-10T10:28:17.372766+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:28:17.372766+0000","last_undegraded":"2024-09-10T10:28:17.372766+0000","last_fullsized":"2024-09-10T10:28:17.372766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206626+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:28:16.206626+0000","last_peered":"2024-09-10T10:28:16.206626+0000","last_clean":"2024-09-10T10:28:16.206626+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:28:16.206626+0000","last_undegraded":"2024-09-10T10:28:16.206626+0000","last_fullsized":"2024-09-10T10:28:16.206626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.272577+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:28:16.272577+0000","last_peered":"2024-09-10T10:28:16.272577+0000","last_clean":"2024-09-10T10:28:16.272577+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:28:16.272577+0000","last_undegraded":"2024-09-10T10:28:16.272577+0000","last_fullsized":"2024-09-10T10:28:16.272577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372563+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:28:17.372563+0000","last_peered":"2024-09-10T10:28:17.372563+0000","last_clean":"2024-09-10T10:28:17.372563+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:28:17.372563+0000","last_undegraded":"2024-09-10T10:28:17.372563+0000","last_fullsized":"2024-09-10T10:28:17.372563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206433+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:28:16.206433+0000","last_peered":"2024-09-10T10:28:16.206433+0000","last_clean":"2024-09-10T10:28:16.206433+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:28:16.206433+0000","last_undegraded":"2024-09-10T10:28:16.206433+0000","last_fullsized":"2024-09-10T10:28:16.206433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":68,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372787+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:28:17.372787+0000","last_peered":"2024-09-10T10:28:17.372787+0000","last_clean":"2024-09-10T10:28:17.372787+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:28:17.372787+0000","last_undegraded":"2024-09-10T10:28:17.372787+0000","last_fullsized":"2024-09-10T10:28:17.372787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783005+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:28:16.783005+0000","last_peered":"2024-09-10T10:28:16.783005+0000","last_clean":"2024-09-10T10:28:16.783005+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:28:16.783005+0000","last_undegraded":"2024-09-10T10:28:16.783005+0000","last_fullsized":"2024-09-10T10:28:16.783005+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.207026+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:28:16.207026+0000","last_peered":"2024-09-10T10:28:16.207026+0000","last_clean":"2024-09-10T10:28:16.207026+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:28:16.207026+0000","last_undegraded":"2024-09-10T10:28:16.207026+0000","last_fullsized":"2024-09-10T10:28:16.207026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.273309+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:28:16.273309+0000","last_peered":"2024-09-10T10:28:16.273309+0000","last_clean":"2024-09-10T10:28:16.273309+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:28:16.273309+0000","last_undegraded":"2024-09-10T10:28:16.273309+0000","last_fullsized":"2024-09-10T10:28:16.273309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205867+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:28:16.205867+0000","last_peered":"2024-09-10T10:28:16.205867+0000","last_clean":"2024-09-10T10:28:16.205867+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:28:16.205867+0000","last_undegraded":"2024-09-10T10:28:16.205867+0000","last_fullsized":"2024-09-10T10:28:16.205867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206897+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:28:16.206897+0000","last_peered":"2024-09-10T10:28:16.206897+0000","last_clean":"2024-09-10T10:28:16.206897+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:28:16.206897+0000","last_undegraded":"2024-09-10T10:28:16.206897+0000","last_fullsized":"2024-09-10T10:28:16.206897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":68,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.791237+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:28:16.791237+0000","last_peered":"2024-09-10T10:28:16.791237+0000","last_clean":"2024-09-10T10:28:16.791237+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:28:16.791237+0000","last_undegraded":"2024-09-10T10:28:16.791237+0000","last_fullsized":"2024-09-10T10:28:16.791237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.790535+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:28:16.790535+0000","last_peered":"2024-09-10T10:28:16.790535+0000","last_clean":"2024-09-10T10:28:16.790535+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:28:16.790535+0000","last_undegraded":"2024-09-10T10:28:16.790535+0000","last_fullsized":"2024-09-10T10:28:16.790535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206585+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:28:16.206585+0000","last_peered":"2024-09-10T10:28:16.206585+0000","last_clean":"2024-09-10T10:28:16.206585+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:28:16.206585+0000","last_undegraded":"2024-09-10T10:28:16.206585+0000","last_fullsized":"2024-09-10T10:28:16.206585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"28'17","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.271707+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:28:16.271707+0000","last_peered":"2024-09-10T10:28:16.271707+0000","last_clean":"2024-09-10T10:28:16.271707+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:28:16.271707+0000","last_undegraded":"2024-09-10T10:28:16.271707+0000","last_fullsized":"2024-09-10T10:28:16.271707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783166+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:28:16.783166+0000","last_peered":"2024-09-10T10:28:16.783166+0000","last_clean":"2024-09-10T10:28:16.783166+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:28:16.783166+0000","last_undegraded":"2024-09-10T10:28:16.783166+0000","last_fullsized":"2024-09-10T10:28:16.783166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372529+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:28:17.372529+0000","last_peered":"2024-09-10T10:28:17.372529+0000","last_clean":"2024-09-10T10:28:17.372529+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:28:17.372529+0000","last_undegraded":"2024-09-10T10:28:17.372529+0000","last_fullsized":"2024-09-10T10:28:17.372529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372617+0000","last_change":"2024-09-10T10:28:11.204059+0000","last_active":"2024-09-10T10:28:17.372617+0000","last_peered":"2024-09-10T10:28:17.372617+0000","last_clean":"2024-09-10T10:28:17.372617+0000","last_became_active":"2024-09-10T10:28:11.198670+0000","last_became_peered":"2024-09-10T10:28:11.198670+0000","last_unstale":"2024-09-10T10:28:17.372617+0000","last_undegraded":"2024-09-10T10:28:17.372617+0000","last_fullsized":"2024-09-10T10:28:17.372617+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:12.071607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.212064+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:28:16.212064+0000","last_peered":"2024-09-10T10:28:16.212064+0000","last_clean":"2024-09-10T10:28:16.212064+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:28:16.212064+0000","last_undegraded":"2024-09-10T10:28:16.212064+0000","last_fullsized":"2024-09-10T10:28:16.212064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206097+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:28:16.206097+0000","last_peered":"2024-09-10T10:28:16.206097+0000","last_clean":"2024-09-10T10:28:16.206097+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:28:16.206097+0000","last_undegraded":"2024-09-10T10:28:16.206097+0000","last_fullsized":"2024-09-10T10:28:16.206097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"28'5","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206031+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:28:16.206031+0000","last_peered":"2024-09-10T10:28:16.206031+0000","last_clean":"2024-09-10T10:28:16.206031+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:28:16.206031+0000","last_undegraded":"2024-09-10T10:28:16.206031+0000","last_fullsized":"2024-09-10T10:28:16.206031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.211468+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:28:16.211468+0000","last_peered":"2024-09-10T10:28:16.211468+0000","last_clean":"2024-09-10T10:28:16.211468+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:28:16.211468+0000","last_undegraded":"2024-09-10T10:28:16.211468+0000","last_fullsized":"2024-09-10T10:28:16.211468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.782816+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:28:16.782816+0000","last_peered":"2024-09-10T10:28:16.782816+0000","last_clean":"2024-09-10T10:28:16.782816+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:28:16.782816+0000","last_undegraded":"2024-09-10T10:28:16.782816+0000","last_fullsized":"2024-09-10T10:28:16.782816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783309+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:28:16.783309+0000","last_peered":"2024-09-10T10:28:16.783309+0000","last_clean":"2024-09-10T10:28:16.783309+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:28:16.783309+0000","last_undegraded":"2024-09-10T10:28:16.783309+0000","last_fullsized":"2024-09-10T10:28:16.783309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206240+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:28:16.206240+0000","last_peered":"2024-09-10T10:28:16.206240+0000","last_clean":"2024-09-10T10:28:16.206240+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:28:16.206240+0000","last_undegraded":"2024-09-10T10:28:16.206240+0000","last_fullsized":"2024-09-10T10:28:16.206240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206583+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:28:16.206583+0000","last_peered":"2024-09-10T10:28:16.206583+0000","last_clean":"2024-09-10T10:28:16.206583+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:28:16.206583+0000","last_undegraded":"2024-09-10T10:28:16.206583+0000","last_fullsized":"2024-09-10T10:28:16.206583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"45'18","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372246+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:28:17.372246+0000","last_peered":"2024-09-10T10:28:17.372246+0000","last_clean":"2024-09-10T10:28:17.372246+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:28:17.372246+0000","last_undegraded":"2024-09-10T10:28:17.372246+0000","last_fullsized":"2024-09-10T10:28:17.372246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7907,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.374158+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:28:17.374158+0000","last_peered":"2024-09-10T10:28:17.374158+0000","last_clean":"2024-09-10T10:28:17.374158+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:28:17.374158+0000","last_undegraded":"2024-09-10T10:28:17.374158+0000","last_fullsized":"2024-09-10T10:28:17.374158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.791074+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:28:16.791074+0000","last_peered":"2024-09-10T10:28:16.791074+0000","last_clean":"2024-09-10T10:28:16.791074+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:28:16.791074+0000","last_undegraded":"2024-09-10T10:28:16.791074+0000","last_fullsized":"2024-09-10T10:28:16.791074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'5","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372907+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:28:17.372907+0000","last_peered":"2024-09-10T10:28:17.372907+0000","last_clean":"2024-09-10T10:28:17.372907+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:28:17.372907+0000","last_undegraded":"2024-09-10T10:28:17.372907+0000","last_fullsized":"2024-09-10T10:28:17.372907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.790890+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:28:16.790890+0000","last_peered":"2024-09-10T10:28:16.790890+0000","last_clean":"2024-09-10T10:28:16.790890+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:28:16.790890+0000","last_undegraded":"2024-09-10T10:28:16.790890+0000","last_fullsized":"2024-09-10T10:28:16.790890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783638+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:28:16.783638+0000","last_peered":"2024-09-10T10:28:16.783638+0000","last_clean":"2024-09-10T10:28:16.783638+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:28:16.783638+0000","last_undegraded":"2024-09-10T10:28:16.783638+0000","last_fullsized":"2024-09-10T10:28:16.783638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206465+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:28:16.206465+0000","last_peered":"2024-09-10T10:28:16.206465+0000","last_clean":"2024-09-10T10:28:16.206465+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:28:16.206465+0000","last_undegraded":"2024-09-10T10:28:16.206465+0000","last_fullsized":"2024-09-10T10:28:16.206465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":73,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.210620+0000","last_change":"2024-09-10T10:27:44.443362+0000","last_active":"2024-09-10T10:28:16.210620+0000","last_peered":"2024-09-10T10:28:16.210620+0000","last_clean":"2024-09-10T10:28:16.210620+0000","last_became_active":"2024-09-10T10:27:44.442834+0000","last_became_peered":"2024-09-10T10:27:44.442834+0000","last_unstale":"2024-09-10T10:28:16.210620+0000","last_undegraded":"2024-09-10T10:28:16.210620+0000","last_fullsized":"2024-09-10T10:28:16.210620+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:53.600117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":47,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.374430+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:28:17.374430+0000","last_peered":"2024-09-10T10:28:17.374430+0000","last_clean":"2024-09-10T10:28:17.374430+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:28:17.374430+0000","last_undegraded":"2024-09-10T10:28:17.374430+0000","last_fullsized":"2024-09-10T10:28:17.374430+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"28'5","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372296+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:28:17.372296+0000","last_peered":"2024-09-10T10:28:17.372296+0000","last_clean":"2024-09-10T10:28:17.372296+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:28:17.372296+0000","last_undegraded":"2024-09-10T10:28:17.372296+0000","last_fullsized":"2024-09-10T10:28:17.372296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.210758+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:28:16.210758+0000","last_peered":"2024-09-10T10:28:16.210758+0000","last_clean":"2024-09-10T10:28:16.210758+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:28:16.210758+0000","last_undegraded":"2024-09-10T10:28:16.210758+0000","last_fullsized":"2024-09-10T10:28:16.210758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783073+0000","last_change":"2024-09-10T10:26:46.193656+0000","last_active":"2024-09-10T10:28:16.783073+0000","last_peered":"2024-09-10T10:28:16.783073+0000","last_clean":"2024-09-10T10:28:16.783073+0000","last_became_active":"2024-09-10T10:26:46.193109+0000","last_became_peered":"2024-09-10T10:26:46.193109+0000","last_unstale":"2024-09-10T10:28:16.783073+0000","last_undegraded":"2024-09-10T10:28:16.783073+0000","last_fullsized":"2024-09-10T10:28:16.783073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.373392+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:28:17.373392+0000","last_peered":"2024-09-10T10:28:17.373392+0000","last_clean":"2024-09-10T10:28:17.373392+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:28:17.373392+0000","last_undegraded":"2024-09-10T10:28:17.373392+0000","last_fullsized":"2024-09-10T10:28:17.373392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205546+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:28:16.205546+0000","last_peered":"2024-09-10T10:28:16.205546+0000","last_clean":"2024-09-10T10:28:16.205546+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:28:16.205546+0000","last_undegraded":"2024-09-10T10:28:16.205546+0000","last_fullsized":"2024-09-10T10:28:16.205546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372870+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:28:17.372870+0000","last_peered":"2024-09-10T10:28:17.372870+0000","last_clean":"2024-09-10T10:28:17.372870+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:28:17.372870+0000","last_undegraded":"2024-09-10T10:28:17.372870+0000","last_fullsized":"2024-09-10T10:28:17.372870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.210661+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:28:16.210661+0000","last_peered":"2024-09-10T10:28:16.210661+0000","last_clean":"2024-09-10T10:28:16.210661+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:28:16.210661+0000","last_undegraded":"2024-09-10T10:28:16.210661+0000","last_fullsized":"2024-09-10T10:28:16.210661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.210797+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:28:16.210797+0000","last_peered":"2024-09-10T10:28:16.210797+0000","last_clean":"2024-09-10T10:28:16.210797+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:28:16.210797+0000","last_undegraded":"2024-09-10T10:28:16.210797+0000","last_fullsized":"2024-09-10T10:28:16.210797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":73,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372240+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:28:17.372240+0000","last_peered":"2024-09-10T10:28:17.372240+0000","last_clean":"2024-09-10T10:28:17.372240+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:28:17.372240+0000","last_undegraded":"2024-09-10T10:28:17.372240+0000","last_fullsized":"2024-09-10T10:28:17.372240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783179+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:28:16.783179+0000","last_peered":"2024-09-10T10:28:16.783179+0000","last_clean":"2024-09-10T10:28:16.783179+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:28:16.783179+0000","last_undegraded":"2024-09-10T10:28:16.783179+0000","last_fullsized":"2024-09-10T10:28:16.783179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.272381+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:28:16.272381+0000","last_peered":"2024-09-10T10:28:16.272381+0000","last_clean":"2024-09-10T10:28:16.272381+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:28:16.272381+0000","last_undegraded":"2024-09-10T10:28:16.272381+0000","last_fullsized":"2024-09-10T10:28:16.272381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'6","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.211247+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:28:16.211247+0000","last_peered":"2024-09-10T10:28:16.211247+0000","last_clean":"2024-09-10T10:28:16.211247+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:28:16.211247+0000","last_undegraded":"2024-09-10T10:28:16.211247+0000","last_fullsized":"2024-09-10T10:28:16.211247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.790805+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:28:16.790805+0000","last_peered":"2024-09-10T10:28:16.790805+0000","last_clean":"2024-09-10T10:28:16.790805+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:28:16.790805+0000","last_undegraded":"2024-09-10T10:28:16.790805+0000","last_fullsized":"2024-09-10T10:28:16.790805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":59,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.211475+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:28:16.211475+0000","last_peered":"2024-09-10T10:28:16.211475+0000","last_clean":"2024-09-10T10:28:16.211475+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:28:16.211475+0000","last_undegraded":"2024-09-10T10:28:16.211475+0000","last_fullsized":"2024-09-10T10:28:16.211475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372446+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:28:17.372446+0000","last_peered":"2024-09-10T10:28:17.372446+0000","last_clean":"2024-09-10T10:28:17.372446+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:28:17.372446+0000","last_undegraded":"2024-09-10T10:28:17.372446+0000","last_fullsized":"2024-09-10T10:28:17.372446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.271865+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:28:16.271865+0000","last_peered":"2024-09-10T10:28:16.271865+0000","last_clean":"2024-09-10T10:28:16.271865+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:28:16.271865+0000","last_undegraded":"2024-09-10T10:28:16.271865+0000","last_fullsized":"2024-09-10T10:28:16.271865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205971+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:28:16.205971+0000","last_peered":"2024-09-10T10:28:16.205971+0000","last_clean":"2024-09-10T10:28:16.205971+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:28:16.205971+0000","last_undegraded":"2024-09-10T10:28:16.205971+0000","last_fullsized":"2024-09-10T10:28:16.205971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783100+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:28:16.783100+0000","last_peered":"2024-09-10T10:28:16.783100+0000","last_clean":"2024-09-10T10:28:16.783100+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:28:16.783100+0000","last_undegraded":"2024-09-10T10:28:16.783100+0000","last_fullsized":"2024-09-10T10:28:16.783100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.211709+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:28:16.211709+0000","last_peered":"2024-09-10T10:28:16.211709+0000","last_clean":"2024-09-10T10:28:16.211709+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:28:16.211709+0000","last_undegraded":"2024-09-10T10:28:16.211709+0000","last_fullsized":"2024-09-10T10:28:16.211709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.374369+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:28:17.374369+0000","last_peered":"2024-09-10T10:28:17.374369+0000","last_clean":"2024-09-10T10:28:17.374369+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:28:17.374369+0000","last_undegraded":"2024-09-10T10:28:17.374369+0000","last_fullsized":"2024-09-10T10:28:17.374369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372937+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:28:17.372937+0000","last_peered":"2024-09-10T10:28:17.372937+0000","last_clean":"2024-09-10T10:28:17.372937+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:28:17.372937+0000","last_undegraded":"2024-09-10T10:28:17.372937+0000","last_fullsized":"2024-09-10T10:28:17.372937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.212099+0000","last_change":"2024-09-10T10:28:11.179535+0000","last_active":"2024-09-10T10:28:16.212099+0000","last_peered":"2024-09-10T10:28:16.212099+0000","last_clean":"2024-09-10T10:28:16.212099+0000","last_became_active":"2024-09-10T10:28:11.179221+0000","last_became_peered":"2024-09-10T10:28:11.179221+0000","last_unstale":"2024-09-10T10:28:16.212099+0000","last_undegraded":"2024-09-10T10:28:16.212099+0000","last_fullsized":"2024-09-10T10:28:16.212099+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:52.639643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205460+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:28:16.205460+0000","last_peered":"2024-09-10T10:28:16.205460+0000","last_clean":"2024-09-10T10:28:16.205460+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:28:16.205460+0000","last_undegraded":"2024-09-10T10:28:16.205460+0000","last_fullsized":"2024-09-10T10:28:16.205460+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.210717+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:28:16.210717+0000","last_peered":"2024-09-10T10:28:16.210717+0000","last_clean":"2024-09-10T10:28:16.210717+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:28:16.210717+0000","last_undegraded":"2024-09-10T10:28:16.210717+0000","last_fullsized":"2024-09-10T10:28:16.210717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205947+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:28:16.205947+0000","last_peered":"2024-09-10T10:28:16.205947+0000","last_clean":"2024-09-10T10:28:16.205947+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:28:16.205947+0000","last_undegraded":"2024-09-10T10:28:16.205947+0000","last_fullsized":"2024-09-10T10:28:16.205947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.783623+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:28:16.783623+0000","last_peered":"2024-09-10T10:28:16.783623+0000","last_clean":"2024-09-10T10:28:16.783623+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:28:16.783623+0000","last_undegraded":"2024-09-10T10:28:16.783623+0000","last_fullsized":"2024-09-10T10:28:16.783623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372772+0000","last_change":"2024-09-10T10:26:41.787435+0000","last_active":"2024-09-10T10:28:17.372772+0000","last_peered":"2024-09-10T10:28:17.372772+0000","last_clean":"2024-09-10T10:28:17.372772+0000","last_became_active":"2024-09-10T10:26:41.787083+0000","last_became_peered":"2024-09-10T10:26:41.787083+0000","last_unstale":"2024-09-10T10:28:17.372772+0000","last_undegraded":"2024-09-10T10:28:17.372772+0000","last_fullsized":"2024-09-10T10:28:17.372772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206896+0000","last_change":"2024-09-10T10:26:41.792587+0000","last_active":"2024-09-10T10:28:16.206896+0000","last_peered":"2024-09-10T10:28:16.206896+0000","last_clean":"2024-09-10T10:28:16.206896+0000","last_became_active":"2024-09-10T10:26:41.792320+0000","last_became_peered":"2024-09-10T10:26:41.792320+0000","last_unstale":"2024-09-10T10:28:16.206896+0000","last_undegraded":"2024-09-10T10:28:16.206896+0000","last_fullsized":"2024-09-10T10:28:16.206896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.273224+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:28:16.273224+0000","last_peered":"2024-09-10T10:28:16.273224+0000","last_clean":"2024-09-10T10:28:16.273224+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:28:16.273224+0000","last_undegraded":"2024-09-10T10:28:16.273224+0000","last_fullsized":"2024-09-10T10:28:16.273224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206675+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:28:16.206675+0000","last_peered":"2024-09-10T10:28:16.206675+0000","last_clean":"2024-09-10T10:28:16.206675+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:28:16.206675+0000","last_undegraded":"2024-09-10T10:28:16.206675+0000","last_fullsized":"2024-09-10T10:28:16.206675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205360+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:28:16.205360+0000","last_peered":"2024-09-10T10:28:16.205360+0000","last_clean":"2024-09-10T10:28:16.205360+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:28:16.205360+0000","last_undegraded":"2024-09-10T10:28:16.205360+0000","last_fullsized":"2024-09-10T10:28:16.205360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.373471+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:28:17.373471+0000","last_peered":"2024-09-10T10:28:17.373471+0000","last_clean":"2024-09-10T10:28:17.373471+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:28:17.373471+0000","last_undegraded":"2024-09-10T10:28:17.373471+0000","last_fullsized":"2024-09-10T10:28:17.373471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.782791+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:28:16.782791+0000","last_peered":"2024-09-10T10:28:16.782791+0000","last_clean":"2024-09-10T10:28:16.782791+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:28:16.782791+0000","last_undegraded":"2024-09-10T10:28:16.782791+0000","last_fullsized":"2024-09-10T10:28:16.782791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.211163+0000","last_change":"2024-09-10T10:26:41.802117+0000","last_active":"2024-09-10T10:28:16.211163+0000","last_peered":"2024-09-10T10:28:16.211163+0000","last_clean":"2024-09-10T10:28:16.211163+0000","last_became_active":"2024-09-10T10:26:41.801798+0000","last_became_peered":"2024-09-10T10:26:41.801798+0000","last_unstale":"2024-09-10T10:28:16.211163+0000","last_undegraded":"2024-09-10T10:28:16.211163+0000","last_fullsized":"2024-09-10T10:28:16.211163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.211275+0000","last_change":"2024-09-10T10:27:58.327792+0000","last_active":"2024-09-10T10:28:16.211275+0000","last_peered":"2024-09-10T10:28:16.211275+0000","last_clean":"2024-09-10T10:28:16.211275+0000","last_became_active":"2024-09-10T10:27:58.327502+0000","last_became_peered":"2024-09-10T10:27:58.327502+0000","last_unstale":"2024-09-10T10:28:16.211275+0000","last_undegraded":"2024-09-10T10:28:16.211275+0000","last_fullsized":"2024-09-10T10:28:16.211275+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372772+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:28:17.372772+0000","last_peered":"2024-09-10T10:28:17.372772+0000","last_clean":"2024-09-10T10:28:17.372772+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:28:17.372772+0000","last_undegraded":"2024-09-10T10:28:17.372772+0000","last_fullsized":"2024-09-10T10:28:17.372772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.372936+0000","last_change":"2024-09-10T10:27:23.958305+0000","last_active":"2024-09-10T10:28:17.372936+0000","last_peered":"2024-09-10T10:28:17.372936+0000","last_clean":"2024-09-10T10:28:17.372936+0000","last_became_active":"2024-09-10T10:27:23.957954+0000","last_became_peered":"2024-09-10T10:27:23.957954+0000","last_unstale":"2024-09-10T10:28:17.372936+0000","last_undegraded":"2024-09-10T10:28:17.372936+0000","last_fullsized":"2024-09-10T10:28:17.372936+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:32.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206980+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:28:16.206980+0000","last_peered":"2024-09-10T10:28:16.206980+0000","last_clean":"2024-09-10T10:28:16.206980+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:28:16.206980+0000","last_undegraded":"2024-09-10T10:28:16.206980+0000","last_fullsized":"2024-09-10T10:28:16.206980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.210543+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:28:16.210543+0000","last_peered":"2024-09-10T10:28:16.210543+0000","last_clean":"2024-09-10T10:28:16.210543+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:28:16.210543+0000","last_undegraded":"2024-09-10T10:28:16.210543+0000","last_fullsized":"2024-09-10T10:28:16.210543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206524+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:28:16.206524+0000","last_peered":"2024-09-10T10:28:16.206524+0000","last_clean":"2024-09-10T10:28:16.206524+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:28:16.206524+0000","last_undegraded":"2024-09-10T10:28:16.206524+0000","last_fullsized":"2024-09-10T10:28:16.206524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205861+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:28:16.205861+0000","last_peered":"2024-09-10T10:28:16.205861+0000","last_clean":"2024-09-10T10:28:16.205861+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:28:16.205861+0000","last_undegraded":"2024-09-10T10:28:16.205861+0000","last_fullsized":"2024-09-10T10:28:16.205861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.205957+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:28:16.205957+0000","last_peered":"2024-09-10T10:28:16.205957+0000","last_clean":"2024-09-10T10:28:16.205957+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:28:16.205957+0000","last_undegraded":"2024-09-10T10:28:16.205957+0000","last_fullsized":"2024-09-10T10:28:16.205957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206351+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:28:16.206351+0000","last_peered":"2024-09-10T10:28:16.206351+0000","last_clean":"2024-09-10T10:28:16.206351+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:28:16.206351+0000","last_undegraded":"2024-09-10T10:28:16.206351+0000","last_fullsized":"2024-09-10T10:28:16.206351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.782991+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:28:16.782991+0000","last_peered":"2024-09-10T10:28:16.782991+0000","last_clean":"2024-09-10T10:28:16.782991+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:28:16.782991+0000","last_undegraded":"2024-09-10T10:28:16.782991+0000","last_fullsized":"2024-09-10T10:28:16.782991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:17.374610+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:28:17.374610+0000","last_peered":"2024-09-10T10:28:17.374610+0000","last_clean":"2024-09-10T10:28:17.374610+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:28:17.374610+0000","last_undegraded":"2024-09-10T10:28:17.374610+0000","last_fullsized":"2024-09-10T10:28:17.374610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T10:28:16.206899+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:28:16.206899+0000","last_peered":"2024-09-10T10:28:16.206899+0000","last_clean":"2024-09-10T10:28:16.206899+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:28:16.206899+0000","last_undegraded":"2024-09-10T10:28:16.206899+0000","last_fullsized":"2024-09-10T10:28:16.206899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":14585,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":588,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":29170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193,"ondisk_log_size":193,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":638976,"data_stored":851293,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":267266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":585728,"data_stored":262501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":259311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":573440,"data_stored":267556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":260031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":260689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":849811,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":8987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":8697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:28:21.505 INFO:tasks.thrashosds.thrasher:Setting 3.2c to [1, 7] 2024-09-10T10:28:21.505 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.2c', '1', '7'] 2024-09-10T10:28:21.505 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.2c 1 7 2024-09-10T10:28:21.540 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:1 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-09-10T10:28:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.269785228, 2024-09-10T10:28:21.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028809700000000001, 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9999999999999999e-07 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268119+0000", 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1810000000000003e-06 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268129+0000", 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9790000000000005e-06 2024-09-10T10:28:21.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268137+0000", 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4800000000000001e-06 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268198+0000", 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0918000000000002e-05 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268226+0000", 2024-09-10T10:28:21.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7267000000000002e-05 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270943+0000", 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027177529999999998 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270996+0000", 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2191999999999998e-05 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:119 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'81, mlcod=16'78)", 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.043934275, 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029329880000000001, 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493966+0000", 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1699999999999998e-07 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494040+0000", 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3883999999999999e-05 2024-09-10T10:28:21.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494048+0000", 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4880000000000007e-06 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494057+0000", 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1310000000000006e-06 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494095+0000", 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8470000000000003e-05 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494108+0000", 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2588999999999999e-05 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496869+0000", 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027610400000000002 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496898+0000", 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9869000000000002e-05 2024-09-10T10:28:21.548 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.043550639, 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005336365, 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494350+0000", 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494416+0000", 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6413000000000003e-05 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494424+0000", 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.576e-06 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494430+0000", 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6579999999999996e-06 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494466+0000", 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5648999999999997e-05 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494478+0000", 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.234e-05 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499654+0000", 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051755209999999998 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499685+0000", 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.167e-05 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.043155829, 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049664970000000003, 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.553 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-10T10:28:21.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494819+0000", 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4690000000000005e-05 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494825+0000", 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7080000000000002e-06 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494828+0000", 2024-09-10T10:28:21.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1109999999999999e-06 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494869+0000", 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1146000000000002e-05 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494881+0000", 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2295e-05 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499687+0000", 2024-09-10T10:28:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048061459999999999 2024-09-10T10:28:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499710+0000", 2024-09-10T10:28:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-05 2024-09-10T10:28:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.757799625999994, 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054008370000000003, 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780104+0000", 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3039999999999998e-06 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0738e-05 2024-09-10T10:28:21.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780141+0000", 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6317999999999999e-05 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2199999999999992e-06 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780405+0000", 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025557599999999998 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780498+0000", 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2860000000000002e-05 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780591+0000", 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3094e-05 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024814469999999999 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785441+0000", 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023678280000000002 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785480+0000", 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9511999999999997e-05 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785501+0000", 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0939999999999999e-05 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.757724792000005, 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054780480000000001, 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.563 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780176+0000", 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6700000000000003e-07 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1289999999999998e-06 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780188+0000", 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.96e-06 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780194+0000", 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9989999999999997e-06 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780882+0000", 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068832400000000003 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780968+0000", 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5959999999999997e-05 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:21.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781098+0000", 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012994100000000001 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783078+0000", 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019803540000000001 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785553+0000", 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002474515 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785608+0000", 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5319999999999999e-05 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785653+0000", 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4879000000000002e-05 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.757693248999999, 2024-09-10T10:28:21.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042025200000000004, 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780217+0000", 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780224+0000", 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2529999999999998e-06 2024-09-10T10:28:21.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780230+0000", 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6110000000000001e-06 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780282+0000", 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2148000000000003e-05 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780394+0000", 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111864 2024-09-10T10:28:21.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780553+0000", 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015913799999999999 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783054+0000", 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025008809999999999 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784349+0000", 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012947480000000001 2024-09-10T10:28:21.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784381+0000", 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2268999999999999e-05 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784409+0000", 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8007000000000001e-05 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.757650897000005, 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054782659999999999, 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780253+0000", 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4199999999999999e-06 2024-09-10T10:28:21.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780260+0000", 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5100000000000001e-06 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780267+0000", 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.443e-06 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781353+0000", 2024-09-10T10:28:21.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010859769999999999 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781457+0000", 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104545 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781561+0000", 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010409200000000001 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783532+0000", 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019702280000000001 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785685+0000", 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002153313 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785710+0000", 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5024999999999998e-05 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785727+0000", 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7175000000000001e-05 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.757624805999995, 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005104407, 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.579 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:21.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780278+0000", 2024-09-10T10:28:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.959e-06 2024-09-10T10:28:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780285+0000", 2024-09-10T10:28:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8040000000000004e-06 2024-09-10T10:28:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780292+0000", 2024-09-10T10:28:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4099999999999996e-06 2024-09-10T10:28:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781345+0000", 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010533579999999999 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781456+0000", 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110624 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781570+0000", 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011417999999999999 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783625+0000", 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020553780000000001 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785303+0000", 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001677827 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785345+0000", 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2364999999999998e-05 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785379+0000", 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4054000000000003e-05 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 94.757600977999999, 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0060500800000000002, 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.587 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:21.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780306+0000", 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780313+0000", 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-06 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1739999999999999e-06 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782543+0000", 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022253419999999999 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010785699999999999 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.786349+0000", 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036978829999999999 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.665898937999998, 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058824769999999997, 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.591 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872002+0000", 2024-09-10T10:28:21.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5149999999999999e-06 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872008+0000", 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.66e-06 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872022+0000", 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3932e-05 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872031+0000", 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5259999999999999e-06 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872111+0000", 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9412000000000005e-05 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872231+0000", 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119843 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:21.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872340+0000", 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109442 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.875579+0000", 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032383899999999998 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877781+0000", 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022022299999999999 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877835+0000", 2024-09-10T10:28:21.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4419000000000002e-05 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877883+0000", 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8108000000000001e-05 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.664810106999994, 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049968149999999999, 2024-09-10T10:28:21.596 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7300000000000005e-07 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.245e-06 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3898e-05 2024-09-10T10:28:21.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873116+0000", 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8050000000000007e-06 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873184+0000", 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7809000000000001e-05 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873257+0000", 2024-09-10T10:28:21.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2602999999999996e-05 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873374+0000", 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011744900000000001 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.876282+0000", 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029079459999999998 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878003+0000", 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001720886 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878054+0000", 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0596999999999997e-05 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878086+0000", 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2804e-05 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.601 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.590417735000003, 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029964739999999998, 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:21.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947484+0000", 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.857e-06 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947490+0000", 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9789999999999998e-06 2024-09-10T10:28:21.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947504+0000", 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3942e-05 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947512+0000", 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5620000000000004e-06 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947599+0000", 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6525000000000003e-05 2024-09-10T10:28:21.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947714+0000", 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114709 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947823+0000", 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109477 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950336+0000", 2024-09-10T10:28:21.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025131509999999999 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950386+0000", 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9974e-05 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950440+0000", 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4011000000000002e-05 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950478+0000", 2024-09-10T10:28:21.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8287000000000001e-05 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.588977337000003, 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043004360000000004, 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948923+0000", 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8999999999999996e-07 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948928+0000", 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948940+0000", 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1759e-05 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948949+0000", 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5149999999999995e-06 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949028+0000", 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8182999999999993e-05 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949137+0000", 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109065 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949245+0000", 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108047 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950193+0000", 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094856200000000002 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953173+0000", 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029801960000000001 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953200+0000", 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6707999999999999e-05 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953223+0000", 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2758000000000001e-05 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 67.588921825, 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046333320000000004, 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.612 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948979+0000", 2024-09-10T10:28:21.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-07 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6490000000000001e-06 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948990+0000", 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2560000000000002e-06 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948996+0000", 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9739999999999999e-06 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949496+0000", 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049934299999999997 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949588+0000", 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1762999999999995e-05 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:21.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949729+0000", 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014145299999999999 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950984+0000", 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012548419999999999 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953505+0000", 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025208129999999998 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953568+0000", 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3695000000000006e-05 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953611+0000", 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2913000000000001e-05 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:21.617 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 61.616870210000002, 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048620219999999997, 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4899999999999995e-07 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921034+0000", 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7050000000000001e-06 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921043+0000", 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1840000000000005e-06 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921049+0000", 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4459999999999998e-06 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921575+0000", 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052602300000000003 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921646+0000", 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0475000000000005e-05 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921769+0000", 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123347 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923077+0000", 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013078759999999999 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925794+0000", 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027172310000000001 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925853+0000", 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8739999999999997e-05 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925892+0000", 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8445999999999997e-05 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.622 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 61.616833176, 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032737339999999999, 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:21.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9919999999999999e-06 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921080+0000", 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0063e-05 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921086+0000", 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921142+0000", 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6174999999999997e-05 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013480499999999999 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921405+0000", 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012819200000000001 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923001+0000", 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015962249999999999 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924243+0000", 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012425820000000001 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924300+0000", 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6901999999999998e-05 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924340+0000", 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9918e-05 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(2.4 epoch 26/18 rep_tid 19 entries 26'1 (0'0) error 2:218afd09:::203.00000003:head by mds.3.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 61.614694731, 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030589089999999998, 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-10T10:28:21.628 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923206+0000", 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7249999999999997e-06 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923229+0000", 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5905000000000001e-05 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923241+0000", 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1990999999999999e-05 2024-09-10T10:28:21.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923317+0000", 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5622000000000002e-05 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926264+0000", 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029470170000000001 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.594336730999999, 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050642550000000001, 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943564+0000", 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.13e-07 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943576+0000", 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2187e-05 2024-09-10T10:28:21.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943585+0000", 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9279999999999999e-06 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943592+0000", 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0110000000000003e-06 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943681+0000", 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9487999999999994e-05 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943767+0000", 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5866999999999996e-05 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943882+0000", 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011443299999999999 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.947042+0000", 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031600629999999998 2024-09-10T10:28:21.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948540+0000", 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001498451 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948581+0000", 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0839e-05 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948627+0000", 2024-09-10T10:28:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6375000000000002e-05 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.594268407999998, 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051386089999999997, 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:21.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943632+0000", 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1500000000000004e-07 2024-09-10T10:28:21.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943643+0000", 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0954e-05 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943652+0000", 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4929999999999998e-06 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943657+0000", 2024-09-10T10:28:21.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6509999999999998e-06 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944230+0000", 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057322999999999999 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944340+0000", 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010986 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:21.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948770+0000", 2024-09-10T10:28:21.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044297060000000003 2024-09-10T10:28:21.641 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.641 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.641 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.641 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:21.641 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:21.641 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:21.642 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:21.643 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:21.641+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:21.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.733+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f5c7015f870 con 0x7f5c700ccc30 2024-09-10T10:28:21.744 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:21.743+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:21.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.777+0000 7fc61c142640 1 Processor -- start 2024-09-10T10:28:21.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.778+0000 7fc61c142640 1 -- start start 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc61c142640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 0x7fc61406d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc61c142640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc614157e30 0x7fc6141582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc61c142640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6140c4820 con 0x7fc61415a790 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc61c142640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc6140c69b0 con 0x7fc614157e30 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc61c142640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc614157950 con 0x7fc6140c7200 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc6196dd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 0x7fc61406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc619ede640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fc61415a790 0x7fc61406e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:47434/0 (socket says 172.21.15.120:47434) 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc619ede640 1 -- 172.21.15.120:0/3234881767 learned_addr learned my addr 172.21.15.120:0/3234881767 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc61a6df640 1 --2- 172.21.15.120:0/3234881767 >> v2:172.21.15.134:3300/0 conn(0x7fc614157e30 0x7fc6141582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc61a6df640 1 -- 172.21.15.120:0/3234881767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 msgr2=0x7fc61406d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc61a6df640 1 --2- 172.21.15.120:0/3234881767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 0x7fc61406d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc61a6df640 1 -- 172.21.15.120:0/3234881767 >> v1:172.21.15.120:6789/0 conn(0x7fc61415a790 legacy=0x7fc61406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc61a6df640 1 -- 172.21.15.120:0/3234881767 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6140c6110 con 0x7fc614157e30 2024-09-10T10:28:21.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.779+0000 7fc6196dd640 1 --2- 172.21.15.120:0/3234881767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 0x7fc61406d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:21.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.780+0000 7fc61a6df640 1 --2- 172.21.15.120:0/3234881767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc614157e30 0x7fc6141582c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00bfa0 tx=0x7fc60c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5fba1cafe8b0140 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.780+0000 7fc618edc640 1 -- 172.21.15.120:0/3234881767 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c036070 con 0x7fc614157e30 2024-09-10T10:28:21.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.780+0000 7fc618edc640 1 -- 172.21.15.120:0/3234881767 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c009540 con 0x7fc614157e30 2024-09-10T10:28:21.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.780+0000 7fc618edc640 1 -- 172.21.15.120:0/3234881767 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c009910 con 0x7fc614157e30 2024-09-10T10:28:21.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.781+0000 7fc61c142640 1 -- 172.21.15.120:0/3234881767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc614157e30 msgr2=0x7fc6141582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.781+0000 7fc61c142640 1 --2- 172.21.15.120:0/3234881767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc614157e30 0x7fc6141582c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00bfa0 tx=0x7fc60c02a660 comp rx=0 tx=0).stop 2024-09-10T10:28:21.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.781+0000 7fc61c142640 1 -- 172.21.15.120:0/3234881767 shutdown_connections 2024-09-10T10:28:21.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.781+0000 7fc61c142640 1 --2- 172.21.15.120:0/3234881767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc614157e30 0x7fc6141582c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.781+0000 7fc61c142640 1 --2- 172.21.15.120:0/3234881767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 0x7fc61406d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.781+0000 7fc61c142640 1 -- 172.21.15.120:0/3234881767 >> 172.21.15.120:0/3234881767 conn(0x7fc6140728b0 msgr2=0x7fc614072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:21.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.781+0000 7fc61c142640 1 -- 172.21.15.120:0/3234881767 shutdown_connections 2024-09-10T10:28:21.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.781+0000 7fc61c142640 1 -- 172.21.15.120:0/3234881767 wait complete. 2024-09-10T10:28:21.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61c142640 1 Processor -- start 2024-09-10T10:28:21.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61c142640 1 -- start start 2024-09-10T10:28:21.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61c142640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 0x7fc6141eff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61c142640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc614157e30 0x7fc6140cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61c142640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc61415a790 0x7fc6140cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61c142640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140c44e0 con 0x7fc61415a790 2024-09-10T10:28:21.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61c142640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140c66d0 con 0x7fc614157e30 2024-09-10T10:28:21.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61c142640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc6140c78e0 con 0x7fc6140c7200 2024-09-10T10:28:21.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc619ede640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 0x7fc6141eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61a6df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc61415a790 0x7fc6140cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61a6df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc61415a790 0x7fc6140cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53928/0 (socket says 172.21.15.120:53928) 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc619ede640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 0x7fc6141eff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:41626/0 (socket says 172.21.15.120:41626) 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc61a6df640 1 -- 172.21.15.120:0/2707590739 learned_addr learned my addr 172.21.15.120:0/2707590739 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.783+0000 7fc6196dd640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc614157e30 0x7fc6140cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.784+0000 7fc61a6df640 1 -- 172.21.15.120:0/2707590739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 msgr2=0x7fc6141eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.784+0000 7fc61a6df640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 0x7fc6141eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.784+0000 7fc61a6df640 1 -- 172.21.15.120:0/2707590739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc614157e30 msgr2=0x7fc6140cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.784+0000 7fc61a6df640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc614157e30 0x7fc6140cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.784+0000 7fc61a6df640 1 -- 172.21.15.120:0/2707590739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc614152050 con 0x7fc61415a790 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.784+0000 7fc61a6df640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc61415a790 0x7fc6140cc060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc60c034040 tx=0x7fc60c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.784+0000 7fc60a7fc640 1 -- 172.21.15.120:0/2707590739 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c03f020 con 0x7fc61415a790 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.784+0000 7fc60a7fc640 1 -- 172.21.15.120:0/2707590739 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c037040 con 0x7fc61415a790 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.784+0000 7fc60a7fc640 1 -- 172.21.15.120:0/2707590739 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c036070 con 0x7fc61415a790 2024-09-10T10:28:21.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.785+0000 7fc61c142640 1 -- 172.21.15.120:0/2707590739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc614152680 con 0x7fc61415a790 2024-09-10T10:28:21.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.785+0000 7fc61c142640 1 -- 172.21.15.120:0/2707590739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6141f0ae0 con 0x7fc61415a790 2024-09-10T10:28:21.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.786+0000 7fc61c142640 1 -- 172.21.15.120:0/2707590739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6140ca650 con 0x7fc61415a790 2024-09-10T10:28:21.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.786+0000 7fc60a7fc640 1 -- 172.21.15.120:0/2707590739 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc60c042020 con 0x7fc61415a790 2024-09-10T10:28:21.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.787+0000 7fc60a7fc640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc5f806be80 0x7fc5f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:21.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.787+0000 7fc60a7fc640 1 -- 172.21.15.120:0/2707590739 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6801+0+0 (secure 0 0 0) 0x7fc60c0ab4b0 con 0x7fc61415a790 2024-09-10T10:28:21.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.788+0000 7fc619ede640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc5f806be80 0x7fc5f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:21.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.788+0000 7fc619ede640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc5f806be80 0x7fc5f806e340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc610007fd0 tx=0x7fc610007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:21.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:21.792+0000 7fc60a7fc640 1 -- 172.21.15.120:0/2707590739 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc6140ca650 con 0x7fc61415a790 2024-09-10T10:28:21.844 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:21.843+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:21.853 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:21.853 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:21.853 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:21.853 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:21.865 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:21.949 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:21.948+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:22.023 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:22.023 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:22.024 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:22.024 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:22.024 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:22.033 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:22.074 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:22.073+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:22.146 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:22.145+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:22.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.163+0000 7fc61c142640 1 -- 172.21.15.120:0/2707590739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pg-upmap", "pgid": "3.2c", "id": [1, 7]} v 0) -- 0x7fc6140731c0 con 0x7fc61415a790 2024-09-10T10:28:22.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.218+0000 7fc60a7fc640 1 -- 172.21.15.120:0/2707590739 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap", "pgid": "3.2c", "id": [1, 7]}]=0 set 3.2c pg_upmap mapping to [1,7] v49) ==== 124+0+0 (secure 0 0 0) 0x7fc6140731c0 con 0x7fc61415a790 2024-09-10T10:28:22.220 INFO:teuthology.orchestra.run.smithi120.stderr:set 3.2c pg_upmap mapping to [1,7] 2024-09-10T10:28:22.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.221+0000 7fc61c142640 1 -- 172.21.15.120:0/2707590739 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc5f806be80 msgr2=0x7fc5f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.221+0000 7fc61c142640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc5f806be80 0x7fc5f806e340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc610007fd0 tx=0x7fc610007e80 comp rx=0 tx=0).stop 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.222+0000 7fc61c142640 1 -- 172.21.15.120:0/2707590739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc61415a790 msgr2=0x7fc6140cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.222+0000 7fc61c142640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc61415a790 0x7fc6140cc060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc60c034040 tx=0x7fc60c004a40 comp rx=0 tx=0).stop 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.224+0000 7fc61c142640 1 -- 172.21.15.120:0/2707590739 shutdown_connections 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.224+0000 7fc61c142640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc5f806be80 0x7fc5f806e340 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.224+0000 7fc61c142640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc61415a790 0x7fc6140cc060 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.224+0000 7fc61c142640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc614157e30 0x7fc6140cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.224+0000 7fc61c142640 1 --2- 172.21.15.120:0/2707590739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6140c7200 0x7fc6141eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.224+0000 7fc61c142640 1 -- 172.21.15.120:0/2707590739 >> 172.21.15.120:0/2707590739 conn(0x7fc6140728b0 msgr2=0x7fc6141558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.224+0000 7fc61c142640 1 -- 172.21.15.120:0/2707590739 shutdown_connections 2024-09-10T10:28:22.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.224+0000 7fc61c142640 1 -- 172.21.15.120:0/2707590739 wait complete. 2024-09-10T10:28:22.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.229+0000 7f5c667fc640 1 -- 172.21.15.120:0/859189187 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v49) ==== 101+0+0 (secure 0 0 0) 0x7f5c6c058980 con 0x7f5c700ccc30 2024-09-10T10:28:22.230 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-10T10:28:22.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5c5406be80 msgr2=0x7f5c5406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:22.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5c5406be80 0x7f5c5406e340 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c60034000 tx=0x7f5c6002e000 comp rx=0 tx=0).stop 2024-09-10T10:28:22.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c700ccc30 msgr2=0x7f5c70154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:22.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c700ccc30 0x7f5c70154940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c430 tx=0x7f5c6c00c8f0 comp rx=0 tx=0).stop 2024-09-10T10:28:22.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 shutdown_connections 2024-09-10T10:28:22.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5c5406be80 0x7f5c5406e340 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:22.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5c70162ed0 0x7f5c7014f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:22.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c70162430 0x7f5c700c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:22.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 --2- 172.21.15.120:0/859189187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c700ccc30 0x7f5c70154940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:22.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 >> 172.21.15.120:0/859189187 conn(0x7f5c700703e0 msgr2=0x7f5c7006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:22.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 shutdown_connections 2024-09-10T10:28:22.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:22.235+0000 7f5c77f4b640 1 -- 172.21.15.120:0/859189187 wait complete. 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.461688495000004, 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049304329999999997, 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.245 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780094+0000", 2024-09-10T10:28:22.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780107+0000", 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3703000000000001e-05 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780116+0000", 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5210000000000002e-06 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6340000000000007e-06 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001735838 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782000+0000", 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013984100000000001 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:22.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782131+0000", 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000131025 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783737+0000", 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016054179999999999 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784937+0000", 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001200638 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784987+0000", 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9591e-05 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785024+0000", 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6751000000000001e-05 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:22.250 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.461649039999998, 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052929559999999997, 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780136+0000", 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9950000000000001e-06 2024-09-10T10:28:22.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780144+0000", 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.174e-06 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4949999999999999e-06 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781163+0000", 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001013438 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781279+0000", 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011579200000000001 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781388+0000", 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108834 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783243+0000", 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00185491 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785355+0000", 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021117990000000001 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785386+0000", 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1612000000000002e-05 2024-09-10T10:28:22.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785426+0000", 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9474999999999997e-05 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:8 2.10 e24/18 2:09401a81:::601.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.459457810000004, 2024-09-10T10:28:22.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005352224, 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4400000000000001e-07 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782467+0000", 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142452 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782598+0000", 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013078300000000001 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782604+0000", 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.533e-06 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:28:22.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7073999999999997e-05 2024-09-10T10:28:22.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782668+0000", 2024-09-10T10:28:22.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6861999999999999e-05 2024-09-10T10:28:22.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787632+0000", 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049636510000000003 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787676+0000", 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4425000000000002e-05 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.261 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:9 2.10 e24/18 2:09401a81:::601.00000000:head v 24'4, mlcod=0'0)", 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.458820794999994, 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047438960000000001, 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782962+0000", 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4499999999999997e-07 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782967+0000", 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2000000000000002e-06 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782980+0000", 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3685e-05 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782987+0000", 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4690000000000002e-06 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783112+0000", 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012481300000000001 2024-09-10T10:28:22.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783130+0000", 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7753000000000001e-05 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787678+0000", 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045487419999999997 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787705+0000", 2024-09-10T10:28:22.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6588999999999999e-05 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:10 2.10 e24/18 2:0bd6d154:::602.00000000:head v 24'5, mlcod=0'0)", 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.458713861000007, 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00466629, 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1300000000000001e-07 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6970000000000001e-06 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783309+0000", 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000236209 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783319+0000", 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7180000000000008e-06 2024-09-10T10:28:22.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783365+0000", 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6179e-05 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783383+0000", 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8382000000000001e-05 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787708+0000", 2024-09-10T10:28:22.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043246059999999999 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787734+0000", 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6086000000000001e-05 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.369311667000005, 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054527359999999997, 2024-09-10T10:28:22.270 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872472+0000", 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5409999999999999e-06 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872479+0000", 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9550000000000003e-06 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872494+0000", 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5452999999999999e-05 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872503+0000", 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1509999999999993e-06 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872571+0000", 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7747e-05 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872704+0000", 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013269900000000001 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872804+0000", 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9934999999999998e-05 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877617+0000", 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048138069999999998 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877819+0000", 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020164799999999999 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877880+0000", 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1296000000000004e-05 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877923+0000", 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2503999999999998e-05 2024-09-10T10:28:22.276 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.368290099999996, 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0098302110000000002, 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873493+0000", 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873498+0000", 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1619999999999999e-06 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873511+0000", 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3319000000000001e-05 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873520+0000", 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3409999999999992e-06 2024-09-10T10:28:22.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873586+0000", 2024-09-10T10:28:22.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6115e-05 2024-09-10T10:28:22.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873660+0000", 2024-09-10T10:28:22.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3103999999999997e-05 2024-09-10T10:28:22.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873775+0000", 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115617 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878285+0000", 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045096049999999999 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883229+0000", 2024-09-10T10:28:22.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049443999999999998 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883283+0000", 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4262999999999999e-05 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883322+0000", 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8417000000000002e-05 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.367800959999997, 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047062800000000002, 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873982+0000", 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2900000000000003e-07 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873986+0000", 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3939999999999998e-06 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873995+0000", 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0750000000000004e-06 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874002+0000", 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874061+0000", 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8770999999999997e-05 2024-09-10T10:28:22.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874178+0000", 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117108 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874290+0000", 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112503 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878269+0000", 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039784870000000002 2024-09-10T10:28:22.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878593+0000", 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032385299999999998 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878644+0000", 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1790999999999999e-05 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878687+0000", 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2738999999999999e-05 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.367200681, 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089185449999999999, 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.288 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874582+0000", 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1699999999999997e-07 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874592+0000", 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0604999999999999e-05 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874602+0000", 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874610+0000", 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1919999999999999e-06 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874696+0000", 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6033999999999997e-05 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874809+0000", 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113373 2024-09-10T10:28:22.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874923+0000", 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114077 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878289+0000", 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033662380000000001 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883420+0000", 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051305760000000004 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883465+0000", 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5252e-05 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883500+0000", 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4415000000000002e-05 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.293337359999995, 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073932399999999997, 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948446+0000", 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3400000000000001e-06 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948451+0000", 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6219999999999997e-06 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948464+0000", 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2955e-05 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948472+0000", 2024-09-10T10:28:22.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0730000000000005e-06 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948534+0000", 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1727e-05 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948649+0000", 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114991 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948757+0000", 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107935 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950179+0000", 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014219440000000001 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055817970000000003 2024-09-10T10:28:22.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955794+0000", 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3012000000000002e-05 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955838+0000", 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3844000000000002e-05 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.292798540999996, 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069680200000000001, 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948984+0000", 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6100000000000001e-07 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948993+0000", 2024-09-10T10:28:22.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9809999999999992e-06 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949002+0000", 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8789999999999992e-06 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949008+0000", 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2759999999999997e-06 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949057+0000", 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9159999999999997e-05 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949128+0000", 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1292999999999997e-05 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949261+0000", 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132564 2024-09-10T10:28:22.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950220+0000", 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00095953300000000004 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955890+0000", 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056692790000000002 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955928+0000", 2024-09-10T10:28:22.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8278000000000002e-05 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955951+0000", 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3215999999999999e-05 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.320822030999999, 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050489669999999997, 2024-09-10T10:28:22.305 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5799999999999999e-07 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920964+0000", 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6720000000000002e-06 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920971+0000", 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5140000000000004e-06 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920977+0000", 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0100000000000001e-06 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921033+0000", 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5899000000000001e-05 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921172+0000", 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139011 2024-09-10T10:28:22.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921288+0000", 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011605200000000001 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922851+0000", 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001562499 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925915+0000", 2024-09-10T10:28:22.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003063727 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925969+0000", 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4083999999999999e-05 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926009+0000", 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9941000000000003e-05 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.320215173999998, 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045836510000000002, 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4499999999999996e-07 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921572+0000", 2024-09-10T10:28:22.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3239999999999996e-06 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921581+0000", 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9190000000000005e-06 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921589+0000", 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0649999999999999e-06 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921635+0000", 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6863999999999997e-05 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921705+0000", 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9046000000000003e-05 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921818+0000", 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011313499999999999 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923033+0000", 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012153590000000001 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926055+0000", 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030223310000000001 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926104+0000", 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8421999999999998e-05 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926150+0000", 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6440999999999998e-05 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.319776230000002, 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051621749999999997, 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.316 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3600000000000003e-07 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922011+0000", 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4449999999999998e-06 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922019+0000", 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0930000000000003e-06 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922025+0000", 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5060000000000001e-06 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922076+0000", 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0671000000000003e-05 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922175+0000", 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9611000000000006e-05 2024-09-10T10:28:22.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922278+0000", 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102565 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925217+0000", 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029384670000000002 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927074+0000", 2024-09-10T10:28:22.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018579499999999999 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927129+0000", 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4428000000000001e-05 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927168+0000", 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8803000000000001e-05 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.31974056, 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059957930000000001, 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922042+0000", 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922045+0000", 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4209999999999999e-06 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922052+0000", 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1430000000000001e-06 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922059+0000", 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2139999999999999e-06 2024-09-10T10:28:22.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922495+0000", 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043517499999999999 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922563+0000", 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8084999999999996e-05 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922679+0000", 2024-09-10T10:28:22.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011609599999999999 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927208+0000", 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045292170000000003 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927952+0000", 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00074421700000000003 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928001+0000", 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9004e-05 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928037+0000", 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.578e-05 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.327 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:22.328 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.319713800999999, 2024-09-10T10:28:22.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045268130000000002, 2024-09-10T10:28:22.328 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.328 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.328 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.328 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:22.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922078+0000", 2024-09-10T10:28:22.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922085+0000", 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3710000000000002e-06 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922091+0000", 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0340000000000002e-06 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922144+0000", 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3106999999999998e-05 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922248+0000", 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010373400000000001 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922399+0000", 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015127200000000001 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925201+0000", 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028019189999999999 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926514+0000", 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001312279 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926561+0000", 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000002e-05 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926595+0000", 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3408000000000003e-05 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.319078953000002, 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005322116, 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.333 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:22.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999996e-07 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922707+0000", 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.766e-06 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922717+0000", 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7229999999999999e-06 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922724+0000", 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8410000000000002e-06 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922792+0000", 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8557000000000001e-05 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922912+0000", 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119354 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928025+0000", 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051131919999999999 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.318602800000001, 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049271260000000004, 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923180+0000", 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923183+0000", 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4000000000000001e-06 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923196+0000", 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3127e-05 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923203+0000", 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.939e-06 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923270+0000", 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7087000000000003e-05 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923359+0000", 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9054999999999994e-05 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928106+0000", 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047469239999999996 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.318142158999997, 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045860270000000003, 2024-09-10T10:28:22.341 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2500000000000005e-07 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923644+0000", 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8240000000000001e-06 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923659+0000", 2024-09-10T10:28:22.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5358000000000001e-05 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923666+0000", 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9659999999999999e-06 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923725+0000", 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8674000000000001e-05 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923850+0000", 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000124797 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928226+0000", 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043757830000000003 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.345 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 56.300619724000001, 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005230859, 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941164+0000", 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.863e-06 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941172+0000", 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2360000000000004e-06 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941190+0000", 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7812000000000002e-05 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941200+0000", 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7599999999999997e-06 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941350+0000", 2024-09-10T10:28:22.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015070400000000001 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941501+0000", 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015044200000000001 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000167076 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943487+0000", 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018190260000000001 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946297+0000", 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028102840000000001 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946349+0000", 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1649e-05 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946393+0000", 2024-09-10T10:28:22.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4007000000000001e-05 2024-09-10T10:28:22.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.351 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.351 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.351 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:22.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:22.354 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:22.353+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:22.455 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:22.454+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:22.462 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:22.462 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:22.462 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:22.462 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:22.471 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:22.556 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:22.555+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:22.622 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:22.623 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:22.623 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:22.623 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:22.623 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:22.632 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:22.676 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:22.675+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:22.753 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:22.753 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.971002822000003, 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078690670000000004, 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:22.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2200000000000004e-07 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780048+0000", 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2427999999999999e-05 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780057+0000", 2024-09-10T10:28:22.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1090000000000004e-06 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780063+0000", 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9780000000000002e-06 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011213480000000001 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781281+0000", 2024-09-10T10:28:22.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6728e-05 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781382+0000", 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101455 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784820+0000", 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034376200000000002 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787779+0000", 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002959373 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787852+0000", 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3135999999999999e-05 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787894+0000", 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1369999999999999e-05 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.758 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.970955281000002, 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079289789999999992, 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780073+0000", 2024-09-10T10:28:22.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780075+0000", 2024-09-10T10:28:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5129999999999999e-06 2024-09-10T10:28:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780082+0000", 2024-09-10T10:28:22.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9410000000000001e-06 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780087+0000", 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1880000000000001e-06 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781620+0000", 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001532346 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781698+0000", 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8232999999999994e-05 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781816+0000", 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011825800000000001 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784824+0000", 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030076349999999998 2024-09-10T10:28:22.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787937+0000", 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031134829999999998 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787969+0000", 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1745999999999997e-05 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788001+0000", 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2132000000000003e-05 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.970925066000007, 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079888619999999994, 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.765 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780103+0000", 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780106+0000", 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1379999999999999e-06 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780115+0000", 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4299999999999995e-06 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780122+0000", 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3380000000000001e-06 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782041+0000", 2024-09-10T10:28:22.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019193700000000001 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782149+0000", 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107493 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782246+0000", 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999996e-05 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785205+0000", 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002958535 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788041+0000", 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028359650000000002 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788069+0000", 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8123000000000002e-05 2024-09-10T10:28:22.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788091+0000", 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2506000000000001e-05 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.970896525000001, 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080531070000000003, 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.771 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780134+0000", 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0570000000000001e-06 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780143+0000", 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-06 2024-09-10T10:28:22.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780156+0000", 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3179e-05 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023278280000000001 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782555+0000", 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0494000000000006e-05 2024-09-10T10:28:22.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782675+0000", 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120463 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785209+0000", 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002533455 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788127+0000", 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029181260000000001 2024-09-10T10:28:22.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788155+0000", 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8495000000000001e-05 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788184+0000", 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8736999999999999e-05 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.776 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.970864008000007, 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008097524, 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780164+0000", 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.13e-07 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9780000000000001e-06 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780174+0000", 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7510000000000002e-06 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780179+0000", 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0869999999999997e-06 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782890+0000", 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027113509999999999 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782985+0000", 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5152e-05 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783090+0000", 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104947 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785222+0000", 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021320269999999999 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788216+0000", 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029940629999999999 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788242+0000", 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5259999999999999e-05 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788261+0000", 2024-09-10T10:28:22.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9395e-05 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.970841637000007, 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081619529999999996, 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.782 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:22.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780189+0000", 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.926e-06 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780197+0000", 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2039999999999997e-06 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780202+0000", 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783326+0000", 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003123855 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783402+0000", 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6548e-05 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783558+0000", 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015539199999999999 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785225+0000", 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001666989 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788290+0000", 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030647890000000001 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788316+0000", 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6662e-05 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788348+0000", 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1417000000000001e-05 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:3 2.10 2:0f65c9b8:::60a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:22.787 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 73.879248125000004, 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066829990000000002, 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:22.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871781+0000", 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2129999999999998e-06 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871790+0000", 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3380000000000005e-06 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871805+0000", 2024-09-10T10:28:22.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5353999999999999e-05 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871817+0000", 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1399999999999999e-05 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871932+0000", 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114982 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872069+0000", 2024-09-10T10:28:22.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013713199999999999 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872208+0000", 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013976900000000001 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874946+0000", 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027378789999999999 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878355+0000", 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034089139999999999 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878413+0000", 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7540000000000001e-05 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878462+0000", 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9478000000000003e-05 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.803804993, 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067332750000000004, 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947223+0000", 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7299999999999995e-07 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947227+0000", 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.895e-06 2024-09-10T10:28:22.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947235+0000", 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947246+0000", 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0738e-05 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948505+0000", 2024-09-10T10:28:22.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001259402 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948578+0000", 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2788999999999997e-05 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948718+0000", 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013953699999999999 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953391+0000", 2024-09-10T10:28:22.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046736670000000003 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953881+0000", 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048956900000000003 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953923+0000", 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2153000000000001e-05 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953956+0000", 2024-09-10T10:28:22.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2526e-05 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.803746204999996, 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066654160000000004, 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.798 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947282+0000", 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947285+0000", 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7689999999999998e-06 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947292+0000", 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6200000000000001e-06 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947304+0000", 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1702000000000001e-05 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947363+0000", 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9426000000000003e-05 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947485+0000", 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012227599999999999 2024-09-10T10:28:22.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947588+0000", 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102641 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950690+0000", 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003102264 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953856+0000", 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031656200000000001 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953907+0000", 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1430000000000001e-05 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953947+0000", 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9164e-05 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.803 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.803709032, 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071148679999999999, 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947319+0000", 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947322+0000", 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.523e-06 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947331+0000", 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3240000000000003e-06 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947336+0000", 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7189999999999998e-06 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947994+0000", 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065770599999999998 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948080+0000", 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5649000000000006e-05 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111783 2024-09-10T10:28:22.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953965+0000", 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057732089999999996 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954343+0000", 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037846199999999998 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954395+0000", 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.126e-05 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954433+0000", 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8693999999999999e-05 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.809 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.803578836, 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067554449999999997, 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6899999999999996e-07 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947454+0000", 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947463+0000", 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0310000000000004e-06 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947477+0000", 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4328000000000001e-05 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3869999999999998e-05 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947631+0000", 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9874e-05 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947750+0000", 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118483 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953644+0000", 2024-09-10T10:28:22.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058946889999999998 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954111+0000", 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00046712099999999998 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954162+0000", 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000003e-05 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954204+0000", 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1718000000000003e-05 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.803191798, 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064551169999999998, 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.815 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:22.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3e-07 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947841+0000", 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3460000000000004e-06 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947856+0000", 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947864+0000", 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4939999999999997e-06 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947997+0000", 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013305399999999999 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948076+0000", 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9146000000000004e-05 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115586 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953707+0000", 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055153470000000003 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954215+0000", 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050850400000000001 2024-09-10T10:28:22.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954258+0000", 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2932000000000002e-05 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954291+0000", 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2322e-05 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.803152478000001, 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088919089999999999, 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:22.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7199999999999999e-07 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947879+0000", 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8380000000000004e-06 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947887+0000", 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5410000000000003e-06 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947893+0000", 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6930000000000004e-06 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948405+0000", 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051236000000000001 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948502+0000", 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6916999999999994e-05 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948647+0000", 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000145265 2024-09-10T10:28:22.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954479+0000", 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058316970000000003 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956680+0000", 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002201494 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956728+0000", 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8161999999999997e-05 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956767+0000", 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.837e-05 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.826 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.802589816999998, 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083279530000000008, 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8999999999999995e-07 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948443+0000", 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2189999999999998e-06 2024-09-10T10:28:22.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948450+0000", 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8879999999999997e-06 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948457+0000", 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0750000000000003e-06 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948501+0000", 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948594+0000", 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3009000000000003e-05 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948696+0000", 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101461 2024-09-10T10:28:22.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953756+0000", 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050607129999999997 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956690+0000", 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029334109999999999 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956737+0000", 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7080999999999999e-05 2024-09-10T10:28:22.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956766+0000", 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8778000000000001e-05 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.802050143000002, 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0077788589999999999, 2024-09-10T10:28:22.832 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948986+0000", 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948998+0000", 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.19e-05 2024-09-10T10:28:22.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949007+0000", 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.081e-06 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949062+0000", 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754999999999997e-05 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949172+0000", 2024-09-10T10:28:22.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110128 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949282+0000", 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109289 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955941+0000", 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066594990000000001 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956654+0000", 2024-09-10T10:28:22.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071279399999999999 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956706+0000", 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2027000000000002e-05 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956756+0000", 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0012999999999997e-05 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.801512048999996, 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073200349999999999, 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:22.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949516+0000", 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999995e-07 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949522+0000", 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3980000000000004e-06 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949534+0000", 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1562e-05 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949541+0000", 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1380000000000001e-06 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950700+0000", 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011593459999999999 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950786+0000", 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5556999999999994e-05 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950898+0000", 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111743 2024-09-10T10:28:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954326+0000", 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034280510000000001 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956774+0000", 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024477430000000001 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956815+0000", 2024-09-10T10:28:22.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1752999999999998e-05 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956835+0000", 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0032999999999999e-05 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:22.843 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.801469748000002, 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073303700000000001, 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:22.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949571+0000", 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2298e-05 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949579+0000", 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5909999999999996e-06 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949585+0000", 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7370000000000003e-06 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953802+0000", 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042170929999999999 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953915+0000", 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011256699999999999 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956888+0000", 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029734639999999999 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:24 2.3 e26/18 2:c216c0d1:::mds2_inotable:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:22.847 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 68.800651747000003, 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0063254399999999999, 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950380+0000", 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2590000000000002e-06 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950387+0000", 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:22.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950392+0000", 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950438+0000", 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5592e-05 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950457+0000", 2024-09-10T10:28:22.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8569000000000001e-05 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956657+0000", 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062003409999999998 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956701+0000", 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4140000000000001e-05 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:29 2.10 2:0e4da237:::203.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 62.828990943999997, 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064228799999999997, 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:22.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922037+0000", 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922040+0000", 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.694e-06 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922054+0000", 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906e-05 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922065+0000", 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922125+0000", 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.982e-05 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922233+0000", 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108175 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928459+0000", 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062264570000000003 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.855 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 56.809727314, 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081214100000000008, 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:22.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941301+0000", 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.062e-06 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:22.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941306+0000", 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0919999999999997e-06 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941316+0000", 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5640000000000002e-06 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941323+0000", 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941405+0000", 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1650000000000006e-05 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941549+0000", 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014447599999999999 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012929 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943549+0000", 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018704049999999999 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949343+0000", 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057943300000000003 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949378+0000", 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4474999999999998e-05 2024-09-10T10:28:22.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:22.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:22.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:22.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949421+0000", 2024-09-10T10:28:22.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3955999999999997e-05 2024-09-10T10:28:22.861 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.861 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.861 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.861 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:22.861 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:22.861 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:22.863 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:22.863 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:22.862+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:22.964 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:22.963+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:23.011 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:23.012 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:23.012 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:23.012 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:23.021 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:23.064 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:23.064+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:23.127 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:23.127 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:23.127 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:23.127 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:23.127 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:23.136 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:23.165 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:23.164+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:23.256 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:23.256 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:23.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:23.256 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:23.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.256 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.473403657000006, 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054521709999999996, 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780407+0000", 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.517e-06 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780414+0000", 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3910000000000003e-06 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780429+0000", 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5379999999999998e-05 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780438+0000", 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9230000000000008e-06 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780885+0000", 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044711300000000001 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780979+0000", 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4312000000000001e-05 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785856+0000", 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048765350000000004 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.260 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.473328929999994, 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028177609999999998, 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780479+0000", 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9800000000000003e-07 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780488+0000", 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3349999999999996e-06 2024-09-10T10:28:23.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780512+0000", 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3156999999999999e-05 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780518+0000", 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.764e-06 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780576+0000", 2024-09-10T10:28:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7813999999999999e-05 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780700+0000", 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123733 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783296+0000", 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025963599999999998 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:19 2.14 e24/18 2:2f55791f:::606.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.472209770000006, 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003505083, 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9400000000000005e-07 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781606+0000", 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6559999999999992e-06 2024-09-10T10:28:23.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781624+0000", 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7575e-05 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781630+0000", 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8630000000000002e-06 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781683+0000", 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475000000000002e-05 2024-09-10T10:28:23.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781700+0000", 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7393000000000001e-05 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785058+0000", 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033575660000000002 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785103+0000", 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.268 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:1 2.14 e24/18 2:292cf221:::200.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.472041263999998, 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033666960000000002, 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781767+0000", 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781772+0000", 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5539999999999997e-06 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781781+0000", 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6149999999999997e-06 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781787+0000", 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1589999999999998e-06 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2859000000000002e-05 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781876+0000", 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5916000000000001e-05 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785108+0000", 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003232146 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785133+0000", 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5015999999999999e-05 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:23 2.e e24/18 2:717a0223:::608.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.471426727999997, 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027543580000000002, 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1699999999999998e-07 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782387+0000", 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3229999999999997e-06 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782399+0000", 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2466000000000001e-05 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782406+0000", 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9460000000000001e-06 2024-09-10T10:28:23.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782468+0000", 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2347999999999993e-05 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6016e-05 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785096+0000", 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026118629999999999 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785135+0000", 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8779000000000003e-05 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:24 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 74.376245022000006, 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062416909999999997, 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.277 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877564+0000", 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-10T10:28:23.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877576+0000", 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2411e-05 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877592+0000", 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5812e-05 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877600+0000", 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3590000000000002e-06 2024-09-10T10:28:23.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877661+0000", 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1749999999999997e-05 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877785+0000", 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123807 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877936+0000", 2024-09-10T10:28:23.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015084300000000001 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.879317+0000", 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001381415 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883686+0000", 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004368701 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883755+0000", 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9140999999999994e-05 2024-09-10T10:28:23.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883804+0000", 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8772e-05 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 74.375419058000006, 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056063609999999998, 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.282 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878389+0000", 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699999999999999e-07 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878394+0000", 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6120000000000001e-06 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878407+0000", 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3704e-05 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878415+0000", 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7100000000000007e-06 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878475+0000", 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878603+0000", 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127525 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883995+0000", 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053920460000000002 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 74.375368691999995, 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057312819999999999, 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.286 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5000000000000003e-07 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878442+0000", 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.632e-06 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878450+0000", 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0509999999999997e-06 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878456+0000", 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7200000000000003e-06 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878847+0000", 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039182099999999999 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878949+0000", 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101458 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.884170+0000", 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052210499999999996 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.306703373999994, 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033173069999999998, 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947105+0000", 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4670000000000001e-06 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947111+0000", 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6230000000000002e-06 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947123+0000", 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1883e-05 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947132+0000", 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8740000000000001e-06 2024-09-10T10:28:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947220+0000", 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7800000000000006e-05 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947375+0000", 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015566399999999999 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947501+0000", 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125609 2024-09-10T10:28:23.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948393+0000", 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00089165399999999997 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950323+0000", 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019308769999999999 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950366+0000", 2024-09-10T10:28:23.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2688999999999997e-05 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950421+0000", 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5167000000000002e-05 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.306647822000002, 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0070341989999999997, 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.295 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947163+0000", 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3440000000000001e-06 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947174+0000", 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0248000000000001e-05 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947180+0000", 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0639999999999999e-06 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947958+0000", 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077789700000000005 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948031+0000", 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3503000000000004e-05 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948151+0000", 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119976 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953625+0000", 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054739360000000004 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954124+0000", 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049858900000000002 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954157+0000", 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3652999999999998e-05 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954194+0000", 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6455000000000002e-05 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.306555978999995, 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032449549999999999, 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947252+0000", 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000003e-07 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947256+0000", 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3760000000000001e-06 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947267+0000", 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0259e-05 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947273+0000", 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2190000000000001e-06 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947332+0000", 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9086000000000003e-05 2024-09-10T10:28:23.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947452+0000", 2024-09-10T10:28:23.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120146 2024-09-10T10:28:23.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950496+0000", 2024-09-10T10:28:23.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003044171 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:13 2.1 e26/18 2:81735765:::619.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.306145848, 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006029375, 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947662+0000", 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7199999999999997e-07 2024-09-10T10:28:23.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6990000000000003e-06 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947681+0000", 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0682999999999999e-05 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947687+0000", 2024-09-10T10:28:23.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7479999999999996e-06 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947742+0000", 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4487000000000002e-05 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947761+0000", 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9497999999999999e-05 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953653+0000", 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058916530000000002 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953691+0000", 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7734999999999997e-05 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:14 2.1 e26/18 2:81735765:::619.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:23.309 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.304825511000004, 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052525849999999997, 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5700000000000002e-07 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948988+0000", 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7490000000000004e-06 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949003+0000", 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4214e-05 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949010+0000", 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3499999999999999e-06 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949059+0000", 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8758000000000003e-05 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949078+0000", 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9806e-05 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954198+0000", 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051191509999999997 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954234+0000", 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.68e-05 2024-09-10T10:28:23.313 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:1 2.1 e26/18 2:85bbe569:::202.00000000:head v 26'4, mlcod=0'0)", 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.304597146999996, 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005049258, 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949211+0000", 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4999999999999998e-07 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949228+0000", 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7808999999999998e-05 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949240+0000", 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1226000000000001e-05 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949246+0000", 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0689999999999998e-06 2024-09-10T10:28:23.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949292+0000", 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6211999999999997e-05 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949311+0000", 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8873e-05 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954236+0000", 2024-09-10T10:28:23.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049253980000000001 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954259+0000", 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3221e-05 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:26 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.311735065000001, 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039596270000000003, 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.318 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942075+0000", 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2680000000000001e-06 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942086+0000", 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1361e-05 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942098+0000", 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2221e-05 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942106+0000", 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0960000000000001e-06 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942209+0000", 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102944 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942346+0000", 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136976 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942516+0000", 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017015699999999999 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945195+0000", 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026790920000000001 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945919+0000", 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00072374500000000003 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945983+0000", 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3418999999999997e-05 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946032+0000", 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9348000000000003e-05 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:2 2.1 e28/18 2:82971f97:::404.00000000:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.310075908999998, 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054707489999999996, 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943732+0000", 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1999999999999998e-07 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943740+0000", 2024-09-10T10:28:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1910000000000003e-06 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943755+0000", 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5014000000000001e-05 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943763+0000", 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9519999999999994e-06 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5569999999999997e-05 2024-09-10T10:28:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943862+0000", 2024-09-10T10:28:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4150000000000003e-05 2024-09-10T10:28:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949157+0000", 2024-09-10T10:28:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052949340000000003 2024-09-10T10:28:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949202+0000", 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5018000000000002e-05 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:6 2.e e28/18 2:75ee5fd8:::629.00000000:head v 28'5, mlcod=24'2)", 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.310014080999998, 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027501769999999999, 2024-09-10T10:28:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943794+0000", 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943799+0000", 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5620000000000003e-06 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943807+0000", 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.486e-06 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943813+0000", 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6419999999999999e-05 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943888+0000", 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9309000000000001e-05 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946498+0000", 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026094669999999999 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946543+0000", 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-10T10:28:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:17 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'12, mlcod=26'10)", 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.309970319999998, 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027214219999999998, 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943849+0000", 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2023e-05 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943857+0000", 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6009999999999997e-06 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943863+0000", 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2899999999999999e-06 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943927+0000", 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3496000000000003e-05 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943949+0000", 2024-09-10T10:28:23.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1801e-05 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946490+0000", 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025409579999999998 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946558+0000", 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8830000000000003e-05 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:18 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'13, mlcod=26'10)", 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.309924117999998, 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027193059999999999, 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943884+0000", 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943889+0000", 2024-09-10T10:28:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.95e-06 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943896+0000", 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3819999999999998e-06 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943901+0000", 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1719999999999998e-06 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944666+0000", 2024-09-10T10:28:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00076445300000000001 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944686+0000", 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-05 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018748879999999999 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946603+0000", 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1947999999999999e-05 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:25 2.12 e28/18 2:4dad84a4:::mds4_sessionmap:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 57.307010155, 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002978994, 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946798+0000", 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0100000000000001e-06 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946807+0000", 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9700000000000005e-06 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946820+0000", 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2458e-05 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946856+0000", 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6427999999999997e-05 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946938+0000", 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2106000000000003e-05 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946958+0000", 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0162000000000001e-05 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949723+0000", 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027645899999999999 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:23.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:23.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949776+0000", 2024-09-10T10:28:23.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3270000000000003e-05 2024-09-10T10:28:23.346 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.346 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.346 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.346 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:23.346 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:23.346 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:23.347 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:23.348 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:23.347+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:23.445 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:28:23.447 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:23.446+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:23.460 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:23.460 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:23.469 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:23.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.545+0000 7f077e186640 1 Processor -- start 2024-09-10T10:28:23.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.545+0000 7f077e186640 1 -- start start 2024-09-10T10:28:23.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.546+0000 7f077e186640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0778154c10 0x7f077815b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.546+0000 7f077e186640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07780cca70 0x7f0778073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.546+0000 7f077e186640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f077815e0b0 con 0x7f07780cafa0 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.546+0000 7f077e186640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f077815de60 con 0x7f0778154c10 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.546+0000 7f077e186640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f077815e390 con 0x7f07780cca70 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.546+0000 7f0777fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f07780cafa0 0x7f07780cb6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:47446/0 (socket says 172.21.15.120:47446) 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.546+0000 7f07777fe640 1 --2- 172.21.15.120:0/3462872879 >> v2:172.21.15.120:3300/0 conn(0x7f0778154c10 0x7f077815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.546+0000 7f0777fff640 1 -- 172.21.15.120:0/3462872879 learned_addr learned my addr 172.21.15.120:0/3462872879 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.546+0000 7f0776ffd640 1 --2- 172.21.15.120:0/3462872879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07780cca70 0x7f0778073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.547+0000 7f07777fe640 1 -- 172.21.15.120:0/3462872879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07780cca70 msgr2=0x7f0778073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.547+0000 7f07777fe640 1 --2- 172.21.15.120:0/3462872879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07780cca70 0x7f0778073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.547+0000 7f07767fc640 1 -- 172.21.15.120:0/3462872879 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2559095915 0 0) 0x7f077815e0b0 con 0x7f07780cafa0 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.547+0000 7f07777fe640 1 -- 172.21.15.120:0/3462872879 >> v1:172.21.15.120:6789/0 conn(0x7f07780cafa0 legacy=0x7f07780cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.547+0000 7f07777fe640 1 -- 172.21.15.120:0/3462872879 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077815e670 con 0x7f0778154c10 2024-09-10T10:28:23.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.548+0000 7f07777fe640 1 --2- 172.21.15.120:0/3462872879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0778154c10 0x7f077815b370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f07680097a0 tx=0x7f076802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c173af1021bf7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.548+0000 7f07767fc640 1 -- 172.21.15.120:0/3462872879 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768037070 con 0x7f0778154c10 2024-09-10T10:28:23.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.548+0000 7f07767fc640 1 -- 172.21.15.120:0/3462872879 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0768032440 con 0x7f0778154c10 2024-09-10T10:28:23.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.548+0000 7f07767fc640 1 -- 172.21.15.120:0/3462872879 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768032810 con 0x7f0778154c10 2024-09-10T10:28:23.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.549+0000 7f077e186640 1 -- 172.21.15.120:0/3462872879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0778154c10 msgr2=0x7f077815b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.549+0000 7f077e186640 1 --2- 172.21.15.120:0/3462872879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0778154c10 0x7f077815b370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f07680097a0 tx=0x7f076802a180 comp rx=0 tx=0).stop 2024-09-10T10:28:23.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.549+0000 7f077e186640 1 -- 172.21.15.120:0/3462872879 shutdown_connections 2024-09-10T10:28:23.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.549+0000 7f077e186640 1 --2- 172.21.15.120:0/3462872879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07780cca70 0x7f0778073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.549+0000 7f077e186640 1 --2- 172.21.15.120:0/3462872879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0778154c10 0x7f077815b370 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.549+0000 7f077e186640 1 -- 172.21.15.120:0/3462872879 >> 172.21.15.120:0/3462872879 conn(0x7f077806ce80 msgr2=0x7f077806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:23.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.549+0000 7f077e186640 1 -- 172.21.15.120:0/3462872879 shutdown_connections 2024-09-10T10:28:23.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.549+0000 7f077e186640 1 -- 172.21.15.120:0/3462872879 wait complete. 2024-09-10T10:28:23.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.550+0000 7f077e186640 1 Processor -- start 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.550+0000 7f077e186640 1 -- start start 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f077e186640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07780cafa0 0x7f07780c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f077e186640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07780cca70 0x7f07781f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f077e186640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0778154c10 0x7f077814e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f077e186640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0778154ff0 con 0x7f07780cca70 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f077e186640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f077815ec30 con 0x7f0778154c10 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f077e186640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f07780cbe00 con 0x7f07780cafa0 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f0776ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07780cca70 0x7f07781f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f0776ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07780cca70 0x7f07781f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53938/0 (socket says 172.21.15.120:53938) 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f0776ffd640 1 -- 172.21.15.120:0/360777381 learned_addr learned my addr 172.21.15.120:0/360777381 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f0777fff640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0778154c10 0x7f077814e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.551+0000 7f07777fe640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07780cafa0 0x7f07780c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f0776ffd640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07780cafa0 msgr2=0x7f07780c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f0776ffd640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07780cafa0 0x7f07780c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f0776ffd640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0778154c10 msgr2=0x7f077814e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f0776ffd640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0778154c10 0x7f077814e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f0776ffd640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077815d5a0 con 0x7f07780cca70 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f0776ffd640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07780cca70 0x7f07781f38f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f076c00e8d0 tx=0x7f076c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c017070 con 0x7f07780cca70 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077815d300 con 0x7f07780cca70 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f076c00c940 con 0x7f07780cca70 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c017070 con 0x7f07780cca70 2024-09-10T10:28:23.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.552+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0778150c20 con 0x7f07780cca70 2024-09-10T10:28:23.556 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:23.553+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:23.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.553+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f077815b370 con 0x7f07780cca70 2024-09-10T10:28:23.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.554+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f076c01a070 con 0x7f07780cca70 2024-09-10T10:28:23.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.555+0000 7f075ffff640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f076406c060 0x7f076406e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.555+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6983+0+0 (secure 0 0 0) 0x7f076c08b260 con 0x7f07780cca70 2024-09-10T10:28:23.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.555+0000 7f07777fe640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f076406c060 0x7f076406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.555+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v50) ==== 54+0+16 (secure 0 0 0) 0x7f076c08d800 con 0x7f07780cca70 2024-09-10T10:28:23.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.555+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f077815d9b0 0x7f07780cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.556+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0778150a00 con 0x7f077815d9b0 2024-09-10T10:28:23.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.556+0000 7f07777fe640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f076406c060 0x7f076406e520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0768034000 tx=0x7f076802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.556+0000 7f0777fff640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f077815d9b0 0x7f07780cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.556+0000 7f0777fff640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f077815d9b0 0x7f07780cf3f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.557+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0778150a00 con 0x7f077815d9b0 2024-09-10T10:28:23.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.571+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f07780cde00 con 0x7f077815d9b0 2024-09-10T10:28:23.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.572+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f07780cde00 con 0x7f077815d9b0 2024-09-10T10:28:23.573 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:23.574 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:23.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.572+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f07780c8c10 0x7f077815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.572+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f07780c8ff0 con 0x7f07780c8c10 2024-09-10T10:28:23.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.573+0000 7f07777fe640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f07780c8c10 0x7f077815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.573+0000 7f07777fe640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f07780c8c10 0x7f077815ba50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.574+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07780c8ff0 con 0x7f07780c8c10 2024-09-10T10:28:23.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.588+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f077815c260 con 0x7f07780c8c10 2024-09-10T10:28:23.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.591+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f077815c260 con 0x7f07780c8c10 2024-09-10T10:28:23.592 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:23.593 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:23.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.591+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f0778062b40 0x7f0778062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.591+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f077815c830 con 0x7f0778062b40 2024-09-10T10:28:23.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.591+0000 7f0776ffd640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f0778062b40 0x7f0778062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.591+0000 7f0776ffd640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f0778062b40 0x7f0778062f20 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.592+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f077815c830 con 0x7f0778062b40 2024-09-10T10:28:23.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.606+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f07780cdb20 con 0x7f0778062b40 2024-09-10T10:28:23.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.608+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f07780cdb20 con 0x7f0778062b40 2024-09-10T10:28:23.609 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:23.610 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:23.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.608+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f077802cf10 0x7f077802d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.608+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f07780b5830 con 0x7f077802cf10 2024-09-10T10:28:23.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.608+0000 7f0777fff640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f077802cf10 0x7f077802d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.609+0000 7f0777fff640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f077802cf10 0x7f077802d350 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.609+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07780b5830 con 0x7f077802cf10 2024-09-10T10:28:23.614 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:23.614 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:23.614 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:23.614 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:23.614 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:23.623 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:23.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.623+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f077802dae0 con 0x7f077802cf10 2024-09-10T10:28:23.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.624+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f077802dae0 con 0x7f077802cf10 2024-09-10T10:28:23.625 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:23.626 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:23.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.624+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f07780b4630 0x7f07780b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.624+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f07780cd840 con 0x7f07780b4630 2024-09-10T10:28:23.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.625+0000 7f07777fe640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f07780b4630 0x7f07780b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.625+0000 7f07777fe640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f07780b4630 0x7f07780b4a90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.626+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07780cd840 con 0x7f07780b4630 2024-09-10T10:28:23.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.639+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f07780cd840 con 0x7f07780b4630 2024-09-10T10:28:23.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.641+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f07780cd840 con 0x7f07780b4630 2024-09-10T10:28:23.642 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:23.643 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:23.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.642+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f0778204c70 0x7f0778207060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.642+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f07780cd5e0 con 0x7f0778204c70 2024-09-10T10:28:23.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.642+0000 7f0776ffd640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f0778204c70 0x7f0778207060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.642+0000 7f0776ffd640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f0778204c70 0x7f0778207060 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.643+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f076c0780b0 con 0x7f0778204c70 2024-09-10T10:28:23.648 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:23.647+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:23.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.656+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f07780cd870 con 0x7f0778204c70 2024-09-10T10:28:23.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.658+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f07780cd870 con 0x7f0778204c70 2024-09-10T10:28:23.660 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:23.660 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:23.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.658+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f0778207ab0 0x7f0778209ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.658+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f077820cc70 con 0x7f0778207ab0 2024-09-10T10:28:23.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.659+0000 7f0777fff640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f0778207ab0 0x7f0778209ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.659+0000 7f0777fff640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f0778207ab0 0x7f0778209ea0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.660+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f077820cc70 con 0x7f0778207ab0 2024-09-10T10:28:23.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.674+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f077820a3e0 con 0x7f0778207ab0 2024-09-10T10:28:23.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.675+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f077820a3e0 con 0x7f0778207ab0 2024-09-10T10:28:23.676 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:23.677 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:23.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.675+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f077820aa60 0x7f077820ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:23.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.675+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f077820fe30 con 0x7f077820aa60 2024-09-10T10:28:23.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.676+0000 7f07777fe640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f077820aa60 0x7f077820ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:23.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.676+0000 7f07777fe640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f077820aa60 0x7f077820ce50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:23.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.677+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f077820fe30 con 0x7f077820aa60 2024-09-10T10:28:23.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.690+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f077820eb70 con 0x7f077820aa60 2024-09-10T10:28:23.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f075ffff640 1 -- 172.21.15.120:0/360777381 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f077820eb70 con 0x7f077820aa60 2024-09-10T10:28:23.693 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f077815d9b0 msgr2=0x7f07780cf3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f077815d9b0 0x7f07780cf3f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f07780c8c10 msgr2=0x7f077815ba50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f07780c8c10 0x7f077815ba50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f0778062b40 msgr2=0x7f0778062f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f0778062b40 0x7f0778062f20 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f077802cf10 msgr2=0x7f077802d350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f077802cf10 0x7f077802d350 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f07780b4630 msgr2=0x7f07780b4a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f07780b4630 0x7f07780b4a90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f0778204c70 msgr2=0x7f0778207060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f0778204c70 0x7f0778207060 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f0778207ab0 msgr2=0x7f0778209ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f0778207ab0 0x7f0778209ea0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f077820aa60 msgr2=0x7f077820ce50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f077820aa60 0x7f077820ce50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f0777fff640 1 -- 172.21.15.120:0/360777381 reap_dead start 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f076406c060 msgr2=0x7f076406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f076406c060 0x7f076406e520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0768034000 tx=0x7f076802e000 comp rx=0 tx=0).stop 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f0777fff640 1 -- 172.21.15.120:0/360777381 reap_dead start 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07780cca70 msgr2=0x7f07781f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 --2- 172.21.15.120:0/360777381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07780cca70 0x7f07781f38f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f076c00e8d0 tx=0x7f076c00ed90 comp rx=0 tx=0).stop 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f0777fff640 1 -- 172.21.15.120:0/360777381 reap_dead start 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f0777fff640 1 -- 172.21.15.120:0/360777381 reap_dead start 2024-09-10T10:28:23.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f0777fff640 1 -- 172.21.15.120:0/360777381 reap_dead start 2024-09-10T10:28:23.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 shutdown_connections 2024-09-10T10:28:23.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.692+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 >> 172.21.15.120:0/360777381 conn(0x7f077806ce80 msgr2=0x7f0778073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:23.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.693+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 shutdown_connections 2024-09-10T10:28:23.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:23.693+0000 7f077e186640 1 -- 172.21.15.120:0/360777381 wait complete. 2024-09-10T10:28:23.749 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:23.749+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:23.766 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:23.766 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:23.775 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:23.850 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:23.849+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:23.878 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:23.879 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:23.887 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:23.951 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:23.950+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:23.991 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:23.991 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:24.000 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:24.067 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:24.066+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:24.127 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:24.127 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:24.136 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:24.177 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:24.176+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:24.254 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:24.253+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:24.286 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:24.286 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:24.295 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:24.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-10T10:28:24.354 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:24.353+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:24.400 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:24.400 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:24.409 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:24.455 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:24.454+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:24.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.462+0000 7fcedd827640 1 Processor -- start 2024-09-10T10:28:24.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.462+0000 7fcedd827640 1 -- start start 2024-09-10T10:28:24.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.463+0000 7fcedd827640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fced80cc0c0 0x7fced815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:24.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.463+0000 7fcedd827640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fced80cc4a0 0x7fced8165430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:24.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.463+0000 7fcedd827640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced815b1d0 con 0x7fced80cb2f0 2024-09-10T10:28:24.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.463+0000 7fcedd827640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fced80cbd00 con 0x7fced80cc0c0 2024-09-10T10:28:24.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.463+0000 7fcedd827640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fced815ba70 con 0x7fced80cc4a0 2024-09-10T10:28:24.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.464+0000 7fced6ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fced80cc0c0 0x7fced815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:24.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.464+0000 7fced67fc640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fced80cb2f0 0x7fced815e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:47450/0 (socket says 172.21.15.120:47450) 2024-09-10T10:28:24.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.464+0000 7fced67fc640 1 -- 172.21.15.120:0/3499122787 learned_addr learned my addr 172.21.15.120:0/3499122787 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:24.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.464+0000 7fced77fe640 1 --2- 172.21.15.120:0/3499122787 >> v2:172.21.15.134:3300/0 conn(0x7fced80cc4a0 0x7fced8165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.464+0000 7fced6ffd640 1 -- 172.21.15.120:0/3499122787 >> v2:172.21.15.134:3300/0 conn(0x7fced80cc4a0 msgr2=0x7fced8165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.464+0000 7fced6ffd640 1 --2- 172.21.15.120:0/3499122787 >> v2:172.21.15.134:3300/0 conn(0x7fced80cc4a0 0x7fced8165430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.464+0000 7fced6ffd640 1 -- 172.21.15.120:0/3499122787 >> v1:172.21.15.120:6789/0 conn(0x7fced80cb2f0 legacy=0x7fced815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.464+0000 7fced6ffd640 1 -- 172.21.15.120:0/3499122787 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced815b4b0 con 0x7fced80cc0c0 2024-09-10T10:28:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.464+0000 7fced5ffb640 1 -- 172.21.15.120:0/3499122787 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2082367983 0 0) 0x7fced815b1d0 con 0x7fced80cb2f0 2024-09-10T10:28:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.465+0000 7fced6ffd640 1 --2- 172.21.15.120:0/3499122787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fced80cc0c0 0x7fced815aab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fced806e100 tx=0x7fcec802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3433290b4509c94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.465+0000 7fced5ffb640 1 -- 172.21.15.120:0/3499122787 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8037070 con 0x7fced80cc0c0 2024-09-10T10:28:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.465+0000 7fced5ffb640 1 -- 172.21.15.120:0/3499122787 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec8032440 con 0x7fced80cc0c0 2024-09-10T10:28:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.465+0000 7fced5ffb640 1 -- 172.21.15.120:0/3499122787 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8037070 con 0x7fced80cc0c0 2024-09-10T10:28:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.466+0000 7fcedd827640 1 -- 172.21.15.120:0/3499122787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fced80cc0c0 msgr2=0x7fced815aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.466+0000 7fcedd827640 1 --2- 172.21.15.120:0/3499122787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fced80cc0c0 0x7fced815aab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fced806e100 tx=0x7fcec802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:28:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.466+0000 7fcedd827640 1 -- 172.21.15.120:0/3499122787 shutdown_connections 2024-09-10T10:28:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.466+0000 7fcedd827640 1 --2- 172.21.15.120:0/3499122787 >> v2:172.21.15.134:3300/0 conn(0x7fced80cc4a0 0x7fced8165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.466+0000 7fcedd827640 1 --2- 172.21.15.120:0/3499122787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fced80cc0c0 0x7fced815aab0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.466+0000 7fcedd827640 1 -- 172.21.15.120:0/3499122787 >> 172.21.15.120:0/3499122787 conn(0x7fced806d560 msgr2=0x7fced806d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.466+0000 7fcedd827640 1 -- 172.21.15.120:0/3499122787 shutdown_connections 2024-09-10T10:28:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.466+0000 7fcedd827640 1 -- 172.21.15.120:0/3499122787 wait complete. 2024-09-10T10:28:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.468+0000 7fcedd827640 1 Processor -- start 2024-09-10T10:28:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.468+0000 7fcedd827640 1 -- start start 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fcedd827640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced80cb2f0 0x7fced814d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fcedd827640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fced80cc0c0 0x7fced80c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fcedd827640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fced80cc4a0 0x7fced80ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fcedd827640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fced815e1d0 con 0x7fced80cc0c0 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fcedd827640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fced815e540 con 0x7fced80cb2f0 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fcedd827640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fced80cc8b0 con 0x7fced80cc4a0 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fced67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fced80cc0c0 0x7fced80c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fced6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced80cb2f0 0x7fced814d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fced6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced80cb2f0 0x7fced814d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:50258/0 (socket says 172.21.15.120:50258) 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fced6ffd640 1 -- 172.21.15.120:0/3445211589 learned_addr learned my addr 172.21.15.120:0/3445211589 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:24.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.469+0000 7fced77fe640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fced80cc4a0 0x7fced80ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fced6ffd640 1 -- 172.21.15.120:0/3445211589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fced80cc4a0 msgr2=0x7fced80ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fced6ffd640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fced80cc4a0 0x7fced80ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fced6ffd640 1 -- 172.21.15.120:0/3445211589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fced80cc0c0 msgr2=0x7fced80c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fced6ffd640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fced80cc0c0 0x7fced80c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fced6ffd640 1 -- 172.21.15.120:0/3445211589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced81f1d10 con 0x7fced80cb2f0 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fced6ffd640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced80cb2f0 0x7fced814d530 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcec802a790 tx=0x7fcec80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fcebffff640 1 -- 172.21.15.120:0/3445211589 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8037050 con 0x7fced80cb2f0 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fcebffff640 1 -- 172.21.15.120:0/3445211589 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec8043070 con 0x7fced80cb2f0 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fcedd827640 1 -- 172.21.15.120:0/3445211589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced814e410 con 0x7fced80cb2f0 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fcebffff640 1 -- 172.21.15.120:0/3445211589 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8006e10 con 0x7fced80cb2f0 2024-09-10T10:28:24.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.470+0000 7fcedd827640 1 -- 172.21.15.120:0/3445211589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced814e9d0 con 0x7fced80cb2f0 2024-09-10T10:28:24.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.472+0000 7fcebffff640 1 -- 172.21.15.120:0/3445211589 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcec8032830 con 0x7fced80cb2f0 2024-09-10T10:28:24.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.472+0000 7fcedd827640 1 -- 172.21.15.120:0/3445211589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced814f050 con 0x7fced80cb2f0 2024-09-10T10:28:24.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.472+0000 7fcebffff640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcec406be30 0x7fcec406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:24.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.472+0000 7fcebffff640 1 -- 172.21.15.120:0/3445211589 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6983+0+0 (secure 0 0 0) 0x7fcec80abc60 con 0x7fced80cb2f0 2024-09-10T10:28:24.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.472+0000 7fced67fc640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcec406be30 0x7fcec406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:24.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.473+0000 7fced67fc640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcec406be30 0x7fcec406e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fced80caf30 tx=0x7fcecc007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:24.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.477+0000 7fcebffff640 1 -- 172.21.15.120:0/3445211589 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fced814f050 con 0x7fced80cb2f0 2024-09-10T10:28:24.513 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:24.513 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:24.522 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:24.563 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:24.562+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:24.657 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:24.656+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:24.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:24.661+0000 7fcedd827640 1 -- 172.21.15.120:0/3445211589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fced80c5560 con 0x7fced80cb2f0 2024-09-10T10:28:24.665 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:24.665 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:24.674 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:24.758 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:24.757+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:24.777 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:24.777 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:24.786 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:24.859 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:24.858+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:24.930 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:24.930 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:24.939 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:24.959 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:24.959+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:25.061 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:25.060+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:25.061 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:25.061 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:25.074 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:25.162 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:25.161+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:25.237 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:25.237 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:25.250 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:25.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.248+0000 7fcebffff640 1 -- 172.21.15.120:0/3445211589 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v52) ==== 87+0+0 (secure 0 0 0) 0x7fcec80044a0 con 0x7fced80cb2f0 2024-09-10T10:28:25.251 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-10T10:28:25.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 -- 172.21.15.120:0/3445211589 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcec406be30 msgr2=0x7fcec406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:25.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcec406be30 0x7fcec406e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fced80caf30 tx=0x7fcecc007de0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 -- 172.21.15.120:0/3445211589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced80cb2f0 msgr2=0x7fced814d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:25.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced80cb2f0 0x7fced814d530 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcec802a790 tx=0x7fcec80046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 -- 172.21.15.120:0/3445211589 shutdown_connections 2024-09-10T10:28:25.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcec406be30 0x7fcec406e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fced80cc4a0 0x7fced80ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fced80cc0c0 0x7fced80c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 --2- 172.21.15.120:0/3445211589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced80cb2f0 0x7fced814d530 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 -- 172.21.15.120:0/3445211589 >> 172.21.15.120:0/3445211589 conn(0x7fced806d560 msgr2=0x7fced80d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:25.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 -- 172.21.15.120:0/3445211589 shutdown_connections 2024-09-10T10:28:25.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.251+0000 7fcedd827640 1 -- 172.21.15.120:0/3445211589 wait complete. 2024-09-10T10:28:25.263 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:25.262+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:25.364 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:25.363+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:25.418 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:25.419 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:25.431 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:25.466 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:25.464+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:25.566 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:25.566+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:25.636 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:25.637 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:25.649 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:25.690 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:25.689+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:25.708 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:28:25.769 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:25.768+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:25.804 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:25.805 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:25.814 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:25.870 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:25.869+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.879+0000 7f75a210a640 1 Processor -- start 2024-09-10T10:28:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.879+0000 7f75a210a640 1 -- start start 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.880+0000 7f75a210a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f759c0cd2c0 0x7f759c154a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.880+0000 7f75a210a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f759c0c7280 0x7f759c154f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f75a210a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f759c0c4580 con 0x7f759c0cd6a0 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f75a210a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f759c06cee0 con 0x7f759c0c7280 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f759bfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f759c0c7280 0x7f759c154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f759bfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f759c0c7280 0x7f759c154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53974/0 (socket says 172.21.15.120:53974) 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f759bfff640 1 -- 172.21.15.120:0/1087704497 learned_addr learned my addr 172.21.15.120:0/1087704497 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f75a210a640 1 -- 172.21.15.120:0/1087704497 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f759c0c47f0 con 0x7f759c0cd2c0 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f759b7fe640 1 --2- 172.21.15.120:0/1087704497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f759c0cd2c0 0x7f759c154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f759bfff640 1 -- 172.21.15.120:0/1087704497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f759c0cd2c0 msgr2=0x7f759c154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f759bfff640 1 --2- 172.21.15.120:0/1087704497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f759c0cd2c0 0x7f759c154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f759bfff640 1 -- 172.21.15.120:0/1087704497 >> v1:172.21.15.120:6789/0 conn(0x7f759c0cd6a0 legacy=0x7f759c1578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f759bfff640 1 -- 172.21.15.120:0/1087704497 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c06d570 con 0x7f759c0c7280 2024-09-10T10:28:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.881+0000 7f759a7fc640 1 -- 172.21.15.120:0/1087704497 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4176200071 0 0) 0x7f759c0c4580 con 0x7f759c0cd6a0 2024-09-10T10:28:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.882+0000 7f759bfff640 1 --2- 172.21.15.120:0/1087704497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f759c0c7280 0x7f759c154f80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f759400f370 tx=0x7f759402eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559ceb900be8260a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.882+0000 7f759a7fc640 1 -- 172.21.15.120:0/1087704497 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594036070 con 0x7f759c0c7280 2024-09-10T10:28:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.882+0000 7f759a7fc640 1 -- 172.21.15.120:0/1087704497 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f759400d4e0 con 0x7f759c0c7280 2024-09-10T10:28:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.882+0000 7f759a7fc640 1 -- 172.21.15.120:0/1087704497 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759400ba20 con 0x7f759c0c7280 2024-09-10T10:28:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.883+0000 7f75a210a640 1 -- 172.21.15.120:0/1087704497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f759c0c7280 msgr2=0x7f759c154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.883+0000 7f75a210a640 1 --2- 172.21.15.120:0/1087704497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f759c0c7280 0x7f759c154f80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f759400f370 tx=0x7f759402eb20 comp rx=0 tx=0).stop 2024-09-10T10:28:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.883+0000 7f75a210a640 1 -- 172.21.15.120:0/1087704497 shutdown_connections 2024-09-10T10:28:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.884+0000 7f75a210a640 1 --2- 172.21.15.120:0/1087704497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f759c0c7280 0x7f759c154f80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.884+0000 7f75a210a640 1 --2- 172.21.15.120:0/1087704497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f759c0cd2c0 0x7f759c154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.884+0000 7f75a210a640 1 -- 172.21.15.120:0/1087704497 >> 172.21.15.120:0/1087704497 conn(0x7f759c067950 msgr2=0x7f759c067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:25.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.884+0000 7f75a210a640 1 -- 172.21.15.120:0/1087704497 shutdown_connections 2024-09-10T10:28:25.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.884+0000 7f75a210a640 1 -- 172.21.15.120:0/1087704497 wait complete. 2024-09-10T10:28:25.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.885+0000 7f75a210a640 1 Processor -- start 2024-09-10T10:28:25.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.886+0000 7f75a210a640 1 -- start start 2024-09-10T10:28:25.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.887+0000 7f75a210a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f759c0c7280 0x7f759c0ba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:25.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.887+0000 7f75a210a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f759c0cd2c0 0x7f759c154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:25.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.888+0000 7f759affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f759c0cd2c0 0x7f759c154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:25.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.888+0000 7f759b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f759c0c7280 0x7f759c0ba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:25.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.888+0000 7f759b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f759c0c7280 0x7f759c0ba5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:50270/0 (socket says 172.21.15.120:50270) 2024-09-10T10:28:25.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.888+0000 7f759affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f759c0cd2c0 0x7f759c154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53984/0 (socket says 172.21.15.120:53984) 2024-09-10T10:28:25.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.888+0000 7f759b7fe640 1 -- 172.21.15.120:0/1416932754 learned_addr learned my addr 172.21.15.120:0/1416932754 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:25.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.888+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f759c0cd6a0 0x7f759c150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:25.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.888+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f759c0735e0 con 0x7f759c0cd2c0 2024-09-10T10:28:25.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.888+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f759c06d110 con 0x7f759c0c7280 2024-09-10T10:28:25.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.888+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f759c0c7940 con 0x7f759c0cd6a0 2024-09-10T10:28:25.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.888+0000 7f759bfff640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f759c0cd6a0 0x7f759c150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.889+0000 7f759affd640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f759c0cd6a0 msgr2=0x7f759c150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.889+0000 7f759affd640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f759c0cd6a0 0x7f759c150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.889+0000 7f759affd640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f759c0c7280 msgr2=0x7f759c0ba5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.889+0000 7f759affd640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f759c0c7280 0x7f759c0ba5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.889+0000 7f759affd640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c0c1c00 con 0x7f759c0cd2c0 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.889+0000 7f759bfff640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f759c0cd6a0 0x7f759c150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.889+0000 7f759affd640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f759c0cd2c0 0x7f759c154a80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f759c073570 tx=0x7f759000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.890+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590005070 con 0x7f759c0cd2c0 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.890+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c151d90 con 0x7f759c0cd2c0 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.890+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7590005210 con 0x7f759c0cd2c0 2024-09-10T10:28:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.890+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759000a640 con 0x7f759c0cd2c0 2024-09-10T10:28:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.891+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c0c2150 con 0x7f759c0cd2c0 2024-09-10T10:28:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.892+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f759c0c4f50 con 0x7f759c0cd2c0 2024-09-10T10:28:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.892+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f759000a800 con 0x7f759c0cd2c0 2024-09-10T10:28:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.893+0000 7f7583fff640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f758806be30 0x7f758806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.893+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6983+0+0 (secure 0 0 0) 0x7f759008aa70 con 0x7f759c0cd2c0 2024-09-10T10:28:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.893+0000 7f759b7fe640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f758806be30 0x7f758806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.893+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v52) ==== 54+0+16 (secure 0 0 0) 0x7f759c0c4f50 con 0x7f759c0cd2c0 2024-09-10T10:28:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.894+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f759c0c2390 0x7f759c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.894+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f759c0c3b40 con 0x7f759c0c2390 2024-09-10T10:28:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.894+0000 7f759bfff640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f759c0c2390 0x7f759c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.894+0000 7f759b7fe640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f758806be30 0x7f758806e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f758c007af0 tx=0x7f758c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.894+0000 7f759bfff640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f759c0c2390 0x7f759c153b20 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.895+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f759c0c3b40 con 0x7f759c0c2390 2024-09-10T10:28:25.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.925+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f759c0680b0 con 0x7f759c0c2390 2024-09-10T10:28:25.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.927+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f759c0680b0 con 0x7f759c0c2390 2024-09-10T10:28:25.928 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:25.929 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:25.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.928+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f759c0b9fd0 0x7f759c1efa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:25.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.928+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f759c152340 con 0x7f759c0b9fd0 2024-09-10T10:28:25.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.928+0000 7f759b7fe640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f759c0b9fd0 0x7f759c1efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:25.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.928+0000 7f759b7fe640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f759c0b9fd0 0x7f759c1efa70 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:25.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.929+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f759c152340 con 0x7f759c0b9fd0 2024-09-10T10:28:25.940 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:25.940 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:25.950 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.957+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f759c0bab30 con 0x7f759c0b9fd0 2024-09-10T10:28:25.961 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.959+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f759c0bab30 con 0x7f759c0b9fd0 2024-09-10T10:28:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.960+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f759c1518a0 0x7f759c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.960+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f759c0c2950 con 0x7f759c1518a0 2024-09-10T10:28:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.960+0000 7f759affd640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f759c1518a0 0x7f759c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.960+0000 7f759affd640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f759c1518a0 0x7f759c062630 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:25.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.961+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f759c0c2950 con 0x7f759c1518a0 2024-09-10T10:28:25.971 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:25.970+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.979+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f759c062b70 con 0x7f759c1518a0 2024-09-10T10:28:25.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.982+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f759c062b70 con 0x7f759c1518a0 2024-09-10T10:28:25.983 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:25.984 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:25.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.982+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f759c0c7f90 0x7f759c0c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:25.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.982+0000 7f759bfff640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f759c0c7f90 0x7f759c0c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:25.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.982+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f759c0c88d0 con 0x7f759c0c7f90 2024-09-10T10:28:25.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.983+0000 7f759bfff640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f759c0c7f90 0x7f759c0c8390 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:25.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:25.983+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f759c0c88d0 con 0x7f759c0c7f90 2024-09-10T10:28:26.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.010+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f759c0c8d50 con 0x7f759c0c7f90 2024-09-10T10:28:26.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.013+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f759c0c8d50 con 0x7f759c0c7f90 2024-09-10T10:28:26.015 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:26.015 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:26.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.014+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f759c02cb50 0x7f759c02cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:26.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.014+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f759c0c2b00 con 0x7f759c02cb50 2024-09-10T10:28:26.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.014+0000 7f759b7fe640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f759c02cb50 0x7f759c02cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:26.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.015+0000 7f759b7fe640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f759c02cb50 0x7f759c02cfb0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:26.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.015+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f759c0c2b00 con 0x7f759c02cb50 2024-09-10T10:28:26.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.043+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f759c0c2b00 con 0x7f759c02cb50 2024-09-10T10:28:26.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.045+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f759c0c2b00 con 0x7f759c02cb50 2024-09-10T10:28:26.047 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:26.047 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:26.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.046+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f759c2008e0 0x7f759c202cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:26.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.046+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f759c0c8e40 con 0x7f759c2008e0 2024-09-10T10:28:26.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.046+0000 7f759affd640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f759c2008e0 0x7f759c202cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:26.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.047+0000 7f759affd640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f759c2008e0 0x7f759c202cd0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:26.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.047+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f759c0c8e40 con 0x7f759c2008e0 2024-09-10T10:28:26.071 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:26.071+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:26.075 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:26.075 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:26.075 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:26.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.074+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f759c0c8e40 con 0x7f759c2008e0 2024-09-10T10:28:26.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.075+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f759c0c8e40 con 0x7f759c2008e0 2024-09-10T10:28:26.077 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:26.077 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:26.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.076+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f759c203720 0x7f759c205b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:26.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.076+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f759c02db30 con 0x7f759c203720 2024-09-10T10:28:26.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.076+0000 7f759bfff640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f759c203720 0x7f759c205b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:26.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.076+0000 7f759bfff640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f759c203720 0x7f759c205b10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:26.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.077+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f759c02db30 con 0x7f759c203720 2024-09-10T10:28:26.084 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:26.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.104+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f759c02db30 con 0x7f759c203720 2024-09-10T10:28:26.107 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:26.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.106+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f759c02db30 con 0x7f759c203720 2024-09-10T10:28:26.108 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:26.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.106+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f759c206560 0x7f759c2069a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:26.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.106+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f759c20baa0 con 0x7f759c206560 2024-09-10T10:28:26.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.106+0000 7f759b7fe640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f759c206560 0x7f759c2069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:26.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.107+0000 7f759b7fe640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f759c206560 0x7f759c2069a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:26.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.108+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f759c20baa0 con 0x7f759c206560 2024-09-10T10:28:26.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.134+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f759c206f50 con 0x7f759c206560 2024-09-10T10:28:26.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.136+0000 7f7583fff640 1 -- 172.21.15.120:0/1416932754 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f759c206f50 con 0x7f759c206560 2024-09-10T10:28:26.138 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f759c0c2390 msgr2=0x7f759c153b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f759c0c2390 0x7f759c153b20 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f759c0b9fd0 msgr2=0x7f759c1efa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f759c0b9fd0 0x7f759c1efa70 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f759c1518a0 msgr2=0x7f759c062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f759c1518a0 0x7f759c062630 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f759bfff640 1 -- 172.21.15.120:0/1416932754 reap_dead start 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f759c0c7f90 msgr2=0x7f759c0c8390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f759c0c7f90 0x7f759c0c8390 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f759c02cb50 msgr2=0x7f759c02cfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f759c02cb50 0x7f759c02cfb0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:26.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f759c2008e0 msgr2=0x7f759c202cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f759c2008e0 0x7f759c202cd0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f759c203720 msgr2=0x7f759c205b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f759c203720 0x7f759c205b10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f759c206560 msgr2=0x7f759c2069a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f759c206560 0x7f759c2069a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f758806be30 msgr2=0x7f758806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f758806be30 0x7f758806e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f758c007af0 tx=0x7f758c008040 comp rx=0 tx=0).stop 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f759c0cd2c0 msgr2=0x7f759c154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 --2- 172.21.15.120:0/1416932754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f759c0cd2c0 0x7f759c154a80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f759c073570 tx=0x7f759000cbe0 comp rx=0 tx=0).stop 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f759bfff640 1 -- 172.21.15.120:0/1416932754 reap_dead start 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f759bfff640 1 -- 172.21.15.120:0/1416932754 reap_dead start 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 shutdown_connections 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.137+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 >> 172.21.15.120:0/1416932754 conn(0x7f759c067950 msgr2=0x7f759c157560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:26.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.138+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 shutdown_connections 2024-09-10T10:28:26.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:26.138+0000 7f75a210a640 1 -- 172.21.15.120:0/1416932754 wait complete. 2024-09-10T10:28:26.172 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:26.171+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:26.211 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:26.211 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:26.212 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:26.212 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:26.212 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:26.220 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:26.273 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:26.272+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:26.332 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:26.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.549553646999996, 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078690670000000004, 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2200000000000004e-07 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780048+0000", 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2427999999999999e-05 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780057+0000", 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1090000000000004e-06 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780063+0000", 2024-09-10T10:28:26.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9780000000000002e-06 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011213480000000001 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781281+0000", 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6728e-05 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781382+0000", 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101455 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784820+0000", 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034376200000000002 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787779+0000", 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002959373 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787852+0000", 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3135999999999999e-05 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787894+0000", 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1369999999999999e-05 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.549506105999995, 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079289789999999992, 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780073+0000", 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780075+0000", 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5129999999999999e-06 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780082+0000", 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9410000000000001e-06 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780087+0000", 2024-09-10T10:28:26.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1880000000000001e-06 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781620+0000", 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001532346 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781698+0000", 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8232999999999994e-05 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781816+0000", 2024-09-10T10:28:26.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011825800000000001 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784824+0000", 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030076349999999998 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787937+0000", 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031134829999999998 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787969+0000", 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1745999999999997e-05 2024-09-10T10:28:26.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788001+0000", 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2132000000000003e-05 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.549475891, 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079888619999999994, 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.344 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780103+0000", 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780106+0000", 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1379999999999999e-06 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780115+0000", 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4299999999999995e-06 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780122+0000", 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3380000000000001e-06 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782041+0000", 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019193700000000001 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782149+0000", 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107493 2024-09-10T10:28:26.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782246+0000", 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999996e-05 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785205+0000", 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002958535 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788041+0000", 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028359650000000002 2024-09-10T10:28:26.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788069+0000", 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8123000000000002e-05 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788091+0000", 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2506000000000001e-05 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.549447349999994, 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080531070000000003, 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780134+0000", 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0570000000000001e-06 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780143+0000", 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-06 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780156+0000", 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3179e-05 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023278280000000001 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782555+0000", 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0494000000000006e-05 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782675+0000", 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120463 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785209+0000", 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002533455 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788127+0000", 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029181260000000001 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788155+0000", 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8495000000000001e-05 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788184+0000", 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8736999999999999e-05 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.354 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.549414833, 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008097524, 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:26.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780164+0000", 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.13e-07 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9780000000000001e-06 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780174+0000", 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7510000000000002e-06 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780179+0000", 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0869999999999997e-06 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782890+0000", 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027113509999999999 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782985+0000", 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5152e-05 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783090+0000", 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104947 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785222+0000", 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021320269999999999 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788216+0000", 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029940629999999999 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788242+0000", 2024-09-10T10:28:26.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5259999999999999e-05 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788261+0000", 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9395e-05 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:26.360 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 99.549392462, 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081619529999999996, 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780189+0000", 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.926e-06 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780197+0000", 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2039999999999997e-06 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780202+0000", 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783326+0000", 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003123855 2024-09-10T10:28:26.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783402+0000", 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6548e-05 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783558+0000", 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015539199999999999 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785225+0000", 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001666989 2024-09-10T10:28:26.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788290+0000", 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030647890000000001 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788316+0000", 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6662e-05 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788348+0000", 2024-09-10T10:28:26.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1417000000000001e-05 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:3 2.10 2:0f65c9b8:::60a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.457798949999997, 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066829990000000002, 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871781+0000", 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2129999999999998e-06 2024-09-10T10:28:26.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871790+0000", 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3380000000000005e-06 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871805+0000", 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5353999999999999e-05 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871817+0000", 2024-09-10T10:28:26.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1399999999999999e-05 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871932+0000", 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114982 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872069+0000", 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013713199999999999 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872208+0000", 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013976900000000001 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874946+0000", 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027378789999999999 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878355+0000", 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034089139999999999 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878413+0000", 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7540000000000001e-05 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878462+0000", 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9478000000000003e-05 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.382355817999994, 2024-09-10T10:28:26.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067332750000000004, 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947223+0000", 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7299999999999995e-07 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947227+0000", 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.895e-06 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947235+0000", 2024-09-10T10:28:26.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947246+0000", 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0738e-05 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948505+0000", 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001259402 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948578+0000", 2024-09-10T10:28:26.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2788999999999997e-05 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948718+0000", 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013953699999999999 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953391+0000", 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046736670000000003 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953881+0000", 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048956900000000003 2024-09-10T10:28:26.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953923+0000", 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2153000000000001e-05 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953956+0000", 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2526e-05 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.382297030000004, 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066654160000000004, 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:26.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947282+0000", 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947285+0000", 2024-09-10T10:28:26.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7689999999999998e-06 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947292+0000", 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6200000000000001e-06 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947304+0000", 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1702000000000001e-05 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947363+0000", 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9426000000000003e-05 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947485+0000", 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012227599999999999 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947588+0000", 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102641 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950690+0000", 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003102264 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953856+0000", 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031656200000000001 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953907+0000", 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1430000000000001e-05 2024-09-10T10:28:26.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953947+0000", 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9164e-05 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.382259856999994, 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071148679999999999, 2024-09-10T10:28:26.382 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947319+0000", 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947322+0000", 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.523e-06 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947331+0000", 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3240000000000003e-06 2024-09-10T10:28:26.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947336+0000", 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7189999999999998e-06 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947994+0000", 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065770599999999998 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948080+0000", 2024-09-10T10:28:26.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5649000000000006e-05 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111783 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953965+0000", 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057732089999999996 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954343+0000", 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037846199999999998 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954395+0000", 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.126e-05 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954433+0000", 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8693999999999999e-05 2024-09-10T10:28:26.387 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.382129660999993, 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067554449999999997, 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:26.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6899999999999996e-07 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947454+0000", 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947463+0000", 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0310000000000004e-06 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947477+0000", 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4328000000000001e-05 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3869999999999998e-05 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947631+0000", 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9874e-05 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947750+0000", 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118483 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953644+0000", 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058946889999999998 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954111+0000", 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00046712099999999998 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954162+0000", 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000003e-05 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954204+0000", 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1718000000000003e-05 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:26.393 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.381742622999994, 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064551169999999998, 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3e-07 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947841+0000", 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3460000000000004e-06 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947856+0000", 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-09-10T10:28:26.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947864+0000", 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4939999999999997e-06 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947997+0000", 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013305399999999999 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948076+0000", 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9146000000000004e-05 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115586 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953707+0000", 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055153470000000003 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954215+0000", 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050850400000000001 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954258+0000", 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2932000000000002e-05 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954291+0000", 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2322e-05 2024-09-10T10:28:26.398 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.381703302999995, 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088919089999999999, 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:26.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7199999999999999e-07 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947879+0000", 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8380000000000004e-06 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947887+0000", 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5410000000000003e-06 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947893+0000", 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6930000000000004e-06 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948405+0000", 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051236000000000001 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948502+0000", 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6916999999999994e-05 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948647+0000", 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000145265 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954479+0000", 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058316970000000003 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956680+0000", 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002201494 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956728+0000", 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8161999999999997e-05 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956767+0000", 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.837e-05 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.404 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.381140642000005, 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083279530000000008, 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8999999999999995e-07 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948443+0000", 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2189999999999998e-06 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948450+0000", 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8879999999999997e-06 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948457+0000", 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0750000000000003e-06 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948501+0000", 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948594+0000", 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3009000000000003e-05 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948696+0000", 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101461 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953756+0000", 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050607129999999997 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956690+0000", 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029334109999999999 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956737+0000", 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7080999999999999e-05 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956766+0000", 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8778000000000001e-05 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.409 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.380600967999996, 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0077788589999999999, 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:26.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948986+0000", 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:26.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948998+0000", 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.19e-05 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949007+0000", 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.081e-06 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949062+0000", 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754999999999997e-05 2024-09-10T10:28:26.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949172+0000", 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110128 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949282+0000", 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109289 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955941+0000", 2024-09-10T10:28:26.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066594990000000001 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956654+0000", 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071279399999999999 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956706+0000", 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2027000000000002e-05 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956756+0000", 2024-09-10T10:28:26.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0012999999999997e-05 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.380062874000004, 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073200349999999999, 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949516+0000", 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999995e-07 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949522+0000", 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3980000000000004e-06 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949534+0000", 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1562e-05 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949541+0000", 2024-09-10T10:28:26.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1380000000000001e-06 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950700+0000", 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011593459999999999 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950786+0000", 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5556999999999994e-05 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950898+0000", 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111743 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954326+0000", 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034280510000000001 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956774+0000", 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024477430000000001 2024-09-10T10:28:26.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956815+0000", 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1752999999999998e-05 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956835+0000", 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0032999999999999e-05 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.420 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.380020572999996, 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073303700000000001, 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949571+0000", 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2298e-05 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949579+0000", 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5909999999999996e-06 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949585+0000", 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7370000000000003e-06 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953802+0000", 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042170929999999999 2024-09-10T10:28:26.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953915+0000", 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011256699999999999 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956888+0000", 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029734639999999999 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.424 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:24 2.3 e26/18 2:c216c0d1:::mds2_inotable:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.379202571999997, 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0063254399999999999, 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950380+0000", 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2590000000000002e-06 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950387+0000", 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950392+0000", 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950438+0000", 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5592e-05 2024-09-10T10:28:26.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950457+0000", 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8569000000000001e-05 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956657+0000", 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062003409999999998 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956701+0000", 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4140000000000001e-05 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:29 2.10 2:0e4da237:::203.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 66.407541769000005, 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064228799999999997, 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922037+0000", 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922040+0000", 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.694e-06 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922054+0000", 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906e-05 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922065+0000", 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922125+0000", 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.982e-05 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922233+0000", 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108175 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928459+0000", 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062264570000000003 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.388278139000001, 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081214100000000008, 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941301+0000", 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.062e-06 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941306+0000", 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0919999999999997e-06 2024-09-10T10:28:26.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941316+0000", 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5640000000000002e-06 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941323+0000", 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941405+0000", 2024-09-10T10:28:26.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1650000000000006e-05 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941549+0000", 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014447599999999999 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012929 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943549+0000", 2024-09-10T10:28:26.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018704049999999999 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949343+0000", 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057943300000000003 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949378+0000", 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4474999999999998e-05 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949421+0000", 2024-09-10T10:28:26.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3955999999999997e-05 2024-09-10T10:28:26.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.438 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.438 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.438 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:26.439 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:26.440 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:26.439+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:26.540 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:26.539+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:26.592 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:26.593 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:26.593 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:26.593 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:26.605 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:26.646 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:26.645+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:26.743 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:26.743+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:26.754 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:26.754 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:26.754 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:26.763 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:26.844 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:26.843+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:26.869 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:26.870 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:26.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:26.870 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:26.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.870 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:26.870 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:26.870 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.087581394, 2024-09-10T10:28:26.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054521709999999996, 2024-09-10T10:28:26.870 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.870 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.871 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:26.871 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:26.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780407+0000", 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.517e-06 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780414+0000", 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3910000000000003e-06 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780429+0000", 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5379999999999998e-05 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780438+0000", 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9230000000000008e-06 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780885+0000", 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044711300000000001 2024-09-10T10:28:26.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780979+0000", 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4312000000000001e-05 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785856+0000", 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048765350000000004 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.087506667, 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028177609999999998, 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780479+0000", 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9800000000000003e-07 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780488+0000", 2024-09-10T10:28:26.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3349999999999996e-06 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780512+0000", 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3156999999999999e-05 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780518+0000", 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.764e-06 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780576+0000", 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7813999999999999e-05 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780700+0000", 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123733 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783296+0000", 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025963599999999998 2024-09-10T10:28:26.878 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.879 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.879 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.879 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:19 2.14 e24/18 2:2f55791f:::606.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:26.879 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:26.879 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.086387507, 2024-09-10T10:28:26.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003505083, 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:26.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9400000000000005e-07 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781606+0000", 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6559999999999992e-06 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781624+0000", 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7575e-05 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781630+0000", 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8630000000000002e-06 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781683+0000", 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475000000000002e-05 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781700+0000", 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7393000000000001e-05 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785058+0000", 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033575660000000002 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785103+0000", 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-10T10:28:26.883 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:1 2.14 e24/18 2:292cf221:::200.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.086219001, 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033666960000000002, 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781767+0000", 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781772+0000", 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5539999999999997e-06 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781781+0000", 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6149999999999997e-06 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781787+0000", 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1589999999999998e-06 2024-09-10T10:28:26.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2859000000000002e-05 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781876+0000", 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5916000000000001e-05 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785108+0000", 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003232146 2024-09-10T10:28:26.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785133+0000", 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5015999999999999e-05 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:23 2.e e24/18 2:717a0223:::608.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 100.085604465, 2024-09-10T10:28:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027543580000000002, 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:26.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1699999999999998e-07 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782387+0000", 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3229999999999997e-06 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782399+0000", 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2466000000000001e-05 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782406+0000", 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9460000000000001e-06 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782468+0000", 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2347999999999993e-05 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6016e-05 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785096+0000", 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026118629999999999 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785135+0000", 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8779000000000003e-05 2024-09-10T10:28:26.892 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:24 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.990422758999998, 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062416909999999997, 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877564+0000", 2024-09-10T10:28:26.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877576+0000", 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2411e-05 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877592+0000", 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5812e-05 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877600+0000", 2024-09-10T10:28:26.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3590000000000002e-06 2024-09-10T10:28:26.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877661+0000", 2024-09-10T10:28:26.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1749999999999997e-05 2024-09-10T10:28:26.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877785+0000", 2024-09-10T10:28:26.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123807 2024-09-10T10:28:26.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:26.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877936+0000", 2024-09-10T10:28:26.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015084300000000001 2024-09-10T10:28:26.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.879317+0000", 2024-09-10T10:28:26.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001381415 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883686+0000", 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004368701 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883755+0000", 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9140999999999994e-05 2024-09-10T10:28:26.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883804+0000", 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8772e-05 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.899 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.989596794999997, 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056063609999999998, 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878389+0000", 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699999999999999e-07 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878394+0000", 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6120000000000001e-06 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878407+0000", 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3704e-05 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878415+0000", 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7100000000000007e-06 2024-09-10T10:28:26.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878475+0000", 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878603+0000", 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127525 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883995+0000", 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053920460000000002 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.989546429000001, 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057312819999999999, 2024-09-10T10:28:26.904 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:26.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5000000000000003e-07 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878442+0000", 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.632e-06 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878450+0000", 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0509999999999997e-06 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878456+0000", 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7200000000000003e-06 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878847+0000", 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039182099999999999 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878949+0000", 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101458 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.884170+0000", 2024-09-10T10:28:26.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052210499999999996 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.920881111, 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033173069999999998, 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.909 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:26.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947105+0000", 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4670000000000001e-06 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947111+0000", 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6230000000000002e-06 2024-09-10T10:28:26.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947123+0000", 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1883e-05 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947132+0000", 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8740000000000001e-06 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947220+0000", 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7800000000000006e-05 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947375+0000", 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015566399999999999 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:26.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947501+0000", 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125609 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948393+0000", 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00089165399999999997 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950323+0000", 2024-09-10T10:28:26.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019308769999999999 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950366+0000", 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2688999999999997e-05 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950421+0000", 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5167000000000002e-05 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.915 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.920825558999994, 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0070341989999999997, 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.916 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:26.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:26.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:26.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:26.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947163+0000", 2024-09-10T10:28:26.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3440000000000001e-06 2024-09-10T10:28:26.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947174+0000", 2024-09-10T10:28:26.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0248000000000001e-05 2024-09-10T10:28:26.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947180+0000", 2024-09-10T10:28:26.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0639999999999999e-06 2024-09-10T10:28:26.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947958+0000", 2024-09-10T10:28:26.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077789700000000005 2024-09-10T10:28:26.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948031+0000", 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3503000000000004e-05 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948151+0000", 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119976 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953625+0000", 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054739360000000004 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954124+0000", 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049858900000000002 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954157+0000", 2024-09-10T10:28:26.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3652999999999998e-05 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954194+0000", 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6455000000000002e-05 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.923 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.920733716000001, 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032449549999999999, 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947252+0000", 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000003e-07 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947256+0000", 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3760000000000001e-06 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947267+0000", 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0259e-05 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947273+0000", 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2190000000000001e-06 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947332+0000", 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9086000000000003e-05 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947452+0000", 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120146 2024-09-10T10:28:26.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950496+0000", 2024-09-10T10:28:26.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003044171 2024-09-10T10:28:26.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.928 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.928 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:13 2.1 e26/18 2:81735765:::619.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:26.929 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:26.929 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.920323585000006, 2024-09-10T10:28:26.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006029375, 2024-09-10T10:28:26.929 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.929 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.929 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.929 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947662+0000", 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7199999999999997e-07 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6990000000000003e-06 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947681+0000", 2024-09-10T10:28:26.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0682999999999999e-05 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947687+0000", 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7479999999999996e-06 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947742+0000", 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4487000000000002e-05 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947761+0000", 2024-09-10T10:28:26.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9497999999999999e-05 2024-09-10T10:28:26.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953653+0000", 2024-09-10T10:28:26.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058916530000000002 2024-09-10T10:28:26.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953691+0000", 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7734999999999997e-05 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:14 2.1 e26/18 2:81735765:::619.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:26.934 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.919003247999996, 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052525849999999997, 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5700000000000002e-07 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948988+0000", 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7490000000000004e-06 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949003+0000", 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4214e-05 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949010+0000", 2024-09-10T10:28:26.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3499999999999999e-06 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949059+0000", 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8758000000000003e-05 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949078+0000", 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9806e-05 2024-09-10T10:28:26.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954198+0000", 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051191509999999997 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954234+0000", 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.68e-05 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.939 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:1 2.1 e26/18 2:85bbe569:::202.00000000:head v 26'4, mlcod=0'0)", 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 72.918774884000001, 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005049258, 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.940 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949211+0000", 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4999999999999998e-07 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949228+0000", 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7808999999999998e-05 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949240+0000", 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1226000000000001e-05 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949246+0000", 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0689999999999998e-06 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949292+0000", 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6211999999999997e-05 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949311+0000", 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8873e-05 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954236+0000", 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049253980000000001 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954259+0000", 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3221e-05 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:26 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.925912801999999, 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039596270000000003, 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.945 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942075+0000", 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2680000000000001e-06 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942086+0000", 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1361e-05 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942098+0000", 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2221e-05 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942106+0000", 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0960000000000001e-06 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942209+0000", 2024-09-10T10:28:26.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102944 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942346+0000", 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136976 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942516+0000", 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017015699999999999 2024-09-10T10:28:26.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945195+0000", 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026790920000000001 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945919+0000", 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00072374500000000003 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945983+0000", 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3418999999999997e-05 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946032+0000", 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9348000000000003e-05 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:2 2.1 e28/18 2:82971f97:::404.00000000:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:26.951 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.924253645999997, 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054707489999999996, 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943732+0000", 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1999999999999998e-07 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943740+0000", 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1910000000000003e-06 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943755+0000", 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5014000000000001e-05 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943763+0000", 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9519999999999994e-06 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5569999999999997e-05 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943862+0000", 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4150000000000003e-05 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949157+0000", 2024-09-10T10:28:26.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052949340000000003 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949202+0000", 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5018000000000002e-05 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:6 2.e e28/18 2:75ee5fd8:::629.00000000:head v 28'5, mlcod=24'2)", 2024-09-10T10:28:26.956 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.924191817999997, 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027501769999999999, 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943794+0000", 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943799+0000", 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5620000000000003e-06 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943807+0000", 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.486e-06 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943813+0000", 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6419999999999999e-05 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943888+0000", 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9309000000000001e-05 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946498+0000", 2024-09-10T10:28:26.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026094669999999999 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946543+0000", 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:17 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'12, mlcod=26'10)", 2024-09-10T10:28:26.961 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.924148056999996, 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027214219999999998, 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943849+0000", 2024-09-10T10:28:26.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2023e-05 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943857+0000", 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6009999999999997e-06 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943863+0000", 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2899999999999999e-06 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943927+0000", 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3496000000000003e-05 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943949+0000", 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1801e-05 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946490+0000", 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025409579999999998 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946558+0000", 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8830000000000003e-05 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:18 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'13, mlcod=26'10)", 2024-09-10T10:28:26.966 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.924101855000004, 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027193059999999999, 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943884+0000", 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943889+0000", 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.95e-06 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943896+0000", 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3819999999999998e-06 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943901+0000", 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1719999999999998e-06 2024-09-10T10:28:26.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944666+0000", 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00076445300000000001 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944686+0000", 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-05 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018748879999999999 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946603+0000", 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1947999999999999e-05 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:25 2.12 e28/18 2:4dad84a4:::mds4_sessionmap:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:26.971 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.921187891999999, 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002978994, 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946798+0000", 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0100000000000001e-06 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:26.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946807+0000", 2024-09-10T10:28:26.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9700000000000005e-06 2024-09-10T10:28:26.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:26.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946820+0000", 2024-09-10T10:28:26.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2458e-05 2024-09-10T10:28:26.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:26.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946856+0000", 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6427999999999997e-05 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946938+0000", 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2106000000000003e-05 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946958+0000", 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0162000000000001e-05 2024-09-10T10:28:26.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:26.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949723+0000", 2024-09-10T10:28:26.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027645899999999999 2024-09-10T10:28:26.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:26.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:26.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:26.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949776+0000", 2024-09-10T10:28:26.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3270000000000003e-05 2024-09-10T10:28:26.977 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.977 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.977 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.977 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:26.977 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:26.977 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:26.979 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:26.979 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:26.977+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:27.080 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:27.079+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:27.094 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:27.094 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:27.094 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:27.095 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:27.104 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:27.181 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:27.180+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:27.210 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:27.210 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:27.210 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:27.210 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:27.211 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:27.219 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:27.270 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-10T10:28:27.275 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:28:27.275 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:28:27.275 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-10T10:28:27.275 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-10T10:28:27.276 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-10T10:28:27.276 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-10T10:28:27.317 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:27.314+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:27.367 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:27.367 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:27.367 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:27.367 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:27.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.367 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:27.367 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:27.367 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.585178239, 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047203150000000001, 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:27.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779674+0000", 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0359999999999995e-06 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779702+0000", 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.7970000000000002e-05 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.369 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779724+0000", 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2544e-05 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779741+0000", 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6781000000000001e-05 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779981+0000", 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000239764 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780147+0000", 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016605100000000001 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017096200000000001 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781749+0000", 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001431288 2024-09-10T10:28:27.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784234+0000", 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024850940000000002 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784338+0000", 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104078 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.372 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784385+0000", 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6746999999999998e-05 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.1a epoch 24/18 rep_tid 2 entries 24'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.58370012899999, 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021396129999999998, 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:27.373 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781144+0000", 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.393e-06 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781156+0000", 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1514e-05 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8445999999999998e-05 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781202+0000", 2024-09-10T10:28:27.375 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7439e-05 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781286+0000", 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4652000000000002e-05 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783282+0000", 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0019961689999999999 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.376 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:31 2.d e24/18 2:b50e409b:::mds_snaptable:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.582780638, 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030643699999999999, 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:27.377 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782063+0000", 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3699999999999999e-07 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.378 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782072+0000", 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9430000000000006e-06 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782083+0000", 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0998e-05 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782092+0000", 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.3270000000000001e-06 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.379 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782158+0000", 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6830999999999998e-05 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782176+0000", 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7370000000000001e-05 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785091+0000", 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029157879999999999 2024-09-10T10:28:27.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785127+0000", 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5176000000000003e-05 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.58272174699999, 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002013515, 2024-09-10T10:28:27.381 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782122+0000", 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.229e-06 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782137+0000", 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4895999999999999e-05 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782151+0000", 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3229e-05 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782198+0000", 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7301e-05 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782271+0000", 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3516000000000006e-05 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782290+0000", 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8921000000000001e-05 2024-09-10T10:28:27.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784094+0000", 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00180331 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784135+0000", 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1112999999999998e-05 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.385 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:28 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.58261013400001, 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028487970000000001, 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7000000000000005e-07 2024-09-10T10:28:27.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782247+0000", 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3740000000000001e-05 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782258+0000", 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0552e-05 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.388 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782265+0000", 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0700000000000001e-06 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782752+0000", 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048707399999999999 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782771+0000", 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9046999999999999e-05 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785037+0000", 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002265806 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785082+0000", 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4938000000000002e-05 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:29 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 100.58106413900001, 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001932907, 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783780+0000", 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9900000000000009e-07 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783789+0000", 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0140000000000004e-06 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783802+0000", 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2751e-05 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783811+0000", 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3950000000000007e-06 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783894+0000", 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2887999999999997e-05 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783913+0000", 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9052999999999999e-05 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785668+0000", 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001754633 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.394 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785712+0000", 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4174000000000001e-05 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:7 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.492432098999998, 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026422899999999998, 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.395 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.396 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.396 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.396 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.396 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:27.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.396 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.561e-06 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872426+0000", 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3577999999999999e-05 2024-09-10T10:28:27.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872462+0000", 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6145e-05 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872473+0000", 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1109e-05 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872543+0000", 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9319000000000005e-05 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872566+0000", 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3768999999999999e-05 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875010+0000", 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024433910000000001 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875053+0000", 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3417999999999997e-05 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.399 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:8 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.491743709999994, 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0044428820000000004, 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:27.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873100+0000", 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3030000000000005e-06 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873121+0000", 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1471000000000001e-05 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873129+0000", 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7339999999999997e-06 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048786499999999998 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873638+0000", 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0735000000000001e-05 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877493+0000", 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0038550189999999999 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877542+0000", 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9135000000000003e-05 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:17 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.491526377, 2024-09-10T10:28:27.404 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042663060000000001, 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:27.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873318+0000", 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7999999999999993e-07 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873326+0000", 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8279999999999992e-06 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873341+0000", 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4688999999999999e-05 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873348+0000", 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4810000000000003e-06 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873843+0000", 2024-09-10T10:28:27.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000495083 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873860+0000", 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7609999999999999e-05 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877545+0000", 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036848839999999998 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877583+0000", 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7750999999999998e-05 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:18 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.490804339999997, 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035807780000000002, 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.409 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:27.410 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874045+0000", 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9909999999999999e-06 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874055+0000", 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0147e-05 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874070+0000", 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4184e-05 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874123+0000", 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3504999999999997e-05 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874141+0000", 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7524999999999999e-05 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877586+0000", 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034458700000000002 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877619+0000", 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2962000000000001e-05 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.413 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.18 epoch 26/18 rep_tid 5 entries 26'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.9:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.489742695000004, 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048872359999999997, 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875101+0000", 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875110+0000", 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2609999999999999e-06 2024-09-10T10:28:27.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875129+0000", 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8685000000000002e-05 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875143+0000", 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3723000000000001e-05 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875222+0000", 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9498000000000004e-05 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879987+0000", 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047652800000000002 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:7 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'5, mlcod=26'3)", 2024-09-10T10:28:27.417 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 73.414980397999997, 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031425310000000001, 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:27.418 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1100000000000005e-07 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949871+0000", 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8620000000000003e-06 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949885+0000", 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4072e-05 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949893+0000", 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5769999999999996e-06 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949953+0000", 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.978e-05 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949972+0000", 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9188e-05 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.952959+0000", 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029873859999999999 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953005+0000", 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5855000000000001e-05 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.421 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:8 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'6, mlcod=26'3)", 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 73.414927809000005, 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034236359999999999, 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.422 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949916+0000", 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6199999999999998e-07 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949922+0000", 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5390000000000004e-06 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949929+0000", 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4339999999999997e-06 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949934+0000", 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2159999999999997e-06 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.424 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950192+0000", 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00025864699999999998 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950213+0000", 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0761999999999999e-05 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953297+0000", 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030842999999999999 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953339+0000", 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1276000000000003e-05 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:34 2.11 e26/18 2:8975f766:::202.00000001:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 73.410954970000006, 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020619190000000002, 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.426 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.427 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.428 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953889+0000", 2024-09-10T10:28:27.428 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.879999999999999e-07 2024-09-10T10:28:27.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.428 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.428 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953895+0000", 2024-09-10T10:28:27.428 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2249999999999997e-06 2024-09-10T10:28:27.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.428 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953906+0000", 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953912+0000", 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4840000000000003e-06 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953959+0000", 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7768000000000001e-05 2024-09-10T10:28:27.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953976+0000", 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6365999999999999e-05 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955917+0000", 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001941524 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955950+0000", 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2573999999999997e-05 2024-09-10T10:28:27.430 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:4 2.1a e26/18 2:5fdc322d:::61e.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.441887441999995, 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0018931340000000001, 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922956+0000", 2024-09-10T10:28:27.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-07 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922962+0000", 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1890000000000004e-06 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922971+0000", 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922978+0000", 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7830000000000001e-06 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923032+0000", 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3637999999999997e-05 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923049+0000", 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7249999999999999e-05 2024-09-10T10:28:27.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924803+0000", 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001753788 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924849+0000", 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6050000000000001e-05 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.435 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:20 2.11 e26/18 2:890f50f7:::626.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.441779736000001, 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003328012, 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.436 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923064+0000", 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9900000000000001e-07 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923070+0000", 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7499999999999997e-06 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923079+0000", 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9080000000000001e-06 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923089+0000", 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0078000000000001e-05 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923150+0000", 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0300999999999997e-05 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923507+0000", 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00035713599999999998 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926368+0000", 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028615469999999999 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926391+0000", 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2793e-05 2024-09-10T10:28:27.439 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:23 2.18 e26/18 2:1c1db442:::103.00000000.inode:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.441720266999994, 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032793050000000002, 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.440 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923124+0000", 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0599999999999999e-07 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923130+0000", 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.81e-06 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923141+0000", 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923149+0000", 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2679999999999997e-06 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4744000000000002e-05 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923233+0000", 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9190000000000001e-05 2024-09-10T10:28:27.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926353+0000", 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031206419999999999 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926402+0000", 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8755000000000003e-05 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.444 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:13 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'6, mlcod=26'2)", 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.441667636999995, 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031818969999999999, 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.445 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923176+0000", 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4799999999999998e-07 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923182+0000", 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.592e-06 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.446 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923191+0000", 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4029999999999996e-06 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923197+0000", 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1850000000000001e-06 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923230+0000", 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3290000000000001e-05 2024-09-10T10:28:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923247+0000", 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6810999999999999e-05 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926308+0000", 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030606159999999999 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926357+0000", 2024-09-10T10:28:27.448 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9351999999999998e-05 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.d epoch 26/18 rep_tid 20 entries 26'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.15:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.441115139000004, 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034028819999999999, 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.449 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923729+0000", 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.5900000000000005e-07 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923739+0000", 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8439999999999993e-06 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.451 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923761+0000", 2024-09-10T10:28:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2821999999999999e-05 2024-09-10T10:28:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923776+0000", 2024-09-10T10:28:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4763000000000001e-05 2024-09-10T10:28:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.452 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923889+0000", 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000112518 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927131+0000", 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032419760000000001 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:14 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'7, mlcod=26'2)", 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:27.453 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.440400178000004, 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027184890000000001, 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5099999999999999e-07 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924449+0000", 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0310000000000004e-06 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924482+0000", 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2845999999999997e-05 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924500+0000", 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.787e-05 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924594+0000", 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3566999999999994e-05 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924619+0000", 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4978999999999999e-05 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927118+0000", 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024995479999999999 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927161+0000", 2024-09-10T10:28:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2997000000000002e-05 2024-09-10T10:28:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.458 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.458 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:27.460 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:27.462 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:27.462 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-10T10:28:27.462 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:27.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.385+0000 7f7c3e718640 1 Processor -- start 2024-09-10T10:28:27.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.385+0000 7f7c3e718640 1 -- start start 2024-09-10T10:28:27.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c3e718640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c380cc0c0 0x7f7c3815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:27.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c3e718640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7c380cb2f0 0x7f7c380cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:27.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c3e718640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c3815e9d0 0x7f7c380cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:27.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c3e718640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7c380cbd00 con 0x7f7c380cb2f0 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c3e718640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7c3815ba70 con 0x7f7c3815e9d0 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c3e718640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3815e540 con 0x7f7c380cc0c0 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c377fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7c380cb2f0 0x7f7c380cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c377fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7c380cb2f0 0x7f7c380cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59232/0 (socket says 172.21.15.120:59232) 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c377fe640 1 -- 172.21.15.120:0/1925276385 learned_addr learned my addr 172.21.15.120:0/1925276385 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c37fff640 1 --2- 172.21.15.120:0/1925276385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c380cc0c0 0x7f7c3815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.386+0000 7f7c3ccb5640 1 --2- 172.21.15.120:0/1925276385 >> v2:172.21.15.134:3300/0 conn(0x7f7c3815e9d0 0x7f7c380cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.387+0000 7f7c377fe640 1 -- 172.21.15.120:0/1925276385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c380cc0c0 msgr2=0x7f7c3815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.387+0000 7f7c377fe640 1 --2- 172.21.15.120:0/1925276385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c380cc0c0 0x7f7c3815aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.387+0000 7f7c377fe640 1 -- 172.21.15.120:0/1925276385 >> v2:172.21.15.134:3300/0 conn(0x7f7c3815e9d0 msgr2=0x7f7c380cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.387+0000 7f7c377fe640 1 --2- 172.21.15.120:0/1925276385 >> v2:172.21.15.134:3300/0 conn(0x7f7c3815e9d0 0x7f7c380cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.387+0000 7f7c377fe640 1 -- 172.21.15.120:0/1925276385 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3815b1d0 con 0x7f7c380cb2f0 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.387+0000 7f7c377fe640 1 --2- 172.21.15.120:0/1925276385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 0x7f7c380cc4a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7c380ce570 tx=0x7f7c2c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27c699942041f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:27.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.388+0000 7f7c36ffd640 1 -- 172.21.15.120:0/1925276385 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c037070 con 0x7f7c380cb2f0 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.388+0000 7f7c36ffd640 1 -- 172.21.15.120:0/1925276385 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c032440 con 0x7f7c380cb2f0 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.388+0000 7f7c36ffd640 1 -- 172.21.15.120:0/1925276385 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c032850 con 0x7f7c380cb2f0 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.388+0000 7f7c3e718640 1 -- 172.21.15.120:0/1925276385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 msgr2=0x7f7c380cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.388+0000 7f7c3e718640 1 --2- 172.21.15.120:0/1925276385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 0x7f7c380cc4a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7c380ce570 tx=0x7f7c2c02a200 comp rx=0 tx=0).stop 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.388+0000 7f7c3e718640 1 -- 172.21.15.120:0/1925276385 shutdown_connections 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.388+0000 7f7c3e718640 1 --2- 172.21.15.120:0/1925276385 >> v2:172.21.15.134:3300/0 conn(0x7f7c3815e9d0 0x7f7c380cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.388+0000 7f7c3e718640 1 --2- 172.21.15.120:0/1925276385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 0x7f7c380cc4a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.388+0000 7f7c3e718640 1 --2- 172.21.15.120:0/1925276385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c380cc0c0 0x7f7c3815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.388+0000 7f7c3e718640 1 -- 172.21.15.120:0/1925276385 >> 172.21.15.120:0/1925276385 conn(0x7f7c3806d560 msgr2=0x7f7c3806d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.389+0000 7f7c3e718640 1 -- 172.21.15.120:0/1925276385 shutdown_connections 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.389+0000 7f7c3e718640 1 -- 172.21.15.120:0/1925276385 wait complete. 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.390+0000 7f7c3e718640 1 Processor -- start 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c3e718640 1 -- start start 2024-09-10T10:28:27.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c3e718640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 0x7f7c380b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c3e718640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c380cc0c0 0x7f7c381e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c3e718640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c3815e9d0 0x7f7c381f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c3e718640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3815e1d0 con 0x7f7c380cb2f0 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c3e718640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3815b790 con 0x7f7c3815e9d0 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c3e718640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7c380c5560 con 0x7f7c380cc0c0 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c37fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 0x7f7c380b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c37fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 0x7f7c380b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59246/0 (socket says 172.21.15.120:59246) 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c37fff640 1 -- 172.21.15.120:0/1912882427 learned_addr learned my addr 172.21.15.120:0/1912882427 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c3ccb5640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c3815e9d0 0x7f7c381f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.391+0000 7f7c377fe640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c380cc0c0 0x7f7c381e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c37fff640 1 -- 172.21.15.120:0/1912882427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c380cc0c0 msgr2=0x7f7c381e74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:27.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c37fff640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c380cc0c0 0x7f7c381e74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c37fff640 1 -- 172.21.15.120:0/1912882427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c3815e9d0 msgr2=0x7f7c381f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c37fff640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c3815e9d0 0x7f7c381f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c37fff640 1 -- 172.21.15.120:0/1912882427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c380ce690 con 0x7f7c380cb2f0 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c3ccb5640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c3815e9d0 0x7f7c381f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c37fff640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 0x7f7c380b5640 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800e890 tx=0x7f7c2800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c34ff9640 1 -- 172.21.15.120:0/1912882427 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28017070 con 0x7f7c380cb2f0 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c34ff9640 1 -- 172.21.15.120:0/1912882427 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2800c940 con 0x7f7c380cb2f0 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c3e718640 1 -- 172.21.15.120:0/1912882427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c380ce970 con 0x7f7c380cb2f0 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c34ff9640 1 -- 172.21.15.120:0/1912882427 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800cd10 con 0x7f7c380cb2f0 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.392+0000 7f7c3e718640 1 -- 172.21.15.120:0/1912882427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c380cef30 con 0x7f7c380cb2f0 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.393+0000 7f7c3e718640 1 -- 172.21.15.120:0/1912882427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c38068ef0 con 0x7f7c380cb2f0 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.395+0000 7f7c34ff9640 1 -- 172.21.15.120:0/1912882427 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c2801a070 con 0x7f7c380cb2f0 2024-09-10T10:28:27.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.395+0000 7f7c34ff9640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7c2406be30 0x7f7c2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:27.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.395+0000 7f7c377fe640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7c2406be30 0x7f7c2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:27.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.395+0000 7f7c34ff9640 1 -- 172.21.15.120:0/1912882427 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6983+0+0 (secure 0 0 0) 0x7f7c2808b3e0 con 0x7f7c380cb2f0 2024-09-10T10:28:27.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.396+0000 7f7c377fe640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7c2406be30 0x7f7c2406e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c034000 tx=0x7f7c2c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:27.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.403+0000 7f7c34ff9640 1 -- 172.21.15.120:0/1912882427 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7c28059110 con 0x7f7c380cb2f0 2024-09-10T10:28:27.470 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:27.458+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:27.560 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:27.559+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:27.585+0000 7f7c3e718640 1 -- 172.21.15.120:0/1912882427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f7c380cbea0 con 0x7f7c380cb2f0 2024-09-10T10:28:27.605 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:27.605 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:27.605 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:27.605 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:27.614 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:27.661 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:27.660+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:27.716 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:27.716 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:27.716 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:27.716 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:27.716 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:27.725 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:27.766 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:27.765+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:27.863 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:27.862+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:27.870 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.996618775000002, 2024-09-10T10:28:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.006807553, 2024-09-10T10:28:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872290+0000", 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2029999999999999e-06 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872310+0000", 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0508000000000001e-05 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872331+0000", 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0577e-05 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872341+0000", 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0421999999999999e-05 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872457+0000", 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115618 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872575+0000", 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118563 2024-09-10T10:28:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872690+0000", 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011438900000000001 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875281+0000", 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002590995 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879035+0000", 2024-09-10T10:28:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037544879999999998 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879069+0000", 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4034e-05 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879095+0000", 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5755999999999999e-05 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.995422153999996, 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0061030149999999998, 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873485+0000", 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2419999999999999e-06 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.669e-06 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873505+0000", 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.325e-05 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873514+0000", 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1949999999999992e-06 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873575+0000", 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873647+0000", 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2346000000000002e-05 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873778+0000", 2024-09-10T10:28:27.880 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013125600000000001 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875444+0000", 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001665962 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879528+0000", 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040837809999999999 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.881 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879563+0000", 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5329e-05 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879587+0000", 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3947999999999998e-05 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.995368103999994, 2024-09-10T10:28:27.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096274789999999996, 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873539+0000", 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0299999999999998e-07 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873554+0000", 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5481999999999998e-05 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873565+0000", 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0733e-05 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873573+0000", 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9270000000000005e-06 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873633+0000", 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9864000000000001e-05 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873752+0000", 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118946 2024-09-10T10:28:27.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873866+0000", 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000113757 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875462+0000", 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015966699999999999 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883084+0000", 2024-09-10T10:28:27.886 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076214280000000004 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883136+0000", 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2182000000000003e-05 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883166+0000", 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.9787000000000001e-05 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.887 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.995310133000004, 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097094669999999994, 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.888 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873597+0000", 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8899999999999998e-07 2024-09-10T10:28:27.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873602+0000", 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9230000000000001e-06 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873611+0000", 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9150000000000002e-06 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5239999999999997e-06 2024-09-10T10:28:27.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874719+0000", 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0011012769999999999 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874791+0000", 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2305000000000007e-05 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:27.891 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874962+0000", 2024-09-10T10:28:27.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000171099 2024-09-10T10:28:27.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.892 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.892 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877786+0000", 2024-09-10T10:28:27.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028241059999999998 2024-09-10T10:28:27.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.892 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883211+0000", 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054250019999999999 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883263+0000", 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1563000000000001e-05 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883306+0000", 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2864000000000002e-05 2024-09-10T10:28:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.994389443000003, 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005164681, 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874518+0000", 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3799999999999996e-07 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874521+0000", 2024-09-10T10:28:27.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8550000000000004e-06 2024-09-10T10:28:27.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.896 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.896 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874531+0000", 2024-09-10T10:28:27.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8079999999999999e-06 2024-09-10T10:28:27.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.896 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874538+0000", 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2760000000000003e-06 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874945+0000", 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000406516 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875045+0000", 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9570999999999999e-05 2024-09-10T10:28:27.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879682+0000", 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046369169999999999 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.994355548000001, 2024-09-10T10:28:27.898 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056651770000000004, 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4600000000000005e-07 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874565+0000", 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4017e-05 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874573+0000", 2024-09-10T10:28:27.900 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9689999999999994e-06 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874678+0000", 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7483000000000002e-05 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874786+0000", 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107914 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.880216+0000", 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005429499 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:24 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.902 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 73.920047225000005, 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0079888200000000006, 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:27.903 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948861+0000", 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9269999999999999e-06 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948869+0000", 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7749999999999993e-06 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948884+0000", 2024-09-10T10:28:27.904 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5713e-05 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948894+0000", 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2050000000000008e-06 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948958+0000", 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4398000000000002e-05 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.905 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949093+0000", 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013508300000000001 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949223+0000", 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012949500000000001 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953128+0000", 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039058360000000002 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.906 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956775+0000", 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036467439999999999 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956812+0000", 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6510000000000001e-05 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956848+0000", 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6134000000000003e-05 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.907 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:23 2.c e26/18 2:3631e767:::102.00000000.inode:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 73.918562975, 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054176620000000002, 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:27.908 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950344+0000", 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9400000000000001e-07 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950362+0000", 2024-09-10T10:28:27.909 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8655999999999999e-05 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950384+0000", 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1929999999999998e-05 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950391+0000", 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7020000000000004e-06 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950624+0000", 2024-09-10T10:28:27.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000233031 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950643+0000", 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9459000000000001e-05 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955726+0000", 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050826389999999999 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:28:27.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4851000000000002e-05 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.947913990000004, 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0070980829999999998, 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920994+0000", 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.697e-06 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921001+0000", 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3719999999999999e-06 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921014+0000", 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2529e-05 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921021+0000", 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9519999999999997e-06 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921097+0000", 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6499000000000001e-05 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921261+0000", 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016399599999999999 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921396+0000", 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013448400000000001 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922936+0000", 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015405970000000001 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927988+0000", 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00505158 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928047+0000", 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9011000000000001e-05 2024-09-10T10:28:27.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928090+0000", 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3365999999999998e-05 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.947862478999994, 2024-09-10T10:28:27.917 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007356853, 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5699999999999998e-07 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921048+0000", 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.754e-06 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921056+0000", 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0469999999999994e-06 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921063+0000", 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5980000000000002e-06 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921747+0000", 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00068475800000000005 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921852+0000", 2024-09-10T10:28:27.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104347 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922002+0000", 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000150233 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924357+0000", 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0023545599999999999 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928286+0000", 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039299540000000003 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928349+0000", 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2044000000000004e-05 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928401+0000", 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2101000000000001e-05 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 67.947823021000005, 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005625166, 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:27.923 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921084+0000", 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0100000000000005e-07 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921093+0000", 2024-09-10T10:28:27.924 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9910000000000008e-06 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921099+0000", 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2269999999999998e-06 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921103+0000", 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0929999999999996e-06 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921167+0000", 2024-09-10T10:28:27.925 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4030999999999997e-05 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921301+0000", 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013388699999999999 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921471+0000", 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017000899999999999 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924553+0000", 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030818199999999999 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926600+0000", 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020471339999999999 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926663+0000", 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3274000000000004e-05 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926708+0000", 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5099000000000003e-05 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 61.925271350999999, 2024-09-10T10:28:27.928 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010012200000000001, 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943637+0000", 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5390000000000002e-06 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943646+0000", 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1340000000000004e-06 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943661+0000", 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4958e-05 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943669+0000", 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5629999999999992e-06 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943785+0000", 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011584800000000001 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943960+0000", 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000174611 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944078+0000", 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118381 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948022+0000", 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039442599999999998 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953546+0000", 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055242449999999997 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.932 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953604+0000", 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7555e-05 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953647+0000", 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3105999999999997e-05 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:27.933 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 61.925077792000003, 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099609380000000008, 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:27.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943829+0000", 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3099999999999996e-07 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943835+0000", 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8610000000000003e-06 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943846+0000", 2024-09-10T10:28:27.935 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0686999999999999e-05 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943854+0000", 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944413+0000", 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055893399999999997 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944505+0000", 2024-09-10T10:28:27.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2788999999999995e-05 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944639+0000", 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013387699999999999 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950701+0000", 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0060612360000000002 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.937 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953700+0000", 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029997689999999998 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953743+0000", 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3024e-05 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953789+0000", 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6109999999999997e-05 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 61.925037770000003, 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054304089999999998, 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1399999999999996e-07 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.940 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943874+0000", 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3830000000000002e-06 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943881+0000", 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1369999999999996e-06 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943886+0000", 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-06 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.941 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944057+0000", 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000170781 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944182+0000", 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012506299999999999 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944309+0000", 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127081 2024-09-10T10:28:27.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.947998+0000", 2024-09-10T10:28:27.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003689323 2024-09-10T10:28:27.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949225+0000", 2024-09-10T10:28:27.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001227005 2024-09-10T10:28:27.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.944 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.944 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949267+0000", 2024-09-10T10:28:27.944 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1391999999999999e-05 2024-09-10T10:28:27.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.944 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.944 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949299+0000", 2024-09-10T10:28:27.945 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2169999999999999e-05 2024-09-10T10:28:27.945 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.945 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.945 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.945 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.945 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:27.945 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 61.924977288000001, 2024-09-10T10:28:27.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0068819399999999996, 2024-09-10T10:28:27.946 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.946 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.946 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.946 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.946 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943930+0000", 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1200000000000002e-07 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943934+0000", 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0969999999999999e-06 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944782+0000", 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00084778299999999996 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944793+0000", 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0993e-05 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944864+0000", 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1241000000000005e-05 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944940+0000", 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6044999999999994e-05 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945068+0000", 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012846 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949683+0000", 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046151040000000001 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950706+0000", 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010222149999999999 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950764+0000", 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8180999999999997e-05 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950811+0000", 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7108999999999999e-05 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:24 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 61.923603661999998, 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055052950000000003, 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.951 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945304+0000", 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4799999999999997e-07 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945311+0000", 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1269999999999997e-06 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945322+0000", 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1134e-05 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945330+0000", 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9039999999999992e-06 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945408+0000", 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8200000000000003e-05 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945533+0000", 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000125518 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945659+0000", 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012583599999999999 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949147+0000", 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003487675 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950716+0000", 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015695279999999999 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950767+0000", 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0577e-05 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950808+0000", 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0847999999999999e-05 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:25 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.956 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 61.922962292999998, 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050290129999999997, 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945945+0000", 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1119999999999999e-06 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945952+0000", 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5189999999999998e-06 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945965+0000", 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3643e-05 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945974+0000", 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5750000000000001e-06 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0532999999999998e-05 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946192+0000", 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015717699999999999 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946341+0000", 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014897200000000001 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949885+0000", 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035449729999999999 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950896+0000", 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00101023 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950938+0000", 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2284999999999999e-05 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950973+0000", 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4993999999999997e-05 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.961 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 61.922909220999998, 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048920780000000002, 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:27.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945998+0000", 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3900000000000004e-07 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946012+0000", 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4799e-05 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946021+0000", 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.6000000000000007e-06 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946029+0000", 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9209999999999992e-06 2024-09-10T10:28:27.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946093+0000", 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3608000000000005e-05 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946242+0000", 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014889699999999999 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950889+0000", 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046476140000000004 2024-09-10T10:28:27.965 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 61.922865194000003, 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047976679999999997, 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:27.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946042+0000", 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5199999999999996e-07 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946046+0000", 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1439999999999996e-06 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946054+0000", 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1140000000000006e-06 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946070+0000", 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6232e-05 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946137+0000", 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6956000000000001e-05 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946260+0000", 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012240299999999999 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950839+0000", 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045791670000000003 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:23 2.1c e28/18 2:399c8d31:::104.00000000.inode:head v 28'5, mlcod=24'1)", 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 61.922345855000003, 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042418079999999997, 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3e-07 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946574+0000", 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.277e-05 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946584+0000", 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7710000000000001e-06 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946590+0000", 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6769999999999998e-06 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946666+0000", 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5420999999999995e-05 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.972 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946687+0000", 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1665999999999999e-05 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950762+0000", 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040749030000000004 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950802+0000", 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9969999999999998e-05 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.973 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.974 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:27.974 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:27.974 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:27.975 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:27.975 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:27.974+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:28.075 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:28.074+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:28.119 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:28.119 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:28.119 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:28.119 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:28.128 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:28.155 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:28:28.176 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:28.175+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:28.231 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:28.231 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:28.231 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:28.231 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:28.231 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:28.239 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:28.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.244+0000 7f7c34ff9640 1 -- 172.21.15.120:0/1912882427 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v53) ==== 97+0+0 (secure 0 0 0) 0x7f7c2805d410 con 0x7f7c380cb2f0 2024-09-10T10:28:28.246 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-10T10:28:28.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.248+0000 7f7c3e718640 1 -- 172.21.15.120:0/1912882427 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7c2406be30 msgr2=0x7f7c2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.248+0000 7f7c3e718640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7c2406be30 0x7f7c2406e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c034000 tx=0x7f7c2c02e000 comp rx=0 tx=0).stop 2024-09-10T10:28:28.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.248+0000 7f7c3e718640 1 -- 172.21.15.120:0/1912882427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 msgr2=0x7f7c380b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.248+0000 7f7c3e718640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 0x7f7c380b5640 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800e890 tx=0x7f7c2800ed50 comp rx=0 tx=0).stop 2024-09-10T10:28:28.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.249+0000 7f7c3e718640 1 -- 172.21.15.120:0/1912882427 shutdown_connections 2024-09-10T10:28:28.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.249+0000 7f7c3e718640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7c2406be30 0x7f7c2406e2f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.249+0000 7f7c3e718640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c3815e9d0 0x7f7c381f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.249+0000 7f7c3e718640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c380cc0c0 0x7f7c381e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.249+0000 7f7c3e718640 1 --2- 172.21.15.120:0/1912882427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c380cb2f0 0x7f7c380b5640 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.249+0000 7f7c3e718640 1 -- 172.21.15.120:0/1912882427 >> 172.21.15.120:0/1912882427 conn(0x7f7c3806d560 msgr2=0x7f7c381527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:28.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.250+0000 7f7c3e718640 1 -- 172.21.15.120:0/1912882427 shutdown_connections 2024-09-10T10:28:28.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.251+0000 7f7c3e718640 1 -- 172.21.15.120:0/1912882427 wait complete. 2024-09-10T10:28:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.266+0000 7f98f48d7640 1 Processor -- start 2024-09-10T10:28:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.266+0000 7f98f48d7640 1 -- start start 2024-09-10T10:28:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98f48d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98f0154a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98f48d7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98f00c7d80 con 0x7f98f00c9490 2024-09-10T10:28:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98f48d7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98f006c670 con 0x7f98f00c78b0 2024-09-10T10:28:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98f48d7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98f006d280 con 0x7f98f00c90b0 2024-09-10T10:28:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98eed9d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f98f00c9490 0x7f98f01653c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41714/0 (socket says 172.21.15.120:41714) 2024-09-10T10:28:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98edd9b640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f98f00c78b0 0x7f98f015e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:54372/0 (socket says 172.21.15.120:54372) 2024-09-10T10:28:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98eed9d640 1 -- 172.21.15.120:0/964329047 learned_addr learned my addr 172.21.15.120:0/964329047 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ee59c640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98f0154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404099372 0 0) 0x7f98f006c670 con 0x7f98f00c78b0 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98d8003520 con 0x7f98f00c78b0 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833836802 0 0) 0x7f98f00c7d80 con 0x7f98f00c9490 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98f006c670 con 0x7f98f00c9490 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 592926407 0 0) 0x7f98d8003520 con 0x7f98f00c78b0 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 msgr2=0x7f98f0154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98f0154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 >> v1:172.21.15.120:6789/0 conn(0x7f98f00c9490 legacy=0x7f98f01653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f006c950 con 0x7f98f00c78b0 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f98e40015f0 con 0x7f98f00c78b0 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 >> v1:172.21.15.134:6789/0 conn(0x7f98f00c78b0 legacy=0x7f98f015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8004ba0 0x7f98d8004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98d80037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98d8003ce0 0x7f98d8004160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98f006d280 con 0x7f98d8003ce0 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98d800ba70 con 0x7f98d8004ba0 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.268+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98d800bbf0 con 0x7f98f00c90b0 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.269+0000 7f98ee59c640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8004ba0 0x7f98d8004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.269+0000 7f98edd9b640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98d80037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.269+0000 7f98eed9d640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98d8003ce0 0x7f98d8004160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.269+0000 7f98edd9b640 1 -- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8004ba0 msgr2=0x7f98d8004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.269+0000 7f98edd9b640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8004ba0 0x7f98d8004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.269+0000 7f98edd9b640 1 -- 172.21.15.120:0/964329047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98d8003ce0 msgr2=0x7f98d8004160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.269+0000 7f98edd9b640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98d8003ce0 0x7f98d8004160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.269+0000 7f98edd9b640 1 -- 172.21.15.120:0/964329047 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d8003520 con 0x7f98f00c90b0 2024-09-10T10:28:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.269+0000 7f98eed9d640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98d8003ce0 0x7f98d8004160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.270+0000 7f98edd9b640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98d80037a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98e4007ea0 tx=0x7f98d8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61478c733b8f1e45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.270+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4007af0 con 0x7f98f00c90b0 2024-09-10T10:28:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.270+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e4005b20 con 0x7f98f00c90b0 2024-09-10T10:28:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.270+0000 7f98ed59a640 1 -- 172.21.15.120:0/964329047 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4007070 con 0x7f98f00c90b0 2024-09-10T10:28:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.271+0000 7f98f48d7640 1 -- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 msgr2=0x7f98d80037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.271+0000 7f98f48d7640 1 --2- 172.21.15.120:0/964329047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98d80037a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98e4007ea0 tx=0x7f98d8005830 comp rx=0 tx=0).stop 2024-09-10T10:28:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.271+0000 7f98eed9d640 1 -- 172.21.15.120:0/964329047 reap_dead start 2024-09-10T10:28:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.271+0000 7f98f48d7640 1 -- 172.21.15.120:0/964329047 shutdown_connections 2024-09-10T10:28:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.271+0000 7f98f48d7640 1 -- 172.21.15.120:0/964329047 >> 172.21.15.120:0/964329047 conn(0x7f98f00728b0 msgr2=0x7f98f0072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.271+0000 7f98f48d7640 1 -- 172.21.15.120:0/964329047 shutdown_connections 2024-09-10T10:28:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.271+0000 7f98f48d7640 1 -- 172.21.15.120:0/964329047 wait complete. 2024-09-10T10:28:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.273+0000 7f98f48d7640 1 Processor -- start 2024-09-10T10:28:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.273+0000 7f98f48d7640 1 -- start start 2024-09-10T10:28:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98f48d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98f00bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98f48d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f00c52f0 0x7f98f00bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98f48d7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98f014fbb0 0x7f98f0150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98f48d7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98f006bda0 con 0x7f98f014fbb0 2024-09-10T10:28:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98f48d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98f006cf10 con 0x7f98f00c52f0 2024-09-10T10:28:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98f48d7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98f015ea80 con 0x7f98f00c90b0 2024-09-10T10:28:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98eed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98f014fbb0 0x7f98f0150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98ee59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98f00bf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98edd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f00c52f0 0x7f98f00bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98eed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98f014fbb0 0x7f98f0150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59252/0 (socket says 172.21.15.120:59252) 2024-09-10T10:28:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98ee59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98f00bf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:40328/0 (socket says 172.21.15.120:40328) 2024-09-10T10:28:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.274+0000 7f98eed9d640 1 -- 172.21.15.120:0/3612045656 learned_addr learned my addr 172.21.15.120:0/3612045656 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98eed9d640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 msgr2=0x7f98f00bf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98eed9d640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98f00bf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98eed9d640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f00c52f0 msgr2=0x7f98f00bdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98eed9d640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f00c52f0 0x7f98f00bdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98eed9d640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f00c7080 con 0x7f98f014fbb0 2024-09-10T10:28:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98ee59c640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98f00c90b0 0x7f98f00bf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98eed9d640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98f014fbb0 0x7f98f0150020 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f98f00c7cb0 tx=0x7f98e800c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e800c0e0 con 0x7f98f014fbb0 2024-09-10T10:28:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f00bd780 con 0x7f98f014fbb0 2024-09-10T10:28:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e8018070 con 0x7f98f014fbb0 2024-09-10T10:28:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e80133f0 con 0x7f98f014fbb0 2024-09-10T10:28:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.275+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f01f09a0 con 0x7f98f014fbb0 2024-09-10T10:28:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.277+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98e8013590 con 0x7f98f014fbb0 2024-09-10T10:28:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.277+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f98f0068ef0 con 0x7f98f014fbb0 2024-09-10T10:28:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.278+0000 7f98deffd640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f98d006be30 0x7f98d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.278+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6983+0+0 (secure 0 0 0) 0x7f98e808b670 con 0x7f98f014fbb0 2024-09-10T10:28:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.278+0000 7f98ee59c640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f98d006be30 0x7f98d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.278+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v53) ==== 54+0+16 (secure 0 0 0) 0x7f98e8059bb0 con 0x7f98f014fbb0 2024-09-10T10:28:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.278+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f98f00c5800 0x7f98f00c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.278+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98f0070d40 con 0x7f98f00c5800 2024-09-10T10:28:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.278+0000 7f98edd9b640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f98f00c5800 0x7f98f00c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.279+0000 7f98ee59c640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f98d006be30 0x7f98d006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009e70 tx=0x7f98e0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.279+0000 7f98edd9b640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f98f00c5800 0x7f98f00c5be0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.283 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:28.280+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:28.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.280+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98f0070d40 con 0x7f98f00c5800 2024-09-10T10:28:28.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.303+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98f006ea60 con 0x7f98f00c5800 2024-09-10T10:28:28.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.306+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98f006ea60 con 0x7f98f00c5800 2024-09-10T10:28:28.307 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:28.307 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:28.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.306+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f98f01f39a0 0x7f98f01f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.306+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f98f00c0020 con 0x7f98f01f39a0 2024-09-10T10:28:28.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.307+0000 7f98ee59c640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f98f01f39a0 0x7f98f01f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.307+0000 7f98ee59c640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f98f01f39a0 0x7f98f01f3dc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.308+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98f00c0020 con 0x7f98f01f39a0 2024-09-10T10:28:28.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.323+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98f00be4a0 con 0x7f98f01f39a0 2024-09-10T10:28:28.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.325+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98f00be4a0 con 0x7f98f01f39a0 2024-09-10T10:28:28.326 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:28.327 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:28.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.325+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f98f0062b40 0x7f98f0062f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.325+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f98f00634c0 con 0x7f98f0062b40 2024-09-10T10:28:28.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.326+0000 7f98edd9b640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f98f0062b40 0x7f98f0062f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.326+0000 7f98edd9b640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f98f0062b40 0x7f98f0062f80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.327+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98f00634c0 con 0x7f98f0062b40 2024-09-10T10:28:28.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.342+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98f015c870 con 0x7f98f0062b40 2024-09-10T10:28:28.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.343+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98f015c870 con 0x7f98f0062b40 2024-09-10T10:28:28.345 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:28.345 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:28.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.344+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f98f002c640 0x7f98f002cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.344+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f98f015d350 con 0x7f98f002c640 2024-09-10T10:28:28.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.344+0000 7f98eed9d640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f98f002c640 0x7f98f002cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.344+0000 7f98eed9d640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f98f002c640 0x7f98f002cac0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.345+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98f015d350 con 0x7f98f002c640 2024-09-10T10:28:28.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.359+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98f002d2d0 con 0x7f98f002c640 2024-09-10T10:28:28.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.362+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98f002d2d0 con 0x7f98f002c640 2024-09-10T10:28:28.363 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:28.364 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:28.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.362+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f98f002d510 0x7f98f0204830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.362+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f98f002d970 con 0x7f98f002d510 2024-09-10T10:28:28.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.363+0000 7f98ee59c640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f98f002d510 0x7f98f0204830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.363+0000 7f98ee59c640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f98f002d510 0x7f98f0204830 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.364+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98f002d970 con 0x7f98f002d510 2024-09-10T10:28:28.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.377+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98f002d970 con 0x7f98f002d510 2024-09-10T10:28:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.379+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98f002d970 con 0x7f98f002d510 2024-09-10T10:28:28.380 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.379+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f98f0205280 0x7f98f0207670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.379+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f98f002d970 con 0x7f98f0205280 2024-09-10T10:28:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.380+0000 7f98edd9b640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f98f0205280 0x7f98f0207670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.380+0000 7f98edd9b640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f98f0205280 0x7f98f0207670 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.381+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98f002d970 con 0x7f98f0205280 2024-09-10T10:28:28.382 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:28.381+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:28.388 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:28.388 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:28.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:28.388 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:28.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.388 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:28.388 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:28.388 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 101.60644919800001, 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:28:28.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:28:28.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 101.606118698, 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:28.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.396 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:28:28.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.399 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 101.60607215500001, 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:28:28.400 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:28.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:28:28.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 79.514629943000003, 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:28.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:28:28.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.410 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 79.514548599999998, 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:28:28.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.413 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.413 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:28:28.413 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:28:28.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.413 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.413 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:28:28.414 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:28:28.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.414 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.414 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:28.414 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:28:28.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.415 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.415 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:28:28.415 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:28:28.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.415 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:28.415 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:28:28.416 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:28:28.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.416 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.416 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:28:28.416 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:28:28.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.416 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.416 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:28:28.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 79.514522045000007, 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.418 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.420 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.421 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 79.514379395999995, 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:28:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 79.511732463000001, 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:28:28.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:28:28.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:28:28.434 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:28:28.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:28:28.434 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 79.510881722999997, 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:28:28.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.437 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 74.438730039999996, 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:28.440 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.441 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.442 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:28:28.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 74.438680903999995, 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:28:28.445 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:28:28.447 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.448 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:28:28.449 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.450 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 74.438569426000001, 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.452 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:28:28.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 68.465168907000006, 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.455 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:28:28.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 68.465138736, 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.461 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.462 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:28:28.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.465 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 68.465082244000001, 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.467 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:28:28.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 68.464981076000001, 2024-09-10T10:28:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:28:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:28:28.474 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.475 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.478 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.444713978999999, 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.479 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.480 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:28:28.481 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:28:28.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.440648760000002, 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:28:28.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:28:28.487 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.488 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.440630890000001, 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.489 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.492 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:28.492 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:28:28.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.492 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.492 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:28:28.492 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:28:28.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.492 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:28:28.493 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 62.440584651999998, 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:28.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:28:28.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:28.498 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:28.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.397+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98f002d970 con 0x7f98f0205280 2024-09-10T10:28:28.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.398+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98f002d970 con 0x7f98f0205280 2024-09-10T10:28:28.500 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:28.501 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:28.502 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:28.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.399+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f98f02080c0 0x7f98f020a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.399+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f98f020d280 con 0x7f98f02080c0 2024-09-10T10:28:28.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.399+0000 7f98eed9d640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f98f02080c0 0x7f98f020a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.399+0000 7f98eed9d640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f98f02080c0 0x7f98f020a4b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.400+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98f020d280 con 0x7f98f02080c0 2024-09-10T10:28:28.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.414+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98f020a9f0 con 0x7f98f02080c0 2024-09-10T10:28:28.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.416+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98f020a9f0 con 0x7f98f02080c0 2024-09-10T10:28:28.503 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:28.504 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:28.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.416+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f98f020b070 0x7f98f020d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:28.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.416+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f98f0210230 con 0x7f98f020b070 2024-09-10T10:28:28.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.416+0000 7f98ee59c640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f98f020b070 0x7f98f020d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:28.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.417+0000 7f98ee59c640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f98f020b070 0x7f98f020d460 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:28.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.417+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98f0210230 con 0x7f98f020b070 2024-09-10T10:28:28.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.431+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98f020d9a0 con 0x7f98f020b070 2024-09-10T10:28:28.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98deffd640 1 -- 172.21.15.120:0/3612045656 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98f020d9a0 con 0x7f98f020b070 2024-09-10T10:28:28.505 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:28.507 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:28.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f98f00c5800 msgr2=0x7f98f00c5be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f98f00c5800 0x7f98f00c5be0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f98f01f39a0 msgr2=0x7f98f01f3dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f98f01f39a0 0x7f98f01f3dc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f98f0062b40 msgr2=0x7f98f0062f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f98f0062b40 0x7f98f0062f80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f98f002c640 msgr2=0x7f98f002cac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f98f002c640 0x7f98f002cac0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f98f002d510 msgr2=0x7f98f0204830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f98f002d510 0x7f98f0204830 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f98f0205280 msgr2=0x7f98f0207670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f98f0205280 0x7f98f0207670 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f98f02080c0 msgr2=0x7f98f020a4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f98f02080c0 0x7f98f020a4b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f98f020b070 msgr2=0x7f98f020d460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f98f020b070 0x7f98f020d460 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98eed9d640 1 -- 172.21.15.120:0/3612045656 reap_dead start 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98eed9d640 1 -- 172.21.15.120:0/3612045656 reap_dead start 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98eed9d640 1 -- 172.21.15.120:0/3612045656 reap_dead start 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98eed9d640 1 -- 172.21.15.120:0/3612045656 reap_dead start 2024-09-10T10:28:28.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.433+0000 7f98eed9d640 1 -- 172.21.15.120:0/3612045656 reap_dead start 2024-09-10T10:28:28.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.434+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f98d006be30 msgr2=0x7f98d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.434+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f98d006be30 0x7f98d006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009e70 tx=0x7f98e0009190 comp rx=0 tx=0).stop 2024-09-10T10:28:28.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.434+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98f014fbb0 msgr2=0x7f98f0150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:28.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.434+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98f014fbb0 0x7f98f0150020 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f98f00c7cb0 tx=0x7f98e800c650 comp rx=0 tx=0).stop 2024-09-10T10:28:28.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.434+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 shutdown_connections 2024-09-10T10:28:28.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.434+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f98d006be30 0x7f98d006e2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.434+0000 7f98f48d7640 1 --2- 172.21.15.120:0/3612045656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98f014fbb0 0x7f98f0150020 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:28.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.434+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 >> 172.21.15.120:0/3612045656 conn(0x7f98f00728b0 msgr2=0x7f98f015bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:28.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.434+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 shutdown_connections 2024-09-10T10:28:28.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:28.434+0000 7f98f48d7640 1 -- 172.21.15.120:0/3612045656 wait complete. 2024-09-10T10:28:28.510 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:28.499+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:28.601 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:28.600+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:28.612 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:28.612 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:28.621 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:28.702 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:28.701+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:28.725 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:28.725 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:28.734 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:28.802 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:28.802+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:28.879 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:28.879 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:28.887 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:28.903 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:28.902+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:28.995 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:28.995 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:29.004 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:29.005 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:29.003+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:29.105 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:29.105+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:29.152 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:29.153 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:29.161 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:29.216 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:29.215+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:29.308 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:29.307+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:29.314 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:29.314 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:29.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:29.409 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:29.408+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:29.474 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:29.474 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:29.485 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:29.510 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:29.510+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:29.592 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:29.592 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:29.601 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:29.611 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:29.611+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:29.708 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:29.708 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:29.712 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:29.711+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:29.717 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:29.813 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:29.813+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:29.826 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:29.826 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:29.836 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:29.914 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:29.913+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:29.942 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:29.942 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:29.951 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:30.036 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:30.035+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:30.106 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:30.107 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:30.116 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:30.115+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:30.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:30.217 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:30.216+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:30.264 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-10T10:28:30.265 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:30.265 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:30.276 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:30.317 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:30.316+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:30.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.371+0000 7f2142037640 1 Processor -- start 2024-09-10T10:28:30.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.372+0000 7f2142037640 1 -- start start 2024-09-10T10:28:30.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.372+0000 7f2142037640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f213c157ef0 0x7f213c0c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.372+0000 7f2142037640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f213c072e00 0x7f213c073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f2142037640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c073790 con 0x7f213c1582d0 2024-09-10T10:28:30.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f2142037640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f213c072970 con 0x7f213c072e00 2024-09-10T10:28:30.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f2142037640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f213c0caa80 con 0x7f213c157ef0 2024-09-10T10:28:30.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f213bfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f213c072e00 0x7f213c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f213affd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f213c1582d0 0x7f213c0c9a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41728/0 (socket says 172.21.15.120:41728) 2024-09-10T10:28:30.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f213affd640 1 -- 172.21.15.120:0/2502558968 learned_addr learned my addr 172.21.15.120:0/2502558968 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:30.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f213b7fe640 1 --2- 172.21.15.120:0/2502558968 >> v2:172.21.15.134:3300/0 conn(0x7f213c157ef0 0x7f213c0c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f213b7fe640 1 -- 172.21.15.120:0/2502558968 >> v2:172.21.15.120:3300/0 conn(0x7f213c072e00 msgr2=0x7f213c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:30.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f213b7fe640 1 --2- 172.21.15.120:0/2502558968 >> v2:172.21.15.120:3300/0 conn(0x7f213c072e00 0x7f213c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f213b7fe640 1 -- 172.21.15.120:0/2502558968 >> v1:172.21.15.120:6789/0 conn(0x7f213c1582d0 legacy=0x7f213c0c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f213a7fc640 1 -- 172.21.15.120:0/2502558968 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2581368591 0 0) 0x7f213c073790 con 0x7f213c1582d0 2024-09-10T10:28:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.373+0000 7f213b7fe640 1 -- 172.21.15.120:0/2502558968 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c15afa0 con 0x7f213c157ef0 2024-09-10T10:28:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.374+0000 7f213b7fe640 1 --2- 172.21.15.120:0/2502558968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c157ef0 0x7f213c0c7000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f213c071db0 tx=0x7f212c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e71648d133d4359 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.375+0000 7f213a7fc640 1 -- 172.21.15.120:0/2502558968 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c037070 con 0x7f213c157ef0 2024-09-10T10:28:30.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.375+0000 7f213a7fc640 1 -- 172.21.15.120:0/2502558968 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f212c032440 con 0x7f213c157ef0 2024-09-10T10:28:30.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.375+0000 7f213a7fc640 1 -- 172.21.15.120:0/2502558968 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c032810 con 0x7f213c157ef0 2024-09-10T10:28:30.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.375+0000 7f2142037640 1 -- 172.21.15.120:0/2502558968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c157ef0 msgr2=0x7f213c0c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.375+0000 7f2142037640 1 --2- 172.21.15.120:0/2502558968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c157ef0 0x7f213c0c7000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f213c071db0 tx=0x7f212c02a230 comp rx=0 tx=0).stop 2024-09-10T10:28:30.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.376+0000 7f2142037640 1 -- 172.21.15.120:0/2502558968 shutdown_connections 2024-09-10T10:28:30.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.376+0000 7f2142037640 1 --2- 172.21.15.120:0/2502558968 >> v2:172.21.15.120:3300/0 conn(0x7f213c072e00 0x7f213c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.376+0000 7f2142037640 1 --2- 172.21.15.120:0/2502558968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c157ef0 0x7f213c0c7000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.376+0000 7f2142037640 1 -- 172.21.15.120:0/2502558968 >> 172.21.15.120:0/2502558968 conn(0x7f213c0703e0 msgr2=0x7f213c0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:30.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.376+0000 7f2142037640 1 -- 172.21.15.120:0/2502558968 shutdown_connections 2024-09-10T10:28:30.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.376+0000 7f2142037640 1 -- 172.21.15.120:0/2502558968 wait complete. 2024-09-10T10:28:30.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.377+0000 7f2142037640 1 Processor -- start 2024-09-10T10:28:30.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.377+0000 7f2142037640 1 -- start start 2024-09-10T10:28:30.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f2142037640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c072e00 0x7f213c0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f2142037640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c157ef0 0x7f213c0c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f2142037640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f213c1582d0 0x7f213c158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f2142037640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f213c15b560 con 0x7f213c1582d0 2024-09-10T10:28:30.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f2142037640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f213c0c7540 con 0x7f213c157ef0 2024-09-10T10:28:30.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f2142037640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f213c15b840 con 0x7f213c072e00 2024-09-10T10:28:30.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f213bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f213c1582d0 0x7f213c158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f213b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c072e00 0x7f213c0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f213bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f213c1582d0 0x7f213c158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59268/0 (socket says 172.21.15.120:59268) 2024-09-10T10:28:30.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f213affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c157ef0 0x7f213c0c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f213bfff640 1 -- 172.21.15.120:0/2658637785 learned_addr learned my addr 172.21.15.120:0/2658637785 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:30.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.378+0000 7f213bfff640 1 -- 172.21.15.120:0/2658637785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c072e00 msgr2=0x7f213c0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.379+0000 7f213bfff640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c072e00 0x7f213c0c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.379+0000 7f213bfff640 1 -- 172.21.15.120:0/2658637785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c157ef0 msgr2=0x7f213c0c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.379+0000 7f213bfff640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c157ef0 0x7f213c0c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.379+0000 7f213bfff640 1 -- 172.21.15.120:0/2658637785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c0c9700 con 0x7f213c1582d0 2024-09-10T10:28:30.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.379+0000 7f213affd640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c157ef0 0x7f213c0c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:30.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.379+0000 7f213bfff640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f213c1582d0 0x7f213c158b10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f213400ec40 tx=0x7f213400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.379+0000 7f2123fff640 1 -- 172.21.15.120:0/2658637785 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134018020 con 0x7f213c1582d0 2024-09-10T10:28:30.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.379+0000 7f2123fff640 1 -- 172.21.15.120:0/2658637785 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213400ee50 con 0x7f213c1582d0 2024-09-10T10:28:30.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.379+0000 7f2123fff640 1 -- 172.21.15.120:0/2658637785 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340146c0 con 0x7f213c1582d0 2024-09-10T10:28:30.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.379+0000 7f2142037640 1 -- 172.21.15.120:0/2658637785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213c154a80 con 0x7f213c1582d0 2024-09-10T10:28:30.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.380+0000 7f2142037640 1 -- 172.21.15.120:0/2658637785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213c1523a0 con 0x7f213c1582d0 2024-09-10T10:28:30.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.381+0000 7f2123fff640 1 -- 172.21.15.120:0/2658637785 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2134005d10 con 0x7f213c1582d0 2024-09-10T10:28:30.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.381+0000 7f2142037640 1 -- 172.21.15.120:0/2658637785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213c0763a0 con 0x7f213c1582d0 2024-09-10T10:28:30.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.382+0000 7f2123fff640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f212806be30 0x7f212806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.382+0000 7f2123fff640 1 -- 172.21.15.120:0/2658637785 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6983+0+0 (secure 0 0 0) 0x7f213408bc30 con 0x7f213c1582d0 2024-09-10T10:28:30.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.382+0000 7f213b7fe640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f212806be30 0x7f212806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.383+0000 7f213b7fe640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f212806be30 0x7f212806e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f212c034000 tx=0x7f212c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.387+0000 7f2123fff640 1 -- 172.21.15.120:0/2658637785 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f213c0763a0 con 0x7f213c1582d0 2024-09-10T10:28:30.418 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:30.417+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:30.455 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:30.455 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:30.469 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:30.510 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:28:30.520 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:30.519+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:30.621 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:30.620+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:30.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.626+0000 7f2142037640 1 -- 172.21.15.120:0/2658637785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f213c15bb20 con 0x7f213c1582d0 2024-09-10T10:28:30.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.656+0000 7f57f1fe4640 1 Processor -- start 2024-09-10T10:28:30.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.656+0000 7f57f1fe4640 1 -- start start 2024-09-10T10:28:30.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57f1fe4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 0x7f57ec0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57f1fe4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f57ec152aa0 0x7f57ec0c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57eaffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f57ec152aa0 0x7f57ec0c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57eaffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f57ec152aa0 0x7f57ec0c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59274/0 (socket says 172.21.15.120:59274) 2024-09-10T10:28:30.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57eaffd640 1 -- 172.21.15.120:0/664884023 learned_addr learned my addr 172.21.15.120:0/664884023 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/664884023 >> v2:172.21.15.134:3300/0 conn(0x7f57ec0cbab0 0x7f57ec0c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57f1fe4640 1 -- 172.21.15.120:0/664884023 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f57ec15bc50 con 0x7f57ec152aa0 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57f1fe4640 1 -- 172.21.15.120:0/664884023 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f57ec0c9f70 con 0x7f57ec0cbab0 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57f1fe4640 1 -- 172.21.15.120:0/664884023 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f57ec0ca210 con 0x7f57ec1526c0 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57eb7fe640 1 --2- 172.21.15.120:0/664884023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 0x7f57ec0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.657+0000 7f57ebfff640 1 --2- 172.21.15.120:0/664884023 >> v2:172.21.15.134:3300/0 conn(0x7f57ec0cbab0 0x7f57ec0c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.658+0000 7f57eaffd640 1 -- 172.21.15.120:0/664884023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 msgr2=0x7f57ec0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.658+0000 7f57eaffd640 1 --2- 172.21.15.120:0/664884023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 0x7f57ec0c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.658+0000 7f57eaffd640 1 -- 172.21.15.120:0/664884023 >> v2:172.21.15.134:3300/0 conn(0x7f57ec0cbab0 msgr2=0x7f57ec0c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.658+0000 7f57eaffd640 1 --2- 172.21.15.120:0/664884023 >> v2:172.21.15.134:3300/0 conn(0x7f57ec0cbab0 0x7f57ec0c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.658+0000 7f57eaffd640 1 -- 172.21.15.120:0/664884023 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec0ca4d0 con 0x7f57ec152aa0 2024-09-10T10:28:30.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.659+0000 7f57ebfff640 1 --2- 172.21.15.120:0/664884023 >> v2:172.21.15.134:3300/0 conn(0x7f57ec0cbab0 0x7f57ec0c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:30.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.659+0000 7f57eaffd640 1 --2- 172.21.15.120:0/664884023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ec152aa0 0x7f57ec0c3dd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0b4940 tx=0x7f57e002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f9dd4cfe7a1435 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.659+0000 7f57ea7fc640 1 -- 172.21.15.120:0/664884023 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0037070 con 0x7f57ec152aa0 2024-09-10T10:28:30.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.659+0000 7f57ea7fc640 1 -- 172.21.15.120:0/664884023 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e0032440 con 0x7f57ec152aa0 2024-09-10T10:28:30.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.659+0000 7f57ea7fc640 1 -- 172.21.15.120:0/664884023 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0032830 con 0x7f57ec152aa0 2024-09-10T10:28:30.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.660+0000 7f57f1fe4640 1 -- 172.21.15.120:0/664884023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ec152aa0 msgr2=0x7f57ec0c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.660+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/664884023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ec152aa0 0x7f57ec0c3dd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0b4940 tx=0x7f57e002a290 comp rx=0 tx=0).stop 2024-09-10T10:28:30.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.660+0000 7f57f1fe4640 1 -- 172.21.15.120:0/664884023 shutdown_connections 2024-09-10T10:28:30.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.660+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/664884023 >> v2:172.21.15.134:3300/0 conn(0x7f57ec0cbab0 0x7f57ec0c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.660+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/664884023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ec152aa0 0x7f57ec0c3dd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.660+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/664884023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 0x7f57ec0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.660+0000 7f57f1fe4640 1 -- 172.21.15.120:0/664884023 >> 172.21.15.120:0/664884023 conn(0x7f57ec06bc10 msgr2=0x7f57ec06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:30.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.660+0000 7f57f1fe4640 1 -- 172.21.15.120:0/664884023 shutdown_connections 2024-09-10T10:28:30.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.660+0000 7f57f1fe4640 1 -- 172.21.15.120:0/664884023 wait complete. 2024-09-10T10:28:30.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.662+0000 7f57f1fe4640 1 Processor -- start 2024-09-10T10:28:30.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.662+0000 7f57f1fe4640 1 -- start start 2024-09-10T10:28:30.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57f1fe4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57ec0cbab0 0x7f57ec15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57f1fe4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 0x7f57ec0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57f1fe4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ec152aa0 0x7f57ec146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57eb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57ec0cbab0 0x7f57ec15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57eaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 0x7f57ec0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57ebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ec152aa0 0x7f57ec146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57eaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 0x7f57ec0b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:40348/0 (socket says 172.21.15.120:40348) 2024-09-10T10:28:30.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57eaffd640 1 -- 172.21.15.120:0/3480635020 learned_addr learned my addr 172.21.15.120:0/3480635020 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:30.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec0cad70 con 0x7f57ec152aa0 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec0caa90 con 0x7f57ec0cbab0 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f57ec0c9b90 con 0x7f57ec1526c0 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57eaffd640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57ec0cbab0 msgr2=0x7f57ec15acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57eaffd640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57ec0cbab0 0x7f57ec15acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57eaffd640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ec152aa0 msgr2=0x7f57ec146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57eaffd640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ec152aa0 0x7f57ec146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.664+0000 7f57eaffd640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec06e750 con 0x7f57ec1526c0 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.665+0000 7f57ebfff640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ec152aa0 0x7f57ec146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.665+0000 7f57eaffd640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 0x7f57ec0b2070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f57e002a2d0 tx=0x7f57e00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.665+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0041020 con 0x7f57ec1526c0 2024-09-10T10:28:30.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.665+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e0030640 con 0x7f57ec1526c0 2024-09-10T10:28:30.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.665+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0037040 con 0x7f57ec1526c0 2024-09-10T10:28:30.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.665+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ec159c30 con 0x7f57ec1526c0 2024-09-10T10:28:30.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.665+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ec15a1f0 con 0x7f57ec1526c0 2024-09-10T10:28:30.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.666+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f57ec0c8df0 con 0x7f57ec1526c0 2024-09-10T10:28:30.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.667+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57e0044020 con 0x7f57ec1526c0 2024-09-10T10:28:30.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.668+0000 7f57d3fff640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f57d806be30 0x7f57d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.668+0000 7f57eb7fe640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f57d806be30 0x7f57d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.668+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 6983+0+0 (secure 0 0 0) 0x7f57e00abae0 con 0x7f57ec1526c0 2024-09-10T10:28:30.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.668+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v53) ==== 54+0+16 (secure 0 0 0) 0x7f57e004b030 con 0x7f57ec1526c0 2024-09-10T10:28:30.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.668+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f57ec145960 0x7f57ec0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.668+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57ec162c00 con 0x7f57ec145960 2024-09-10T10:28:30.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.668+0000 7f57eb7fe640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f57d806be30 0x7f57d806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009ac0 tx=0x7f57dc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.669+0000 7f57ebfff640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f57ec145960 0x7f57ec0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.669+0000 7f57ebfff640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f57ec145960 0x7f57ec0b40a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.672 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:30.672 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:30.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.670+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57ec162c00 con 0x7f57ec145960 2024-09-10T10:28:30.680 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:30.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.698+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57ec162920 con 0x7f57ec145960 2024-09-10T10:28:30.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.701+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57ec162920 con 0x7f57ec145960 2024-09-10T10:28:30.702 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:30.703 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:30.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.701+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f57ec144c50 0x7f57ec145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.701+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f57ec145570 con 0x7f57ec144c50 2024-09-10T10:28:30.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.701+0000 7f57eb7fe640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f57ec144c50 0x7f57ec145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.702+0000 7f57eb7fe640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f57ec144c50 0x7f57ec145030 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.702+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57ec145570 con 0x7f57ec144c50 2024-09-10T10:28:30.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.717+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57ec162360 con 0x7f57ec144c50 2024-09-10T10:28:30.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.719+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57ec162360 con 0x7f57ec144c50 2024-09-10T10:28:30.721 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:30.721 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:30.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.719+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f57ec062b40 0x7f57ec062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.719+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f57ec0cf240 con 0x7f57ec062b40 2024-09-10T10:28:30.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.720+0000 7f57eaffd640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f57ec062b40 0x7f57ec062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.722 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:30.720+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:30.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.720+0000 7f57eaffd640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f57ec062b40 0x7f57ec062f20 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.721+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57ec0cf240 con 0x7f57ec062b40 2024-09-10T10:28:30.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.735+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57ec0cf240 con 0x7f57ec062b40 2024-09-10T10:28:30.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.737+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57ec0cf240 con 0x7f57ec062b40 2024-09-10T10:28:30.738 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:30.739 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:30.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.737+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f57ec06d6c0 0x7f57ec06dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.737+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f57ec0cf240 con 0x7f57ec06d6c0 2024-09-10T10:28:30.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.738+0000 7f57ebfff640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f57ec06d6c0 0x7f57ec06dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.738+0000 7f57ebfff640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f57ec06d6c0 0x7f57ec06dae0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.738+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57ec0cf240 con 0x7f57ec06d6c0 2024-09-10T10:28:30.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.752+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57ec0cf240 con 0x7f57ec06d6c0 2024-09-10T10:28:30.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.754+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57ec0cf240 con 0x7f57ec06d6c0 2024-09-10T10:28:30.755 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:30.756 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:30.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.754+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f57ec158ed0 0x7f57ec159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.754+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f57ec159870 con 0x7f57ec158ed0 2024-09-10T10:28:30.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.754+0000 7f57eb7fe640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f57ec158ed0 0x7f57ec159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.755+0000 7f57eb7fe640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f57ec158ed0 0x7f57ec159330 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.756+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57ec159870 con 0x7f57ec158ed0 2024-09-10T10:28:30.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.782+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57ec159870 con 0x7f57ec158ed0 2024-09-10T10:28:30.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.784+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57ec159870 con 0x7f57ec158ed0 2024-09-10T10:28:30.786 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:30.786 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:30.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.784+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f57ec02cb50 0x7f57ec02cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.784+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f57ec02d730 con 0x7f57ec02cb50 2024-09-10T10:28:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.785+0000 7f57eaffd640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f57ec02cb50 0x7f57ec02cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.785+0000 7f57eaffd640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f57ec02cb50 0x7f57ec02cfb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.786+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57ec02d730 con 0x7f57ec02cb50 2024-09-10T10:28:30.797 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:30.797 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:30.797 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:30.797 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:30.807 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:30.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.813+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57ec02d700 con 0x7f57ec02cb50 2024-09-10T10:28:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.815+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57ec02d700 con 0x7f57ec02cb50 2024-09-10T10:28:30.816 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.815+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f57ec2073a0 0x7f57ec209790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.815+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f57ec20c7c0 con 0x7f57ec2073a0 2024-09-10T10:28:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.815+0000 7f57ebfff640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f57ec2073a0 0x7f57ec209790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.816+0000 7f57ebfff640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f57ec2073a0 0x7f57ec209790 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.816+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57ec20c7c0 con 0x7f57ec2073a0 2024-09-10T10:28:30.822 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:30.821+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:30.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.843+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57ec20b500 con 0x7f57ec2073a0 2024-09-10T10:28:30.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.845+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57ec20b500 con 0x7f57ec2073a0 2024-09-10T10:28:30.846 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:30.847 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:30.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.845+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f57ec20bc20 0x7f57ec20c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:30.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.845+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f57ec20fc10 con 0x7f57ec20bc20 2024-09-10T10:28:30.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.845+0000 7f57eb7fe640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f57ec20bc20 0x7f57ec20c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:30.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.846+0000 7f57eb7fe640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f57ec20bc20 0x7f57ec20c080 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:30.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.847+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57ec20fc10 con 0x7f57ec20bc20 2024-09-10T10:28:30.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.873+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57ec20ea10 con 0x7f57ec20bc20 2024-09-10T10:28:30.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.875+0000 7f57d3fff640 1 -- 172.21.15.120:0/3480635020 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57ec20ea10 con 0x7f57ec20bc20 2024-09-10T10:28:30.877 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:30.878 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:30.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f57ec145960 msgr2=0x7f57ec0b40a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f57ec145960 0x7f57ec0b40a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f57ec144c50 msgr2=0x7f57ec145030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f57ec144c50 0x7f57ec145030 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f57ec062b40 msgr2=0x7f57ec062f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f57ec062b40 0x7f57ec062f20 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f57ec06d6c0 msgr2=0x7f57ec06dae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f57ec06d6c0 0x7f57ec06dae0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f57ec158ed0 msgr2=0x7f57ec159330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f57ec158ed0 0x7f57ec159330 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f57ec02cb50 msgr2=0x7f57ec02cfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f57ec02cb50 0x7f57ec02cfb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f57ec2073a0 msgr2=0x7f57ec209790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f57ec2073a0 0x7f57ec209790 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f57ec20bc20 msgr2=0x7f57ec20c080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f57ec20bc20 0x7f57ec20c080 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f57d806be30 msgr2=0x7f57d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f57d806be30 0x7f57d806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009ac0 tx=0x7f57dc009190 comp rx=0 tx=0).stop 2024-09-10T10:28:30.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 msgr2=0x7f57ec0b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:30.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 --2- 172.21.15.120:0/3480635020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57ec1526c0 0x7f57ec0b2070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f57e002a2d0 tx=0x7f57e00046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:30.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57ebfff640 1 -- 172.21.15.120:0/3480635020 reap_dead start 2024-09-10T10:28:30.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57ebfff640 1 -- 172.21.15.120:0/3480635020 reap_dead start 2024-09-10T10:28:30.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57ebfff640 1 -- 172.21.15.120:0/3480635020 reap_dead start 2024-09-10T10:28:30.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57ebfff640 1 -- 172.21.15.120:0/3480635020 reap_dead start 2024-09-10T10:28:30.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57ebfff640 1 -- 172.21.15.120:0/3480635020 reap_dead start 2024-09-10T10:28:30.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 shutdown_connections 2024-09-10T10:28:30.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 >> 172.21.15.120:0/3480635020 conn(0x7f57ec06bc10 msgr2=0x7f57ec1567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:30.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 shutdown_connections 2024-09-10T10:28:30.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:30.876+0000 7f57f1fe4640 1 -- 172.21.15.120:0/3480635020 wait complete. 2024-09-10T10:28:30.912 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:30.912 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:30.912 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:30.912 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:30.912 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:30.921 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:30.933 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:30.932+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:31.023 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:31.022+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:31.026 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.24534959499999, 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047203150000000001, 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779674+0000", 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0359999999999995e-06 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779702+0000", 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.7970000000000002e-05 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779724+0000", 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2544e-05 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779741+0000", 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6781000000000001e-05 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779981+0000", 2024-09-10T10:28:31.030 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000239764 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780147+0000", 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016605100000000001 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017096200000000001 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.031 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.032 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781749+0000", 2024-09-10T10:28:31.032 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001431288 2024-09-10T10:28:31.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.032 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.032 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784234+0000", 2024-09-10T10:28:31.032 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024850940000000002 2024-09-10T10:28:31.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784338+0000", 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104078 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784385+0000", 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6746999999999998e-05 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.033 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.1a epoch 24/18 rep_tid 2 entries 24'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.243871485, 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021396129999999998, 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781144+0000", 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.393e-06 2024-09-10T10:28:31.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781156+0000", 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1514e-05 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8445999999999998e-05 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781202+0000", 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7439e-05 2024-09-10T10:28:31.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781286+0000", 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4652000000000002e-05 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783282+0000", 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0019961689999999999 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:31 2.d e24/18 2:b50e409b:::mds_snaptable:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.24295199399999, 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030643699999999999, 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782063+0000", 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3699999999999999e-07 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782072+0000", 2024-09-10T10:28:31.039 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9430000000000006e-06 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782083+0000", 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0998e-05 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782092+0000", 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.3270000000000001e-06 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.040 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782158+0000", 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6830999999999998e-05 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782176+0000", 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7370000000000001e-05 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785091+0000", 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029157879999999999 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785127+0000", 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5176000000000003e-05 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.242893103, 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002013515, 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.042 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782122+0000", 2024-09-10T10:28:31.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.229e-06 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782137+0000", 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4895999999999999e-05 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782151+0000", 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3229e-05 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.044 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782198+0000", 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7301e-05 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782271+0000", 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3516000000000006e-05 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782290+0000", 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8921000000000001e-05 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.045 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784094+0000", 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00180331 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784135+0000", 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1112999999999998e-05 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:28 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.24278149, 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028487970000000001, 2024-09-10T10:28:31.046 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7000000000000005e-07 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782247+0000", 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3740000000000001e-05 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782258+0000", 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0552e-05 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782265+0000", 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0700000000000001e-06 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782752+0000", 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048707399999999999 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782771+0000", 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9046999999999999e-05 2024-09-10T10:28:31.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785037+0000", 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002265806 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785082+0000", 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4938000000000002e-05 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.050 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:29 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 104.241235495, 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001932907, 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783780+0000", 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9900000000000009e-07 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783789+0000", 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0140000000000004e-06 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783802+0000", 2024-09-10T10:28:31.052 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2751e-05 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783811+0000", 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3950000000000007e-06 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783894+0000", 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2887999999999997e-05 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.053 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783913+0000", 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9052999999999999e-05 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785668+0000", 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001754633 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785712+0000", 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4174000000000001e-05 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:7 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.152603455000005, 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026422899999999998, 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.561e-06 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872426+0000", 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3577999999999999e-05 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872462+0000", 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6145e-05 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872473+0000", 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1109e-05 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872543+0000", 2024-09-10T10:28:31.057 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9319000000000005e-05 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872566+0000", 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3768999999999999e-05 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875010+0000", 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024433910000000001 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.058 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875053+0000", 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3417999999999997e-05 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:8 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.151915066000001, 2024-09-10T10:28:31.059 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0044428820000000004, 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873100+0000", 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3030000000000005e-06 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873121+0000", 2024-09-10T10:28:31.061 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1471000000000001e-05 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873129+0000", 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7339999999999997e-06 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048786499999999998 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873638+0000", 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0735000000000001e-05 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877493+0000", 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0038550189999999999 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877542+0000", 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9135000000000003e-05 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.063 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:17 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.151697733000006, 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042663060000000001, 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873318+0000", 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7999999999999993e-07 2024-09-10T10:28:31.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873326+0000", 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8279999999999992e-06 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873341+0000", 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4688999999999999e-05 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873348+0000", 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4810000000000003e-06 2024-09-10T10:28:31.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873843+0000", 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000495083 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873860+0000", 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7609999999999999e-05 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877545+0000", 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036848839999999998 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877583+0000", 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7750999999999998e-05 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.068 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:18 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.150975696000003, 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035807780000000002, 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.070 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874045+0000", 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9909999999999999e-06 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874055+0000", 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0147e-05 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874070+0000", 2024-09-10T10:28:31.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4184e-05 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874123+0000", 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3504999999999997e-05 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874141+0000", 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7524999999999999e-05 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877586+0000", 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034458700000000002 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877619+0000", 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2962000000000001e-05 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.18 epoch 26/18 rep_tid 5 entries 26'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.9:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.149914050999996, 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048872359999999997, 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:31.074 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875101+0000", 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.075 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875110+0000", 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2609999999999999e-06 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875129+0000", 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8685000000000002e-05 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875143+0000", 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3723000000000001e-05 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875222+0000", 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9498000000000004e-05 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879987+0000", 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047652800000000002 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:7 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'5, mlcod=26'3)", 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.075151754000004, 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031425310000000001, 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1100000000000005e-07 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949871+0000", 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8620000000000003e-06 2024-09-10T10:28:31.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949885+0000", 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4072e-05 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949893+0000", 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5769999999999996e-06 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949953+0000", 2024-09-10T10:28:31.080 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.978e-05 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949972+0000", 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9188e-05 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.952959+0000", 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029873859999999999 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.081 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953005+0000", 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5855000000000001e-05 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:8 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'6, mlcod=26'3)", 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.075099164999997, 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034236359999999999, 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949916+0000", 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6199999999999998e-07 2024-09-10T10:28:31.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949922+0000", 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5390000000000004e-06 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949929+0000", 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4339999999999997e-06 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949934+0000", 2024-09-10T10:28:31.084 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2159999999999997e-06 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950192+0000", 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00025864699999999998 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950213+0000", 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0761999999999999e-05 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953297+0000", 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030842999999999999 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953339+0000", 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1276000000000003e-05 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.086 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:34 2.11 e26/18 2:8975f766:::202.00000001:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.071126325999998, 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020619190000000002, 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953889+0000", 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.879999999999999e-07 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953895+0000", 2024-09-10T10:28:31.088 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2249999999999997e-06 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953906+0000", 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953912+0000", 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4840000000000003e-06 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.089 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953959+0000", 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7768000000000001e-05 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953976+0000", 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6365999999999999e-05 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955917+0000", 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001941524 2024-09-10T10:28:31.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955950+0000", 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2573999999999997e-05 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:4 2.1a e26/18 2:5fdc322d:::61e.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.102058798000002, 2024-09-10T10:28:31.091 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0018931340000000001, 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:31.092 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922956+0000", 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-07 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922962+0000", 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1890000000000004e-06 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.093 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922971+0000", 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922978+0000", 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7830000000000001e-06 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923032+0000", 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3637999999999997e-05 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923049+0000", 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7249999999999999e-05 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924803+0000", 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001753788 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924849+0000", 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6050000000000001e-05 2024-09-10T10:28:31.095 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:20 2.11 e26/18 2:890f50f7:::626.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.101951091999993, 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003328012, 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923064+0000", 2024-09-10T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9900000000000001e-07 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923070+0000", 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7499999999999997e-06 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923079+0000", 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9080000000000001e-06 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923089+0000", 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0078000000000001e-05 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923150+0000", 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0300999999999997e-05 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923507+0000", 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00035713599999999998 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.099 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926368+0000", 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028615469999999999 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926391+0000", 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2793e-05 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:23 2.18 e26/18 2:1c1db442:::103.00000000.inode:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:31.100 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.101891623, 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032793050000000002, 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.101 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923124+0000", 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0599999999999999e-07 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923130+0000", 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.81e-06 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923141+0000", 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923149+0000", 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2679999999999997e-06 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4744000000000002e-05 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923233+0000", 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9190000000000001e-05 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926353+0000", 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031206419999999999 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926402+0000", 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8755000000000003e-05 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.104 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:13 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'6, mlcod=26'2)", 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.101838993000001, 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031818969999999999, 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.105 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923176+0000", 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4799999999999998e-07 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923182+0000", 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.592e-06 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923191+0000", 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4029999999999996e-06 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.107 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923197+0000", 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1850000000000001e-06 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923230+0000", 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3290000000000001e-05 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923247+0000", 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6810999999999999e-05 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926308+0000", 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030606159999999999 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926357+0000", 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9351999999999998e-05 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.109 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.d epoch 26/18 rep_tid 20 entries 26'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.15:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.101286494999997, 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034028819999999999, 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.110 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923729+0000", 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.5900000000000005e-07 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923739+0000", 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8439999999999993e-06 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923761+0000", 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2821999999999999e-05 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923776+0000", 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4763000000000001e-05 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923889+0000", 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000112518 2024-09-10T10:28:31.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927131+0000", 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032419760000000001 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:14 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'7, mlcod=26'2)", 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.100571533999997, 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027184890000000001, 2024-09-10T10:28:31.113 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5099999999999999e-07 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924449+0000", 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0310000000000004e-06 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924482+0000", 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2845999999999997e-05 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924500+0000", 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.787e-05 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924594+0000", 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3566999999999994e-05 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924619+0000", 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4978999999999999e-05 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927118+0000", 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024995479999999999 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927161+0000", 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2997000000000002e-05 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.117 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:31.119 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:31.160 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:31.159+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:31.225 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:31.224+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:31.264 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:31.264 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:31.264 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:31.264 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:31.273 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:31.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.302+0000 7f2123fff640 1 -- 172.21.15.120:0/2658637785 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v54) ==== 91+0+0 (secure 0 0 0) 0x7f2134022070 con 0x7f213c1582d0 2024-09-10T10:28:31.304 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-10T10:28:31.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 -- 172.21.15.120:0/2658637785 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f212806be30 msgr2=0x7f212806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:31.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f212806be30 0x7f212806e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f212c034000 tx=0x7f212c02e000 comp rx=0 tx=0).stop 2024-09-10T10:28:31.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 -- 172.21.15.120:0/2658637785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f213c1582d0 msgr2=0x7f213c158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:31.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f213c1582d0 0x7f213c158b10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f213400ec40 tx=0x7f213400c490 comp rx=0 tx=0).stop 2024-09-10T10:28:31.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 -- 172.21.15.120:0/2658637785 shutdown_connections 2024-09-10T10:28:31.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f212806be30 0x7f212806e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:31.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f213c1582d0 0x7f213c158b10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:31.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f213c157ef0 0x7f213c0c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:31.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 --2- 172.21.15.120:0/2658637785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f213c072e00 0x7f213c0c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:31.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 -- 172.21.15.120:0/2658637785 >> 172.21.15.120:0/2658637785 conn(0x7f213c0703e0 msgr2=0x7f213c074ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:31.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 -- 172.21.15.120:0/2658637785 shutdown_connections 2024-09-10T10:28:31.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:31.307+0000 7f2142037640 1 -- 172.21.15.120:0/2658637785 wait complete. 2024-09-10T10:28:31.365 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:31.364+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:31.378 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:31.378 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:31.387 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:31.427 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:31.426+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:31.492 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:31.492 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:31.492 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.618179683999998, 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.006807553, 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:31.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872290+0000", 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2029999999999999e-06 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872310+0000", 2024-09-10T10:28:31.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0508000000000001e-05 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872331+0000", 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0577e-05 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872341+0000", 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0421999999999999e-05 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872457+0000", 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115618 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872575+0000", 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118563 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872690+0000", 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011438900000000001 2024-09-10T10:28:31.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875281+0000", 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002590995 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879035+0000", 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037544879999999998 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879069+0000", 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4034e-05 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879095+0000", 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5755999999999999e-05 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.498 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.616983063000006, 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0061030149999999998, 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.499 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873485+0000", 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2419999999999999e-06 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.669e-06 2024-09-10T10:28:31.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873505+0000", 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.325e-05 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873514+0000", 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1949999999999992e-06 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873575+0000", 2024-09-10T10:28:31.501 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873647+0000", 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2346000000000002e-05 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873778+0000", 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013125600000000001 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875444+0000", 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001665962 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879528+0000", 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040837809999999999 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.503 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879563+0000", 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5329e-05 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879587+0000", 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3947999999999998e-05 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.616929013000004, 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096274789999999996, 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.505 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873539+0000", 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0299999999999998e-07 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873554+0000", 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5481999999999998e-05 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.506 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873565+0000", 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0733e-05 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873573+0000", 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9270000000000005e-06 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873633+0000", 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9864000000000001e-05 2024-09-10T10:28:31.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873752+0000", 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118946 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873866+0000", 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000113757 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.508 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875462+0000", 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015966699999999999 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883084+0000", 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076214280000000004 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883136+0000", 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2182000000000003e-05 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.509 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883166+0000", 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.9787000000000001e-05 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.616871042, 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097094669999999994, 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.510 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.511 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873597+0000", 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8899999999999998e-07 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873602+0000", 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9230000000000001e-06 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873611+0000", 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9150000000000002e-06 2024-09-10T10:28:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5239999999999997e-06 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874719+0000", 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0011012769999999999 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874791+0000", 2024-09-10T10:28:31.513 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2305000000000007e-05 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874962+0000", 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000171099 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877786+0000", 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028241059999999998 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.514 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883211+0000", 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054250019999999999 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883263+0000", 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1563000000000001e-05 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883306+0000", 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2864000000000002e-05 2024-09-10T10:28:31.515 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.615950351999999, 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005164681, 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874518+0000", 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3799999999999996e-07 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874521+0000", 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8550000000000004e-06 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874531+0000", 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8079999999999999e-06 2024-09-10T10:28:31.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874538+0000", 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2760000000000003e-06 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874945+0000", 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000406516 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875045+0000", 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9570999999999999e-05 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879682+0000", 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046369169999999999 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.520 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 82.615916456999997, 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056651770000000004, 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4600000000000005e-07 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874565+0000", 2024-09-10T10:28:31.522 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4017e-05 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874573+0000", 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9689999999999994e-06 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.523 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874678+0000", 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7483000000000002e-05 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874786+0000", 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107914 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.880216+0000", 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005429499 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.524 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:24 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.541608134000001, 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0079888200000000006, 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.525 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948861+0000", 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9269999999999999e-06 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948869+0000", 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7749999999999993e-06 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948884+0000", 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5713e-05 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948894+0000", 2024-09-10T10:28:31.527 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2050000000000008e-06 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948958+0000", 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4398000000000002e-05 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949093+0000", 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013508300000000001 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:31.528 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949223+0000", 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012949500000000001 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953128+0000", 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039058360000000002 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956775+0000", 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036467439999999999 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.529 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956812+0000", 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6510000000000001e-05 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956848+0000", 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6134000000000003e-05 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.530 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:23 2.c e26/18 2:3631e767:::102.00000000.inode:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 77.540123883999996, 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054176620000000002, 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.531 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950344+0000", 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9400000000000001e-07 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950362+0000", 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8655999999999999e-05 2024-09-10T10:28:31.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950384+0000", 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1929999999999998e-05 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950391+0000", 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7020000000000004e-06 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950624+0000", 2024-09-10T10:28:31.533 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000233031 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950643+0000", 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9459000000000001e-05 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955726+0000", 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050826389999999999 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.534 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4851000000000002e-05 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.569474898999999, 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0070980829999999998, 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.535 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.536 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920994+0000", 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.697e-06 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921001+0000", 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3719999999999999e-06 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921014+0000", 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2529e-05 2024-09-10T10:28:31.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921021+0000", 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9519999999999997e-06 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921097+0000", 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6499000000000001e-05 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921261+0000", 2024-09-10T10:28:31.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016399599999999999 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921396+0000", 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013448400000000001 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922936+0000", 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015405970000000001 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927988+0000", 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00505158 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928047+0000", 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9011000000000001e-05 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928090+0000", 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3365999999999998e-05 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.540 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.569423388000004, 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007356853, 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.541 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5699999999999998e-07 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.542 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921048+0000", 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.754e-06 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921056+0000", 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0469999999999994e-06 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921063+0000", 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5980000000000002e-06 2024-09-10T10:28:31.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921747+0000", 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00068475800000000005 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921852+0000", 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104347 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:31.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922002+0000", 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000150233 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924357+0000", 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0023545599999999999 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928286+0000", 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039299540000000003 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.545 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928349+0000", 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2044000000000004e-05 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928401+0000", 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2101000000000001e-05 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 71.569383930000001, 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005625166, 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921084+0000", 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0100000000000005e-07 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921093+0000", 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9910000000000008e-06 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921099+0000", 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2269999999999998e-06 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921103+0000", 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0929999999999996e-06 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921167+0000", 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4030999999999997e-05 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921301+0000", 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013388699999999999 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921471+0000", 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017000899999999999 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924553+0000", 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030818199999999999 2024-09-10T10:28:31.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926600+0000", 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020471339999999999 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926663+0000", 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3274000000000004e-05 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926708+0000", 2024-09-10T10:28:31.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5099000000000003e-05 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 65.546832260000002, 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010012200000000001, 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943637+0000", 2024-09-10T10:28:31.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5390000000000002e-06 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943646+0000", 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1340000000000004e-06 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943661+0000", 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4958e-05 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943669+0000", 2024-09-10T10:28:31.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5629999999999992e-06 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943785+0000", 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011584800000000001 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943960+0000", 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000174611 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:31.555 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944078+0000", 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118381 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948022+0000", 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039442599999999998 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953546+0000", 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055242449999999997 2024-09-10T10:28:31.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953604+0000", 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7555e-05 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953647+0000", 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3105999999999997e-05 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 65.546638701000006, 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099609380000000008, 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943829+0000", 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3099999999999996e-07 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943835+0000", 2024-09-10T10:28:31.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8610000000000003e-06 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943846+0000", 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0686999999999999e-05 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943854+0000", 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944413+0000", 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055893399999999997 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944505+0000", 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2788999999999995e-05 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944639+0000", 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013387699999999999 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950701+0000", 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0060612360000000002 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953700+0000", 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029997689999999998 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953743+0000", 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3024e-05 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953789+0000", 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6109999999999997e-05 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 65.546598678999999, 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054304089999999998, 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:31.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1399999999999996e-07 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943874+0000", 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3830000000000002e-06 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943881+0000", 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1369999999999996e-06 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943886+0000", 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-06 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944057+0000", 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000170781 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944182+0000", 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012506299999999999 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944309+0000", 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127081 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.947998+0000", 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003689323 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949225+0000", 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001227005 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949267+0000", 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1391999999999999e-05 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949299+0000", 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2169999999999999e-05 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.568 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 65.546538197000004, 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0068819399999999996, 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943930+0000", 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1200000000000002e-07 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943934+0000", 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0969999999999999e-06 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944782+0000", 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00084778299999999996 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944793+0000", 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0993e-05 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944864+0000", 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1241000000000005e-05 2024-09-10T10:28:31.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944940+0000", 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6044999999999994e-05 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945068+0000", 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012846 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949683+0000", 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046151040000000001 2024-09-10T10:28:31.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950706+0000", 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010222149999999999 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950764+0000", 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8180999999999997e-05 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.573 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950811+0000", 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7108999999999999e-05 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:24 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 65.545164571000001, 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055052950000000003, 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.574 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945304+0000", 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4799999999999997e-07 2024-09-10T10:28:31.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945311+0000", 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1269999999999997e-06 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945322+0000", 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1134e-05 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945330+0000", 2024-09-10T10:28:31.576 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9039999999999992e-06 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945408+0000", 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8200000000000003e-05 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945533+0000", 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000125518 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:31.577 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945659+0000", 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012583599999999999 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949147+0000", 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003487675 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950716+0000", 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015695279999999999 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950767+0000", 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0577e-05 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950808+0000", 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0847999999999999e-05 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:25 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 65.544523201999993, 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050290129999999997, 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:31.580 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945945+0000", 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1119999999999999e-06 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.581 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945952+0000", 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5189999999999998e-06 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945965+0000", 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3643e-05 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945974+0000", 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5750000000000001e-06 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0532999999999998e-05 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946192+0000", 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015717699999999999 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946341+0000", 2024-09-10T10:28:31.583 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014897200000000001 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949885+0000", 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035449729999999999 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950896+0000", 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00101023 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.584 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950938+0000", 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2284999999999999e-05 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950973+0000", 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4993999999999997e-05 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 65.544470129999993, 2024-09-10T10:28:31.585 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048920780000000002, 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.586 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945998+0000", 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3900000000000004e-07 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946012+0000", 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4799e-05 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.587 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946021+0000", 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.6000000000000007e-06 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946029+0000", 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9209999999999992e-06 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946093+0000", 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3608000000000005e-05 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.588 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946242+0000", 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014889699999999999 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950889+0000", 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046476140000000004 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.589 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 65.544426103000006, 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047976679999999997, 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946042+0000", 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5199999999999996e-07 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946046+0000", 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1439999999999996e-06 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946054+0000", 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1140000000000006e-06 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946070+0000", 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6232e-05 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946137+0000", 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6956000000000001e-05 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.592 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946260+0000", 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012240299999999999 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950839+0000", 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045791670000000003 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.593 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:23 2.1c e28/18 2:399c8d31:::104.00000000.inode:head v 28'5, mlcod=24'1)", 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 65.543906763999999, 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042418079999999997, 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.594 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3e-07 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946574+0000", 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.277e-05 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.595 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946584+0000", 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7710000000000001e-06 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946590+0000", 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6769999999999998e-06 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946666+0000", 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5420999999999995e-05 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946687+0000", 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1665999999999999e-05 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950762+0000", 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040749030000000004 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950802+0000", 2024-09-10T10:28:31.597 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9969999999999998e-05 2024-09-10T10:28:31.598 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.598 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.598 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.598 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.598 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.598 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:31.599 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:31.600 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:31.598+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:31.700 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:31.699+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:31.743 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:31.743 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:31.743 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:31.743 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:31.752 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:31.801 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:31.800+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:31.858 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:31.858 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:31.858 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:31.858 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:31.859 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:31.867 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:31.911 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:31.910+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:31.972 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:31.972 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:31.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:31.972 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:31.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.972 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:31.972 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:31.972 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 105.190183648, 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:31.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:28:31.974 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:28:31.975 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.976 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.977 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 105.189853148, 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.979 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:28:31.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:28:31.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 105.189806605, 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.983 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:31.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:28:31.986 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 83.098364392999997, 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.987 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.988 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:28:31.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:28:31.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 83.098283050000006, 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.992 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.993 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:31.994 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:31.995 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:31.996 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:31.997 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 83.098256495000001, 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:31.998 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:31.999 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.000 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.001 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:28:32.002 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 83.098113846000004, 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.003 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:28:32.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:28:32.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:28:32.007 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:32.008 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 83.095466912999996, 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:32.009 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:28:32.010 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.011 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.012 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.013 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:32.013 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:32.013 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 83.094616173000006, 2024-09-10T10:28:32.013 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:28:32.013 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.016 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.016 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.016 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.016 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:28:32.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:28:32.018 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.019 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.020 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.022464490000004, 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.021 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:28:32.022 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.024 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.022415354000003, 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.026 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.027 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:28:32.029 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.030 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:28:32.031 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 78.022303875999995, 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.034 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 72.048903357, 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.036 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:28:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.039 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:28:32.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:28:32.041 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 72.048873185999994, 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.042 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.044 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:28:32.046 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.047 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 72.048816693999996, 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.048 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.049 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:28:32.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:28:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 72.048715525999995, 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:32.054 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.055 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:28:32.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:28:32.058 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:32.059 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.028448428999994, 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.060 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:28:32.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:28:32.064 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.024383209999996, 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:32.067 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.068 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:28:32.069 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.024365340000003, 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.072 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.024319102000007, 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.074 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.075 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:28:32.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.078 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:32.080 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:32.080 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:32.079+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:32.181 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:32.180+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:32.224 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:32.224 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:32.225 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:32.225 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:32.233 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:32.282 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:32.281+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:32.339 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:32.339 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:32.339 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:32.339 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:32.339 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:32.348 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:32.383 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:32.382+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:32.453 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:32.453 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:32.453 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:32.453 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:32.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 120.35469651299999, 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.17492211699999999, 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 1 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096318+0000", 2024-09-10T10:28:32.455 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.657e-06 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096326+0000", 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0560000000000005e-06 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096339+0000", 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2965999999999999e-05 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096349+0000", 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0784e-05 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096432+0000", 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2657999999999994e-05 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op must wait for map", 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096441+0000", 2024-09-10T10:28:32.457 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4590000000000004e-06 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096935+0000", 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049331899999999998 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for peered", 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096944+0000", 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9539999999999993e-06 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.458 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267505+0000", 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.170561622 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267700+0000", 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019420400000000001 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267923+0000", 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000223451 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.270575+0000", 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026522030000000001 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271131+0000", 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055599000000000002 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271191+0000", 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9947000000000001e-05 2024-09-10T10:28:32.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271238+0000", 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6847000000000001e-05 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.965862269, 2024-09-10T10:28:32.461 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078035889999999997, 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 115 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1300000000000001e-07 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485209+0000", 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8628999999999997e-05 2024-09-10T10:28:32.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485214+0000", 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0849999999999996e-06 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485218+0000", 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3840000000000001e-06 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490185+0000", 2024-09-10T10:28:32.464 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0049675270000000002 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490231+0000", 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5488000000000002e-05 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490307+0000", 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6038000000000006e-05 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.465 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491263+0000", 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095625099999999998 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492908+0000", 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001645355 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492934+0000", 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5431000000000001e-05 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492954+0000", 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0088000000000001e-05 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.96578778, 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077741099999999999, 2024-09-10T10:28:32.467 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 116 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1100000000000002e-07 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485227+0000", 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1890000000000001e-06 2024-09-10T10:28:32.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485231+0000", 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4010000000000001e-06 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485233+0000", 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.728e-06 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490628+0000", 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0053952109999999996 2024-09-10T10:28:32.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490674+0000", 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5946999999999999e-05 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490750+0000", 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5227000000000003e-05 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.471 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491443+0000", 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069346800000000004 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492959+0000", 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015154470000000001 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492983+0000", 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.406e-05 2024-09-10T10:28:32.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492999+0000", 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6121e-05 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:32.473 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.965775652, 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078059490000000004, 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 117 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.474 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.23e-07 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.475 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485240+0000", 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780000000000001e-06 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485243+0000", 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8430000000000003e-06 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485247+0000", 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2279999999999998e-06 2024-09-10T10:28:32.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490995+0000", 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057480730000000002 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491043+0000", 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7936000000000003e-05 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491113+0000", 2024-09-10T10:28:32.477 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0872999999999997e-05 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491636+0000", 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00052242300000000005 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493002+0000", 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001365808 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.478 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493025+0000", 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2932999999999999e-05 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493043+0000", 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8031000000000001e-05 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.479 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.63342050999999, 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095639920000000003, 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 131 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.480 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1199999999999998e-07 2024-09-10T10:28:32.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817663+0000", 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1099000000000005e-05 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817670+0000", 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.8539999999999999e-06 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.482 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817674+0000", 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.574e-06 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818465+0000", 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000791445 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818509+0000", 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3569000000000002e-05 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818580+0000", 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1503999999999998e-05 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821292+0000", 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027120529999999999 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827108+0000", 2024-09-10T10:28:32.484 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0058160759999999999 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827134+0000", 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5859000000000001e-05 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827156+0000", 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1546999999999999e-05 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.485 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.633331302, 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095198769999999995, 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 132 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.486 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2899999999999999e-07 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817683+0000", 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0379999999999998e-06 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817690+0000", 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3110000000000002e-06 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817699+0000", 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820828+0000", 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031296829999999999 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820873+0000", 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4869e-05 2024-09-10T10:28:32.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820943+0000", 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9919999999999995e-05 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823683+0000", 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027395140000000002 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827161+0000", 2024-09-10T10:28:32.490 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003477857 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827183+0000", 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2612999999999998e-05 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827201+0000", 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7842999999999998e-05 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.491 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.63326399499999, 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0094944009999999995, 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 133 2024-09-10T10:28:32.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.493 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817749+0000", 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2399999999999998e-07 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817752+0000", 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1970000000000001e-06 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817760+0000", 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8080000000000005e-06 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817764+0000", 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9419999999999997e-06 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821319+0000", 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035547740000000001 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821364+0000", 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5398000000000001e-05 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821434+0000", 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0382999999999993e-05 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823717+0000", 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022825089999999998 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827205+0000", 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034879009999999998 2024-09-10T10:28:32.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827225+0000", 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0611999999999998e-05 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827243+0000", 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7353000000000001e-05 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.63321115399999, 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095863660000000007, 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 134 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817802+0000", 2024-09-10T10:28:32.499 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8999999999999996e-07 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817871+0000", 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9572000000000004e-05 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817878+0000", 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2079999999999997e-06 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.500 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817882+0000", 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5889999999999996e-06 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821625+0000", 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037431069999999999 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821669+0000", 2024-09-10T10:28:32.501 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3455000000000003e-05 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821740+0000", 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1390999999999995e-05 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823826+0000", 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020863399999999999 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827343+0000", 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003516267 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827369+0000", 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.6044999999999999e-05 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827388+0000", 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8802000000000001e-05 2024-09-10T10:28:32.503 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.633118459, 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095738179999999996, 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 135 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5999999999999999e-07 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817897+0000", 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0149999999999999e-06 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817904+0000", 2024-09-10T10:28:32.506 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5089999999999999e-06 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817909+0000", 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3120000000000001e-06 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823862+0000", 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059524199999999999 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.507 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823911+0000", 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9323000000000002e-05 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823983+0000", 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1736000000000006e-05 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827101+0000", 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031181009999999999 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.508 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827392+0000", 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00029090700000000003 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827426+0000", 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4127e-05 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.509 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827468+0000", 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1907999999999999e-05 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.633091443, 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.009594488, 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.510 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 136 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:32.511 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3700000000000001e-07 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817925+0000", 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4659999999999999e-06 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817931+0000", 2024-09-10T10:28:32.512 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5710000000000003e-06 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817936+0000", 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7550000000000002e-06 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824178+0000", 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062415700000000001 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.513 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824225+0000", 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6966999999999999e-05 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824296+0000", 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1284000000000004e-05 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827290+0000", 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002993697 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.514 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827473+0000", 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018376699999999999 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827497+0000", 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3445e-05 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827515+0000", 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8529000000000002e-05 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.515 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.633059754, 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096066220000000004, 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.516 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 137 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3000000000000001e-07 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818004+0000", 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0902000000000002e-05 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818013+0000", 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5420000000000005e-06 2024-09-10T10:28:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818019+0000", 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7939999999999997e-06 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824489+0000", 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0064692580000000003 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824532+0000", 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3631000000000003e-05 2024-09-10T10:28:32.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824601+0000", 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9028000000000004e-05 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827334+0000", 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027325409999999998 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827520+0000", 2024-09-10T10:28:32.520 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018642499999999999 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827541+0000", 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1151000000000001e-05 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827559+0000", 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7819999999999999e-05 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.632980433, 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010100303, 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 138 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818033+0000", 2024-09-10T10:28:32.523 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818037+0000", 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0419999999999996e-06 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818045+0000", 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9759999999999995e-06 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.524 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818052+0000", 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5909999999999998e-06 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826594+0000", 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0085421660000000003 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826671+0000", 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7121999999999998e-05 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826795+0000", 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012315199999999999 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828081+0000", 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0012869139999999999 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828088+0000", 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9870000000000002e-06 2024-09-10T10:28:32.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828112+0000", 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3405e-05 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828132+0000", 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0394000000000002e-05 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.63295006200001, 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01125819, 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 139 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.528 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818063+0000", 2024-09-10T10:28:32.529 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7800000000000002e-07 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818067+0000", 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8500000000000004e-06 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818075+0000", 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.530 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818081+0000", 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1829999999999999e-06 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827594+0000", 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095129340000000007 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827669+0000", 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5239000000000002e-05 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.531 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827789+0000", 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000119757 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829195+0000", 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014064279999999999 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829232+0000", 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7058999999999999e-05 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.532 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829281+0000", 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8884000000000002e-05 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829321+0000", 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9527999999999998e-05 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.632915993, 2024-09-10T10:28:32.533 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012263886, 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 140 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818097+0000", 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8999999999999997e-07 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818145+0000", 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8040000000000001e-05 2024-09-10T10:28:32.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818160+0000", 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4671999999999999e-05 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818169+0000", 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1900000000000001e-06 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828158+0000", 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099891440000000001 2024-09-10T10:28:32.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828228+0000", 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0407000000000005e-05 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828343+0000", 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000114821 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.537 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829914+0000", 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001570866 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830312+0000", 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00039770499999999999 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830339+0000", 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.69e-05 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.538 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830360+0000", 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1651e-05 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.63282875199999, 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012742035000000001, 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.539 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 141 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7700000000000004e-07 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818188+0000", 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8659999999999999e-06 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818196+0000", 2024-09-10T10:28:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9480000000000008e-06 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818205+0000", 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8570000000000001e-06 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828883+0000", 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010678600999999999 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.542 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828927+0000", 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3887999999999997e-05 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829001+0000", 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3442000000000006e-05 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830548+0000", 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001547194 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.543 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830884+0000", 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00033638100000000002 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830907+0000", 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3051999999999999e-05 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830926+0000", 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8229000000000001e-05 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.544 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.619681015, 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097270880000000001, 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:32.545 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 145 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831332+0000", 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8900000000000001e-07 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831379+0000", 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7134e-05 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831387+0000", 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7409999999999992e-06 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831391+0000", 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6190000000000002e-06 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833155+0000", 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017641969999999999 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833237+0000", 2024-09-10T10:28:32.548 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1957000000000002e-05 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833356+0000", 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011843100000000001 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.834068+0000", 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00071179000000000001 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841004+0000", 2024-09-10T10:28:32.549 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069359749999999996 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841034+0000", 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.042e-05 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841058+0000", 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4535000000000001e-05 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:20 2.9 e26/18 2:92e99a1d:::612.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 83.575484751000005, 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074964979999999999, 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875534+0000", 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9109999999999998e-06 2024-09-10T10:28:32.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875543+0000", 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8319999999999995e-06 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875549+0000", 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2380000000000002e-06 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875699+0000", 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014945900000000001 2024-09-10T10:28:32.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875717+0000", 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8584e-05 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882982+0000", 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0072648039999999997 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883024+0000", 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2135000000000002e-05 2024-09-10T10:28:32.554 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:27 2.9 e26/18 2:91bd965c:::501.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 83.575424116999997, 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074993129999999996, 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.555 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875589+0000", 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3999999999999997e-07 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875595+0000", 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9680000000000003e-06 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875602+0000", 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875608+0000", 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0430000000000004e-06 2024-09-10T10:28:32.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875887+0000", 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000279578 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875904+0000", 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6439e-05 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883027+0000", 2024-09-10T10:28:32.558 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0071234480000000001 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883088+0000", 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0424000000000003e-05 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.559 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.509345039999999, 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01193195, 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2399999999999999e-07 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941671+0000", 2024-09-10T10:28:32.561 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8329999999999998e-06 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2919999999999998e-06 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941686+0000", 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.562 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944874+0000", 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031881639999999998 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945305+0000", 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00043084599999999997 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945438+0000", 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013344600000000001 2024-09-10T10:28:32.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950653+0000", 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005214733 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953496+0000", 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028433989999999999 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953554+0000", 2024-09-10T10:28:32.564 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7933000000000003e-05 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953599+0000", 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5130999999999999e-05 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 66.509319094000006, 2024-09-10T10:28:32.565 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0093592050000000006, 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.566 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941694+0000", 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7899999999999999e-07 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941711+0000", 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7530999999999999e-05 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941720+0000", 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4279999999999996e-06 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.567 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941725+0000", 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2390000000000002e-06 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942690+0000", 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00096541400000000005 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942807+0000", 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000116777 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942967+0000", 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015997599999999999 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944708+0000", 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001740492 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950956+0000", 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062481580000000002 2024-09-10T10:28:32.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951016+0000", 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0299e-05 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951052+0000", 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6412000000000003e-05 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:32.570 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:32.571 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:32.571 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:28:32.571 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:28:32.571 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-10T10:28:32.571 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 1.000000 2024-09-10T10:28:32.571 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-10T10:28:32.573 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:32.571+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:32.573 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:32.673 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:32.672+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:32.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.729+0000 7f1381b6d640 1 Processor -- start 2024-09-10T10:28:32.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.729+0000 7f1381b6d640 1 -- start start 2024-09-10T10:28:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.731+0000 7f1381b6d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 0x7f137c0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.731+0000 7f1381b6d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f137c0d0630 0x7f137c0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.731+0000 7f1381b6d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f137c15ba20 con 0x7f137c15e660 2024-09-10T10:28:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.731+0000 7f1381b6d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f137c15e1d0 con 0x7f137c0d0630 2024-09-10T10:28:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.731+0000 7f1381b6d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f137c15aea0 con 0x7f137c0cc0a0 2024-09-10T10:28:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.731+0000 7f137affd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f137c0d0630 0x7f137c0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.731+0000 7f137b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 0x7f137c0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.731+0000 7f137bfff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f137c15e660 0x7f137c0cd0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:54378/0 (socket says 172.21.15.120:54378) 2024-09-10T10:28:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.731+0000 7f137bfff640 1 -- 172.21.15.120:0/1049153114 learned_addr learned my addr 172.21.15.120:0/1049153114 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.732+0000 7f137a7fc640 1 -- 172.21.15.120:0/1049153114 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763802972 0 0) 0x7f137c15ba20 con 0x7f137c15e660 2024-09-10T10:28:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.732+0000 7f137a7fc640 1 -- 172.21.15.120:0/1049153114 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1364003520 con 0x7f137c15e660 2024-09-10T10:28:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.732+0000 7f137b7fe640 1 -- 172.21.15.120:0/1049153114 >> v2:172.21.15.134:3300/0 conn(0x7f137c0d0630 msgr2=0x7f137c0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.732+0000 7f137b7fe640 1 --2- 172.21.15.120:0/1049153114 >> v2:172.21.15.134:3300/0 conn(0x7f137c0d0630 0x7f137c0d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.732+0000 7f137b7fe640 1 -- 172.21.15.120:0/1049153114 >> v1:172.21.15.134:6789/0 conn(0x7f137c15e660 legacy=0x7f137c0cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.732+0000 7f137a7fc640 1 -- 172.21.15.120:0/1049153114 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2130946503 0 0) 0x7f1364003520 con 0x7f137c15e660 2024-09-10T10:28:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.732+0000 7f137b7fe640 1 -- 172.21.15.120:0/1049153114 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c15b180 con 0x7f137c0cc0a0 2024-09-10T10:28:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.732+0000 7f137affd640 1 --2- 172.21.15.120:0/1049153114 >> v2:172.21.15.134:3300/0 conn(0x7f137c0d0630 0x7f137c0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.733+0000 7f137b7fe640 1 --2- 172.21.15.120:0/1049153114 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 0x7f137c0cc480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f136c0097a0 tx=0x7f136c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c77aad81705a1f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.733+0000 7f137a7fc640 1 -- 172.21.15.120:0/1049153114 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c037070 con 0x7f137c0cc0a0 2024-09-10T10:28:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.733+0000 7f137a7fc640 1 -- 172.21.15.120:0/1049153114 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c02ac30 con 0x7f137c0cc0a0 2024-09-10T10:28:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.733+0000 7f137a7fc640 1 -- 172.21.15.120:0/1049153114 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13740019f0 con 0x7f137c0cc0a0 2024-09-10T10:28:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.734+0000 7f1381b6d640 1 -- 172.21.15.120:0/1049153114 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 msgr2=0x7f137c0cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.734+0000 7f1381b6d640 1 --2- 172.21.15.120:0/1049153114 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 0x7f137c0cc480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f136c0097a0 tx=0x7f136c02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:28:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.734+0000 7f1381b6d640 1 -- 172.21.15.120:0/1049153114 shutdown_connections 2024-09-10T10:28:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.734+0000 7f1381b6d640 1 --2- 172.21.15.120:0/1049153114 >> v2:172.21.15.134:3300/0 conn(0x7f137c0d0630 0x7f137c0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.734+0000 7f1381b6d640 1 --2- 172.21.15.120:0/1049153114 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 0x7f137c0cc480 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.734+0000 7f1381b6d640 1 -- 172.21.15.120:0/1049153114 >> 172.21.15.120:0/1049153114 conn(0x7f137c06d560 msgr2=0x7f137c073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.734+0000 7f1381b6d640 1 -- 172.21.15.120:0/1049153114 shutdown_connections 2024-09-10T10:28:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.734+0000 7f1381b6d640 1 -- 172.21.15.120:0/1049153114 wait complete. 2024-09-10T10:28:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.736+0000 7f1381b6d640 1 Processor -- start 2024-09-10T10:28:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.736+0000 7f1381b6d640 1 -- start start 2024-09-10T10:28:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f1381b6d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 0x7f137c151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f1381b6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f137c0d0630 0x7f137c1f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f1381b6d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c15e660 0x7f137c1f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f137affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f137c0d0630 0x7f137c1f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f137affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f137c0d0630 0x7f137c1f3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59292/0 (socket says 172.21.15.120:59292) 2024-09-10T10:28:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f137b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 0x7f137c151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f137b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 0x7f137c151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:40366/0 (socket says 172.21.15.120:40366) 2024-09-10T10:28:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f137affd640 1 -- 172.21.15.120:0/2485536053 learned_addr learned my addr 172.21.15.120:0/2485536053 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f137c0cbde0 con 0x7f137c0d0630 2024-09-10T10:28:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f137bfff640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c15e660 0x7f137c1f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:32.740 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:32.740 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:32.740 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:32.741 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f137c15b740 con 0x7f137c15e660 2024-09-10T10:28:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.738+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f137c15aaf0 con 0x7f137c0cc0a0 2024-09-10T10:28:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.739+0000 7f137affd640 1 -- 172.21.15.120:0/2485536053 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 msgr2=0x7f137c151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.739+0000 7f137affd640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 0x7f137c151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.739+0000 7f137affd640 1 -- 172.21.15.120:0/2485536053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c15e660 msgr2=0x7f137c1f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.739+0000 7f137affd640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c15e660 0x7f137c1f1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.739+0000 7f137affd640 1 -- 172.21.15.120:0/2485536053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c1f4450 con 0x7f137c0d0630 2024-09-10T10:28:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.739+0000 7f137affd640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f137c0d0630 0x7f137c1f3b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f137000ebb0 tx=0x7f137000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.740+0000 7f1363fff640 1 -- 172.21.15.120:0/2485536053 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370017070 con 0x7f137c0d0630 2024-09-10T10:28:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.740+0000 7f1363fff640 1 -- 172.21.15.120:0/2485536053 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137000cc30 con 0x7f137c0d0630 2024-09-10T10:28:32.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.740+0000 7f1363fff640 1 -- 172.21.15.120:0/2485536053 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370013470 con 0x7f137c0d0630 2024-09-10T10:28:32.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.740+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137c1f4730 con 0x7f137c0d0630 2024-09-10T10:28:32.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.740+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137c0cf590 con 0x7f137c0d0630 2024-09-10T10:28:32.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.741+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137c0cf790 con 0x7f137c0d0630 2024-09-10T10:28:32.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.742+0000 7f1363fff640 1 -- 172.21.15.120:0/2485536053 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f137001a070 con 0x7f137c0d0630 2024-09-10T10:28:32.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.743+0000 7f1363fff640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f136806be80 0x7f136806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:32.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.743+0000 7f1363fff640 1 -- 172.21.15.120:0/2485536053 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6983+0+0 (secure 0 0 0) 0x7f137008b4a0 con 0x7f137c0d0630 2024-09-10T10:28:32.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.743+0000 7f137b7fe640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f136806be80 0x7f136806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:32.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.744+0000 7f137b7fe640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f136806be80 0x7f136806e340 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f136c034000 tx=0x7f136c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:32.753 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:32.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:32.752+0000 7f1363fff640 1 -- 172.21.15.120:0/2485536053 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f13700593e0 con 0x7f137c0d0630 2024-09-10T10:28:32.775 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:32.773+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:32.875 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:32.874+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:32.892 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:28:32.920 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:32.920 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:32.920 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:32.920 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:32.920 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:32.934 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:32.977 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:32.976+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:33.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.008+0000 7f1ec1847640 1 Processor -- start 2024-09-10T10:28:33.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.008+0000 7f1ec1847640 1 -- start start 2024-09-10T10:28:33.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.010+0000 7f1ec1847640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1ebc159100 0x7f1ebc15f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.010+0000 7f1ec1847640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1ebc0c8500 0x7f1ebc071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.010+0000 7f1ec1847640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ebc070060 con 0x7f1ebc072100 2024-09-10T10:28:33.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.010+0000 7f1ec1847640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1ebc0c8900 con 0x7f1ebc0c8500 2024-09-10T10:28:33.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.010+0000 7f1ec1847640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1ebc06ece0 con 0x7f1ebc159100 2024-09-10T10:28:33.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.010+0000 7f1ebb7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f1ebc072100 0x7f1ebc0c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41742/0 (socket says 172.21.15.120:41742) 2024-09-10T10:28:33.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.010+0000 7f1ebb7fe640 1 -- 172.21.15.120:0/1875508467 learned_addr learned my addr 172.21.15.120:0/1875508467 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:33.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.010+0000 7f1eb3fff640 1 --2- 172.21.15.120:0/1875508467 >> v2:172.21.15.120:3300/0 conn(0x7f1ebc0c8500 0x7f1ebc071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.011+0000 7f1eb3fff640 1 -- 172.21.15.120:0/1875508467 >> v2:172.21.15.134:3300/0 conn(0x7f1ebc159100 msgr2=0x7f1ebc15f8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:28:33.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.011+0000 7f1eb3fff640 1 --2- 172.21.15.120:0/1875508467 >> v2:172.21.15.134:3300/0 conn(0x7f1ebc159100 0x7f1ebc15f8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.011+0000 7f1eb3fff640 1 -- 172.21.15.120:0/1875508467 >> v1:172.21.15.120:6789/0 conn(0x7f1ebc072100 legacy=0x7f1ebc0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:33.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.011+0000 7f1eb3fff640 1 -- 172.21.15.120:0/1875508467 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ebc06efc0 con 0x7f1ebc0c8500 2024-09-10T10:28:33.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.012+0000 7f1eb3fff640 1 --2- 172.21.15.120:0/1875508467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 0x7f1ebc071bc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc074240 tx=0x7f1ea402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b5a91c120eec3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.012+0000 7f1eba7fc640 1 -- 172.21.15.120:0/1875508467 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4037070 con 0x7f1ebc0c8500 2024-09-10T10:28:33.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.012+0000 7f1eba7fc640 1 -- 172.21.15.120:0/1875508467 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea4032440 con 0x7f1ebc0c8500 2024-09-10T10:28:33.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.012+0000 7f1eba7fc640 1 -- 172.21.15.120:0/1875508467 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4032810 con 0x7f1ebc0c8500 2024-09-10T10:28:33.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.013+0000 7f1ec1847640 1 -- 172.21.15.120:0/1875508467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 msgr2=0x7f1ebc071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.013+0000 7f1ec1847640 1 --2- 172.21.15.120:0/1875508467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 0x7f1ebc071bc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc074240 tx=0x7f1ea402a1c0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.013+0000 7f1ec1847640 1 -- 172.21.15.120:0/1875508467 shutdown_connections 2024-09-10T10:28:33.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.013+0000 7f1ec1847640 1 --2- 172.21.15.120:0/1875508467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 0x7f1ebc071bc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.013+0000 7f1ec1847640 1 --2- 172.21.15.120:0/1875508467 >> v2:172.21.15.134:3300/0 conn(0x7f1ebc159100 0x7f1ebc15f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.013+0000 7f1ec1847640 1 -- 172.21.15.120:0/1875508467 >> 172.21.15.120:0/1875508467 conn(0x7f1ebc0cc380 msgr2=0x7f1ebc0cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:33.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.014+0000 7f1ec1847640 1 -- 172.21.15.120:0/1875508467 shutdown_connections 2024-09-10T10:28:33.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.014+0000 7f1ec1847640 1 -- 172.21.15.120:0/1875508467 wait complete. 2024-09-10T10:28:33.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.016+0000 7f1ec1847640 1 Processor -- start 2024-09-10T10:28:33.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.016+0000 7f1ec1847640 1 -- start start 2024-09-10T10:28:33.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.017+0000 7f1ec1847640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc072100 0x7f1ebc153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.017+0000 7f1ec1847640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 0x7f1ebc1f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1eb3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 0x7f1ebc1f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1ebaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc072100 0x7f1ebc153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1eb3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 0x7f1ebc1f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59304/0 (socket says 172.21.15.120:59304) 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1ebaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc072100 0x7f1ebc153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:34474/0 (socket says 172.21.15.120:34474) 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1eb3fff640 1 -- 172.21.15.120:0/4194928734 learned_addr learned my addr 172.21.15.120:0/4194928734 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1ec1847640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ebc159100 0x7f1ebc14b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1ec1847640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc06f860 con 0x7f1ebc0c8500 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1ec1847640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc06f580 con 0x7f1ebc072100 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1ec1847640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1ebc06fb40 con 0x7f1ebc159100 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1ebb7fe640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ebc159100 0x7f1ebc14b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1eb3fff640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ebc159100 msgr2=0x7f1ebc14b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1eb3fff640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ebc159100 0x7f1ebc14b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1eb3fff640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc072100 msgr2=0x7f1ebc153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1eb3fff640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc072100 0x7f1ebc153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.018+0000 7f1eb3fff640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ebc14be70 con 0x7f1ebc0c8500 2024-09-10T10:28:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.019+0000 7f1eb3fff640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 0x7f1ebc1f5940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009890 tx=0x7f1ea40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.019+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4041070 con 0x7f1ebc0c8500 2024-09-10T10:28:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.019+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea403c950 con 0x7f1ebc0c8500 2024-09-10T10:28:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.019+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4037040 con 0x7f1ebc0c8500 2024-09-10T10:28:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.019+0000 7f1ec1847640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ebc1e8e00 con 0x7f1ebc0c8500 2024-09-10T10:28:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.019+0000 7f1ec1847640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ebc1e8840 con 0x7f1ebc0c8500 2024-09-10T10:28:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.020+0000 7f1ec1847640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f1ebc1594e0 con 0x7f1ebc0c8500 2024-09-10T10:28:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.022+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ea4044070 con 0x7f1ebc0c8500 2024-09-10T10:28:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.023+0000 7f1eb8ff9640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ea006be80 0x7f1ea006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.023+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6983+0+0 (secure 0 0 0) 0x7f1ea40ac3f0 con 0x7f1ebc0c8500 2024-09-10T10:28:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.023+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v54) ==== 54+0+16 (secure 0 0 0) 0x7f1ebc14be70 con 0x7f1ebc0c8500 2024-09-10T10:28:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.024+0000 7f1ebaffd640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ea006be80 0x7f1ea006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.024+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f1ebc0c3180 0x7f1ebc1f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.024+0000 7f1ebb7fe640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f1ebc0c3180 0x7f1ebc1f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.024+0000 7f1ebaffd640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ea006be80 0x7f1ea006e340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009980 tx=0x7f1eac009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.025+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ebc0c80d0 con 0x7f1ebc0c3180 2024-09-10T10:28:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.025+0000 7f1ebb7fe640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f1ebc0c3180 0x7f1ebc1f8ee0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.026+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ebc0c80d0 con 0x7f1ebc0c3180 2024-09-10T10:28:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.054+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1ebc070300 con 0x7f1ebc0c3180 2024-09-10T10:28:33.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.059+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1ebc070300 con 0x7f1ebc0c3180 2024-09-10T10:28:33.061 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:33.061 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:33.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.060+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f1ebc157e80 0x7f1ebc158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.060+0000 7f1ebaffd640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f1ebc157e80 0x7f1ebc158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.061+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f1ebc1587a0 con 0x7f1ebc157e80 2024-09-10T10:28:33.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.061+0000 7f1ebaffd640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f1ebc157e80 0x7f1ebc158260 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.062+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ebc1587a0 con 0x7f1ebc157e80 2024-09-10T10:28:33.076 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:33.075+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.090+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1ebc0c8bb0 con 0x7f1ebc157e80 2024-09-10T10:28:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.093+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1ebc0c8bb0 con 0x7f1ebc157e80 2024-09-10T10:28:33.094 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.093+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f1ebc062f00 0x7f1ebc0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.093+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f1ebc1eb470 con 0x7f1ebc062f00 2024-09-10T10:28:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.093+0000 7f1eb3fff640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f1ebc062f00 0x7f1ebc0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.094+0000 7f1eb3fff640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f1ebc062f00 0x7f1ebc0ca160 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.095+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ebc1eb470 con 0x7f1ebc062f00 2024-09-10T10:28:33.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.123+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1ebc063550 con 0x7f1ebc062f00 2024-09-10T10:28:33.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.124+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} v 0) -- 0x7f137c155140 con 0x7f137c0d0630 2024-09-10T10:28:33.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.125+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1ebc063550 con 0x7f1ebc062f00 2024-09-10T10:28:33.126 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:33.127 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:33.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.126+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f1ebc0caf70 0x7f1ebc0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.126+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f1ebc068e50 con 0x7f1ebc0caf70 2024-09-10T10:28:33.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.126+0000 7f1ebb7fe640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f1ebc0caf70 0x7f1ebc0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.126+0000 7f1ebb7fe640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f1ebc0caf70 0x7f1ebc0cb350 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.127+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ebc068e50 con 0x7f1ebc0caf70 2024-09-10T10:28:33.139 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:33.139 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:33.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.141+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1ebc068e50 con 0x7f1ebc0caf70 2024-09-10T10:28:33.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.143+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1ebc068e50 con 0x7f1ebc0caf70 2024-09-10T10:28:33.145 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:33.145 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:33.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.144+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f1ebc207cd0 0x7f1ebc20a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.144+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f1ebc068e50 con 0x7f1ebc207cd0 2024-09-10T10:28:33.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.144+0000 7f1ebaffd640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f1ebc207cd0 0x7f1ebc20a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.145+0000 7f1ebaffd640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f1ebc207cd0 0x7f1ebc20a0c0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.145+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ebc068e50 con 0x7f1ebc207cd0 2024-09-10T10:28:33.151 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:33.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.159+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1ebc068e50 con 0x7f1ebc207cd0 2024-09-10T10:28:33.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.161+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1ebc068e50 con 0x7f1ebc207cd0 2024-09-10T10:28:33.162 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:33.163 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:33.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.161+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f1ebc20ab10 0x7f1ebc20cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.161+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f1ebc20fcd0 con 0x7f1ebc20ab10 2024-09-10T10:28:33.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.162+0000 7f1eb3fff640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f1ebc20ab10 0x7f1ebc20cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.162+0000 7f1eb3fff640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f1ebc20ab10 0x7f1ebc20cf00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.163+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ebc20fcd0 con 0x7f1ebc20ab10 2024-09-10T10:28:33.177 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:33.176+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:33.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.178+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1ebc20d440 con 0x7f1ebc20ab10 2024-09-10T10:28:33.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.180+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1ebc20d440 con 0x7f1ebc20ab10 2024-09-10T10:28:33.181 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:33.182 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:33.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.181+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f1ebc20dac0 0x7f1ebc20feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.181+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f1ebc212e60 con 0x7f1ebc20dac0 2024-09-10T10:28:33.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.181+0000 7f1ebb7fe640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f1ebc20dac0 0x7f1ebc20feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.182+0000 7f1ebb7fe640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f1ebc20dac0 0x7f1ebc20feb0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.183+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ebc212e60 con 0x7f1ebc20dac0 2024-09-10T10:28:33.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.200+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1ebc211b80 con 0x7f1ebc20dac0 2024-09-10T10:28:33.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.202+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1ebc211b80 con 0x7f1ebc20dac0 2024-09-10T10:28:33.203 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:33.204 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:33.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.202+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f1ebc2122a0 0x7f1ebc2126e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.202+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f1ebc216230 con 0x7f1ebc2122a0 2024-09-10T10:28:33.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.203+0000 7f1ebaffd640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f1ebc2122a0 0x7f1ebc2126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.203+0000 7f1ebaffd640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f1ebc2122a0 0x7f1ebc2126e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.204+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ebc216230 con 0x7f1ebc2122a0 2024-09-10T10:28:33.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.230+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1ebc2150b0 con 0x7f1ebc2122a0 2024-09-10T10:28:33.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.232+0000 7f1eb8ff9640 1 -- 172.21.15.120:0/4194928734 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1ebc2150b0 con 0x7f1ebc2122a0 2024-09-10T10:28:33.234 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f1ebc0c3180 msgr2=0x7f1ebc1f8ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f1ebc0c3180 0x7f1ebc1f8ee0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f1ebc157e80 msgr2=0x7f1ebc158260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f1ebc157e80 0x7f1ebc158260 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f1ebc062f00 msgr2=0x7f1ebc0ca160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f1ebc062f00 0x7f1ebc0ca160 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1ebb7fe640 1 -- 172.21.15.120:0/4194928734 reap_dead start 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f1ebc0caf70 msgr2=0x7f1ebc0cb350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f1ebc0caf70 0x7f1ebc0cb350 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f1ebc207cd0 msgr2=0x7f1ebc20a0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f1ebc207cd0 0x7f1ebc20a0c0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f1ebc20ab10 msgr2=0x7f1ebc20cf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f1ebc20ab10 0x7f1ebc20cf00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f1ebc20dac0 msgr2=0x7f1ebc20feb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f1ebc20dac0 0x7f1ebc20feb0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f1ebc2122a0 msgr2=0x7f1ebc2126e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f1ebc2122a0 0x7f1ebc2126e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.233+0000 7f1ebb7fe640 1 -- 172.21.15.120:0/4194928734 reap_dead start 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.234+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ea006be80 msgr2=0x7f1ea006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.234+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ea006be80 0x7f1ea006e340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009980 tx=0x7f1eac009060 comp rx=0 tx=0).stop 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.234+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 msgr2=0x7f1ebc1f5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.234+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 0x7f1ebc1f5940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009890 tx=0x7f1ea40046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.234+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 shutdown_connections 2024-09-10T10:28:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.234+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ea006be80 0x7f1ea006e340 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.234+0000 7f1eb1ffb640 1 --2- 172.21.15.120:0/4194928734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc0c8500 0x7f1ebc1f5940 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.234+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 >> 172.21.15.120:0/4194928734 conn(0x7f1ebc0cc380 msgr2=0x7f1ebc0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:33.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.234+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 shutdown_connections 2024-09-10T10:28:33.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.234+0000 7f1eb1ffb640 1 -- 172.21.15.120:0/4194928734 wait complete. 2024-09-10T10:28:33.278 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:33.276+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:33.283 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:33.283 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:33.293 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:33.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.305+0000 7f1363fff640 1 -- 172.21.15.120:0/2485536053 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]=0 set osd.5 primary-affinity to 1 (8655362) v55) ==== 133+0+0 (secure 0 0 0) 0x7f137005d6e0 con 0x7f137c0d0630 2024-09-10T10:28:33.306 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-10T10:28:33.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.308+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f136806be80 msgr2=0x7f136806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.308+0000 7f1381b6d640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f136806be80 0x7f136806e340 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f136c034000 tx=0x7f136c02e000 comp rx=0 tx=0).stop 2024-09-10T10:28:33.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.308+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f137c0d0630 msgr2=0x7f137c1f3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.308+0000 7f1381b6d640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f137c0d0630 0x7f137c1f3b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f137000ebb0 tx=0x7f137000c520 comp rx=0 tx=0).stop 2024-09-10T10:28:33.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.309+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 shutdown_connections 2024-09-10T10:28:33.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.309+0000 7f1381b6d640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f136806be80 0x7f136806e340 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.309+0000 7f1381b6d640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c15e660 0x7f137c1f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.309+0000 7f1381b6d640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f137c0d0630 0x7f137c1f3b90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.309+0000 7f1381b6d640 1 --2- 172.21.15.120:0/2485536053 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c0cc0a0 0x7f137c151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.309+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 >> 172.21.15.120:0/2485536053 conn(0x7f137c06d560 msgr2=0x7f137c0cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.309+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 shutdown_connections 2024-09-10T10:28:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.310+0000 7f1381b6d640 1 -- 172.21.15.120:0/2485536053 wait complete. 2024-09-10T10:28:33.324 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-10T10:28:33.378 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:33.377+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:33.422 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:33.422 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:33.431 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:33.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.476+0000 7fd0a8ee6640 1 Processor -- start 2024-09-10T10:28:33.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.476+0000 7fd0a8ee6640 1 -- start start 2024-09-10T10:28:33.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.477+0000 7fd0a8ee6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd0a40d0520 0x7fd0a40b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.477+0000 7fd0a8ee6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd0a40d0900 0x7fd0a41628d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.477+0000 7fd0a8ee6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a415f140 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.477+0000 7fd0a8ee6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a415f420 con 0x7fd0a40d0520 2024-09-10T10:28:33.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.477+0000 7fd0a8ee6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a415f700 con 0x7fd0a40d0900 2024-09-10T10:28:33.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.477+0000 7fd0a259c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd0a40d0520 0x7fd0a40b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.477+0000 7fd0a259c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd0a40d0520 0x7fd0a40b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59320/0 (socket says 172.21.15.120:59320) 2024-09-10T10:28:33.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.477+0000 7fd0a259c640 1 -- 172.21.15.120:0/1797820045 learned_addr learned my addr 172.21.15.120:0/1797820045 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:33.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.477+0000 7fd0a1d9b640 1 --2- 172.21.15.120:0/1797820045 >> v2:172.21.15.134:3300/0 conn(0x7fd0a40d0900 0x7fd0a41628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.478+0000 7fd0a259c640 1 -- 172.21.15.120:0/1797820045 >> v2:172.21.15.134:3300/0 conn(0x7fd0a40d0900 msgr2=0x7fd0a41628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:33.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.478+0000 7fd0a259c640 1 --2- 172.21.15.120:0/1797820045 >> v2:172.21.15.134:3300/0 conn(0x7fd0a40d0900 0x7fd0a41628d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.478+0000 7fd0a259c640 1 -- 172.21.15.120:0/1797820045 >> v1:172.21.15.134:6789/0 conn(0x7fd0a40cc2a0 legacy=0x7fd0a4158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.478+0000 7fd0a259c640 1 -- 172.21.15.120:0/1797820045 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a415f9e0 con 0x7fd0a40d0520 2024-09-10T10:28:33.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.478+0000 7fd0a1d9b640 1 --2- 172.21.15.120:0/1797820045 >> v2:172.21.15.134:3300/0 conn(0x7fd0a40d0900 0x7fd0a41628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:33.480 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:33.477+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.479+0000 7fd0a259c640 1 --2- 172.21.15.120:0/1797820045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40d0520 0x7fd0a40b4240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd0a415cbb0 tx=0x7fd09402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbd3e49a5964bb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.479+0000 7fd0a159a640 1 -- 172.21.15.120:0/1797820045 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094036070 con 0x7fd0a40d0520 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.479+0000 7fd0a159a640 1 -- 172.21.15.120:0/1797820045 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd094031600 con 0x7fd0a40d0520 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.479+0000 7fd0a159a640 1 -- 172.21.15.120:0/1797820045 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0940319d0 con 0x7fd0a40d0520 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.480+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/1797820045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40d0520 msgr2=0x7fd0a40b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.480+0000 7fd0a8ee6640 1 --2- 172.21.15.120:0/1797820045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40d0520 0x7fd0a40b4240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd0a415cbb0 tx=0x7fd09402a560 comp rx=0 tx=0).stop 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.480+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/1797820045 shutdown_connections 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.480+0000 7fd0a8ee6640 1 --2- 172.21.15.120:0/1797820045 >> v2:172.21.15.134:3300/0 conn(0x7fd0a40d0900 0x7fd0a41628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.480+0000 7fd0a8ee6640 1 --2- 172.21.15.120:0/1797820045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40d0520 0x7fd0a40b4240 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.480+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/1797820045 >> 172.21.15.120:0/1797820045 conn(0x7fd0a406ca90 msgr2=0x7fd0a406e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.480+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/1797820045 shutdown_connections 2024-09-10T10:28:33.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.481+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/1797820045 wait complete. 2024-09-10T10:28:33.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.482+0000 7fd0a8ee6640 1 Processor -- start 2024-09-10T10:28:33.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.482+0000 7fd0a8ee6640 1 -- start start 2024-09-10T10:28:33.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a8ee6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40cc2a0 0x7fd0a406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a8ee6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40d0520 0x7fd0a4072150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a8ee6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40d0900 0x7fd0a4072690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a8ee6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a40b4780 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a8ee6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a40cbde0 con 0x7fd0a40d0520 2024-09-10T10:28:33.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a8ee6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a41592f0 con 0x7fd0a40d0900 2024-09-10T10:28:33.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40cc2a0 0x7fd0a406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40cc2a0 0x7fd0a406f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59322/0 (socket says 172.21.15.120:59322) 2024-09-10T10:28:33.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a259c640 1 -- 172.21.15.120:0/180599455 learned_addr learned my addr 172.21.15.120:0/180599455 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:33.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a1d9b640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40d0520 0x7fd0a4072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.483+0000 7fd0a2d9d640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40d0900 0x7fd0a4072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.484+0000 7fd0a259c640 1 -- 172.21.15.120:0/180599455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40d0900 msgr2=0x7fd0a4072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.484+0000 7fd0a259c640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40d0900 0x7fd0a4072690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.484+0000 7fd0a259c640 1 -- 172.21.15.120:0/180599455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40d0520 msgr2=0x7fd0a4072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:33.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.484+0000 7fd0a259c640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40d0520 0x7fd0a4072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:33.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.484+0000 7fd0a259c640 1 -- 172.21.15.120:0/180599455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a40c5d40 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.484+0000 7fd0a259c640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40cc2a0 0x7fd0a406f4c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0940090b0 tx=0x7fd0940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.484+0000 7fd092ffd640 1 -- 172.21.15.120:0/180599455 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094040070 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.484+0000 7fd092ffd640 1 -- 172.21.15.120:0/180599455 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd094031be0 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.484+0000 7fd092ffd640 1 -- 172.21.15.120:0/180599455 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094036040 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.484+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/180599455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a40c6000 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.485+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/180599455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a40c65c0 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.486+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/180599455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a4159680 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.486+0000 7fd092ffd640 1 -- 172.21.15.120:0/180599455 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd094043070 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.487+0000 7fd092ffd640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd08406be60 0x7fd08406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:33.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.487+0000 7fd092ffd640 1 -- 172.21.15.120:0/180599455 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7015+0+0 (secure 0 0 0) 0x7fd0940abb30 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.487+0000 7fd0a1d9b640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd08406be60 0x7fd08406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:33.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.488+0000 7fd0a1d9b640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd08406be60 0x7fd08406e320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40ccb40 tx=0x7fd098009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:33.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.496+0000 7fd092ffd640 1 -- 172.21.15.120:0/180599455 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd094079a00 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.553 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:33.553 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:33.562 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:33.606 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:33.605+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:33.668 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:33.668 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:33.678 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:33.680 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:33.679+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:33.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:33.682+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/180599455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fd0a4158c60 con 0x7fd0a40cc2a0 2024-09-10T10:28:33.781 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:33.780+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:33.790 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:33.790 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:33.801 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:33.885 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:33.884+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:33.949 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:33.950 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:33.959 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:33.984 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:33.982+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:34.085 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:34.084+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:34.105 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:34.105 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:34.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:34.199 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:34.198+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:34.264 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:34.264 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:34.273 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:34.315 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:34.314+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:34.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.318+0000 7fd092ffd640 1 -- 172.21.15.120:0/180599455 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v56) ==== 101+0+0 (secure 0 0 0) 0x7fd09407dd00 con 0x7fd0a40cc2a0 2024-09-10T10:28:34.319 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-10T10:28:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.322+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/180599455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd08406be60 msgr2=0x7fd08406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.322+0000 7fd0a8ee6640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd08406be60 0x7fd08406e320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40ccb40 tx=0x7fd098009190 comp rx=0 tx=0).stop 2024-09-10T10:28:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.322+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/180599455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40cc2a0 msgr2=0x7fd0a406f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.322+0000 7fd0a8ee6640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40cc2a0 0x7fd0a406f4c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0940090b0 tx=0x7fd0940046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.322+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/180599455 shutdown_connections 2024-09-10T10:28:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.322+0000 7fd0a8ee6640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd08406be60 0x7fd08406e320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.322+0000 7fd0a8ee6640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40d0900 0x7fd0a4072690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.322+0000 7fd0a8ee6640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40d0520 0x7fd0a4072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.322+0000 7fd0a8ee6640 1 --2- 172.21.15.120:0/180599455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a40cc2a0 0x7fd0a406f4c0 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0940090b0 tx=0x7fd0940046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:34.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.322+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/180599455 >> 172.21.15.120:0/180599455 conn(0x7fd0a406ca90 msgr2=0x7fd0a4153a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:34.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.323+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/180599455 shutdown_connections 2024-09-10T10:28:34.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:34.323+0000 7fd0a8ee6640 1 -- 172.21.15.120:0/180599455 wait complete. 2024-09-10T10:28:34.388 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:34.387+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:34.425 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:34.426 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:34.436 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:34.489 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:34.488+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:34.550 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:34.551 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:34.560 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:34.604 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:34.602+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:34.663 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:34.663 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:34.672 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:34.690 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:34.689+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:34.790 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:34.790+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:34.814 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:34.814 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:34.823 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:34.891 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:34.889+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:34.926 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:34.926 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:34.935 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:34.993 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:34.992+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:35.079 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:35.079 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:35.089 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:35.093 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:35.092+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:35.194 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:35.194 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:35.194 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:35.192+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:35.203 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:35.251 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:28:35.295 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:35.294+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:35.347 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:35.348 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:35.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.353+0000 7fdb9ac23640 1 Processor -- start 2024-09-10T10:28:35.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.353+0000 7fdb9ac23640 1 -- start start 2024-09-10T10:28:35.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.354+0000 7fdb9ac23640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdb94162fb0 0x7fdb94160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.354+0000 7fdb9ac23640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb940cd030 0x7fdb940cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.354+0000 7fdb9ac23640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb94162510 con 0x7fdb9415fbc0 2024-09-10T10:28:35.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.354+0000 7fdb9ac23640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fdb941633d0 con 0x7fdb94162fb0 2024-09-10T10:28:35.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.354+0000 7fdb9ac23640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb94162b20 con 0x7fdb940cd030 2024-09-10T10:28:35.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.354+0000 7fdb93fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdb94162fb0 0x7fdb94160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.354+0000 7fdb93fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdb94162fb0 0x7fdb94160370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59336/0 (socket says 172.21.15.120:59336) 2024-09-10T10:28:35.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.354+0000 7fdb93fff640 1 -- 172.21.15.120:0/2651548539 learned_addr learned my addr 172.21.15.120:0/2651548539 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:35.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.354+0000 7fdb991c0640 1 --2- 172.21.15.120:0/2651548539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb940cd030 0x7fdb940cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.355+0000 7fdb93fff640 1 -- 172.21.15.120:0/2651548539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb940cd030 msgr2=0x7fdb940cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:35.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.355+0000 7fdb93fff640 1 --2- 172.21.15.120:0/2651548539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb940cd030 0x7fdb940cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.355+0000 7fdb93fff640 1 -- 172.21.15.120:0/2651548539 >> v1:172.21.15.120:6789/0 conn(0x7fdb9415fbc0 legacy=0x7fdb940b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.355+0000 7fdb93fff640 1 -- 172.21.15.120:0/2651548539 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb940b4ed0 con 0x7fdb94162fb0 2024-09-10T10:28:35.359 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:35.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.355+0000 7fdb93fff640 1 --2- 172.21.15.120:0/2651548539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb94162fb0 0x7fdb94160370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdb94162ca0 tx=0x7fdb8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f7bf0ffe63cc7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb937fe640 1 -- 172.21.15.120:0/2651548539 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80004030 con 0x7fdb94162fb0 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb937fe640 1 -- 172.21.15.120:0/2651548539 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb800041d0 con 0x7fdb94162fb0 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb937fe640 1 -- 172.21.15.120:0/2651548539 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80003610 con 0x7fdb94162fb0 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb9ac23640 1 -- 172.21.15.120:0/2651548539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb94162fb0 msgr2=0x7fdb94160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/2651548539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb94162fb0 0x7fdb94160370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdb94162ca0 tx=0x7fdb8002a230 comp rx=0 tx=0).stop 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb9ac23640 1 -- 172.21.15.120:0/2651548539 shutdown_connections 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/2651548539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb940cd030 0x7fdb940cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/2651548539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb94162fb0 0x7fdb94160370 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb9ac23640 1 -- 172.21.15.120:0/2651548539 >> 172.21.15.120:0/2651548539 conn(0x7fdb9406ce80 msgr2=0x7fdb9406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb9ac23640 1 -- 172.21.15.120:0/2651548539 shutdown_connections 2024-09-10T10:28:35.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.356+0000 7fdb9ac23640 1 -- 172.21.15.120:0/2651548539 wait complete. 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.358+0000 7fdb9ac23640 1 Processor -- start 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.358+0000 7fdb9ac23640 1 -- start start 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb9ac23640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb940cd030 0x7fdb941f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb9ac23640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9415fbc0 0x7fdb940d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb9ac23640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb94162fb0 0x7fdb940ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb9ac23640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb940b6030 con 0x7fdb9415fbc0 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb9ac23640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941627b0 con 0x7fdb940cd030 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb9ac23640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb940b5490 con 0x7fdb94162fb0 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9415fbc0 0x7fdb940d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9415fbc0 0x7fdb940d0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59340/0 (socket says 172.21.15.120:59340) 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb93fff640 1 -- 172.21.15.120:0/176141019 learned_addr learned my addr 172.21.15.120:0/176141019 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb991c0640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb94162fb0 0x7fdb940ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb989bf640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb940cd030 0x7fdb941f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb93fff640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb94162fb0 msgr2=0x7fdb940ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb93fff640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb94162fb0 0x7fdb940ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb93fff640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb940cd030 msgr2=0x7fdb941f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb93fff640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb940cd030 0x7fdb941f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.359+0000 7fdb93fff640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9406f4e0 con 0x7fdb9415fbc0 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.360+0000 7fdb93fff640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9415fbc0 0x7fdb940d0d80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdb80034040 tx=0x7fdb800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.360+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80041030 con 0x7fdb9415fbc0 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.360+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb941f8260 con 0x7fdb9415fbc0 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.360+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb80003bd0 con 0x7fdb9415fbc0 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.360+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80003dc0 con 0x7fdb9415fbc0 2024-09-10T10:28:35.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.360+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb941f8820 con 0x7fdb9415fbc0 2024-09-10T10:28:35.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.361+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fdb941f8070 con 0x7fdb9415fbc0 2024-09-10T10:28:35.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.363+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb80044070 con 0x7fdb9415fbc0 2024-09-10T10:28:35.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.364+0000 7fdb917fa640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb7806be30 0x7fdb7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.364+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7015+0+0 (secure 0 0 0) 0x7fdb800ac960 con 0x7fdb9415fbc0 2024-09-10T10:28:35.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.364+0000 7fdb989bf640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb7806be30 0x7fdb7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.364+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v56) ==== 54+0+16 (secure 0 0 0) 0x7fdb800ad0c0 con 0x7fdb9415fbc0 2024-09-10T10:28:35.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.364+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fdb940713e0 0x7fdb9415c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.364+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb94068ef0 con 0x7fdb940713e0 2024-09-10T10:28:35.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.364+0000 7fdb991c0640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fdb940713e0 0x7fdb9415c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.365+0000 7fdb989bf640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb7806be30 0x7fdb7806e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdb88000f60 tx=0x7fdb88009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.365+0000 7fdb991c0640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fdb940713e0 0x7fdb9415c840 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.366+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb94068ef0 con 0x7fdb940713e0 2024-09-10T10:28:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.380+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdb94073530 con 0x7fdb940713e0 2024-09-10T10:28:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.383+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdb94073530 con 0x7fdb940713e0 2024-09-10T10:28:35.384 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.383+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fdb941f8d40 0x7fdb940c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.383+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fdb9415d290 con 0x7fdb941f8d40 2024-09-10T10:28:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.383+0000 7fdb989bf640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fdb941f8d40 0x7fdb940c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.383+0000 7fdb989bf640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fdb941f8d40 0x7fdb940c7920 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.385+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb88001460 con 0x7fdb941f8d40 2024-09-10T10:28:35.409 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:35.407+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:35.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.413+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdb940cc450 con 0x7fdb941f8d40 2024-09-10T10:28:35.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.416+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdb940cc450 con 0x7fdb941f8d40 2024-09-10T10:28:35.417 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:35.418 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:35.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.416+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fdb940c8370 0x7fdb94062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.416+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fdb940c87e0 con 0x7fdb940c8370 2024-09-10T10:28:35.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.416+0000 7fdb93fff640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fdb940c8370 0x7fdb94062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.417+0000 7fdb93fff640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fdb940c8370 0x7fdb94062630 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.418+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb940c87e0 con 0x7fdb940c8370 2024-09-10T10:28:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.445+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdb940c87e0 con 0x7fdb940c8370 2024-09-10T10:28:35.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.447+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdb940c87e0 con 0x7fdb940c8370 2024-09-10T10:28:35.448 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:35.449 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:35.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.447+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fdb94063080 0x7fdb940ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.447+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fdb941614f0 con 0x7fdb94063080 2024-09-10T10:28:35.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.448+0000 7fdb991c0640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fdb94063080 0x7fdb940ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.448+0000 7fdb991c0640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fdb94063080 0x7fdb940ce3a0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.449+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb941614f0 con 0x7fdb94063080 2024-09-10T10:28:35.467 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:35.467 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:35.476 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:35.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.476+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdb941614f0 con 0x7fdb94063080 2024-09-10T10:28:35.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.479+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdb941614f0 con 0x7fdb94063080 2024-09-10T10:28:35.480 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:35.480 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:35.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.479+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fdb9402c640 0x7fdb9402caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.479+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fdb941614f0 con 0x7fdb9402c640 2024-09-10T10:28:35.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.479+0000 7fdb989bf640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fdb9402c640 0x7fdb9402caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.480+0000 7fdb989bf640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fdb9402c640 0x7fdb9402caa0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.480+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb941614f0 con 0x7fdb9402c640 2024-09-10T10:28:35.497 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:35.496+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:35.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.507+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdb941614f0 con 0x7fdb9402c640 2024-09-10T10:28:35.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.509+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdb941614f0 con 0x7fdb9402c640 2024-09-10T10:28:35.510 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:35.511 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:35.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.509+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fdb9402d4f0 0x7fdb9420b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.509+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fdb940cf1b0 con 0x7fdb9402d4f0 2024-09-10T10:28:35.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.509+0000 7fdb93fff640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fdb9402d4f0 0x7fdb9420b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.510+0000 7fdb93fff640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fdb9402d4f0 0x7fdb9420b450 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.511+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb940cf1b0 con 0x7fdb9402d4f0 2024-09-10T10:28:35.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.537+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdb940ccad0 con 0x7fdb9402d4f0 2024-09-10T10:28:35.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.539+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdb940ccad0 con 0x7fdb9402d4f0 2024-09-10T10:28:35.541 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:35.541 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:35.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.539+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fdb9420bea0 0x7fdb9420e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.539+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fdb9402da00 con 0x7fdb9420bea0 2024-09-10T10:28:35.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.540+0000 7fdb991c0640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fdb9420bea0 0x7fdb9420e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.540+0000 7fdb991c0640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fdb9420bea0 0x7fdb9420e290 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.541+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb9402da00 con 0x7fdb9420bea0 2024-09-10T10:28:35.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.567+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdb9402d980 con 0x7fdb9420bea0 2024-09-10T10:28:35.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.569+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdb9402d980 con 0x7fdb9420bea0 2024-09-10T10:28:35.571 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:35.571 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:35.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.570+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fdb9420ece0 0x7fdb942110d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:35.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.570+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fdb94213f50 con 0x7fdb9420ece0 2024-09-10T10:28:35.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.570+0000 7fdb989bf640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fdb9420ece0 0x7fdb942110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:35.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.570+0000 7fdb989bf640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fdb9420ece0 0x7fdb942110d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:35.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.571+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb94213f50 con 0x7fdb9420ece0 2024-09-10T10:28:35.598 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:35.597+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:35.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.598+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdb94211690 con 0x7fdb9420ece0 2024-09-10T10:28:35.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb917fa640 1 -- 172.21.15.120:0/176141019 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdb94211690 con 0x7fdb9420ece0 2024-09-10T10:28:35.602 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:35.603 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:35.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fdb940713e0 msgr2=0x7fdb9415c840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fdb940713e0 0x7fdb9415c840 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fdb941f8d40 msgr2=0x7fdb940c7920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fdb941f8d40 0x7fdb940c7920 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fdb940c8370 msgr2=0x7fdb94062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fdb940c8370 0x7fdb94062630 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fdb94063080 msgr2=0x7fdb940ce3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fdb94063080 0x7fdb940ce3a0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fdb9402c640 msgr2=0x7fdb9402caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fdb9402c640 0x7fdb9402caa0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fdb9402d4f0 msgr2=0x7fdb9420b450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fdb9402d4f0 0x7fdb9420b450 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fdb9420bea0 msgr2=0x7fdb9420e290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fdb9420bea0 0x7fdb9420e290 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fdb9420ece0 msgr2=0x7fdb942110d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fdb9420ece0 0x7fdb942110d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb7806be30 msgr2=0x7fdb7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdb7806be30 0x7fdb7806e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdb88000f60 tx=0x7fdb88009040 comp rx=0 tx=0).stop 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb991c0640 1 -- 172.21.15.120:0/176141019 reap_dead start 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb991c0640 1 -- 172.21.15.120:0/176141019 reap_dead start 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9415fbc0 msgr2=0x7fdb940d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.601+0000 7fdb9ac23640 1 --2- 172.21.15.120:0/176141019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9415fbc0 0x7fdb940d0d80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdb80034040 tx=0x7fdb800046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.602+0000 7fdb991c0640 1 -- 172.21.15.120:0/176141019 reap_dead start 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.602+0000 7fdb991c0640 1 -- 172.21.15.120:0/176141019 reap_dead start 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.602+0000 7fdb991c0640 1 -- 172.21.15.120:0/176141019 reap_dead start 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.602+0000 7fdb991c0640 1 -- 172.21.15.120:0/176141019 reap_dead start 2024-09-10T10:28:35.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.602+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 shutdown_connections 2024-09-10T10:28:35.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.602+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 >> 172.21.15.120:0/176141019 conn(0x7fdb9406ce80 msgr2=0x7fdb940c6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:35.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.602+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 shutdown_connections 2024-09-10T10:28:35.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:35.602+0000 7fdb9ac23640 1 -- 172.21.15.120:0/176141019 wait complete. 2024-09-10T10:28:35.622 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:35.622 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:35.622 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:35.622 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:35.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 108.840881596, 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:35.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:28:35.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:28:35.627 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 108.840551096, 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:35.629 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.630 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:28:35.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 108.840504553, 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.634 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.635 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:28:35.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.749062340999998, 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.639 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:28:35.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:35.641 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.642 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:35.643 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.748980997999993, 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.645 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.646 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.647 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.648 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.649 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.649 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.649 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.649 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.649 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:35.649 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.748954443000002, 2024-09-10T10:28:35.649 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:28:35.649 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.649 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.649 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:28:35.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.652 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.654 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.748811794000005, 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.655 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:35.656 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.656 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.656 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:35.656 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.656 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:28:35.659 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.746164860999997, 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:28:35.662 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.663 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.745314121000007, 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.665 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.666 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:28:35.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.673162438000006, 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:28:35.671 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.672 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:28:35.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.673113302000004, 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:28:35.675 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.677 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.678 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.679 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.680 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.673001823999996, 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.682 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:28:35.684 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 75.699601305000002, 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.685 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:28:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 75.699571133999996, 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:28:35.690 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:28:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:28:35.693 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.694 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.695 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 75.699514641999997, 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.696 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:28:35.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:28:35.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:35.699 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.700 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.701 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 75.699413473999996, 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:35.702 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:28:35.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:28:35.704 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.705 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 69.679146376999995, 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:28:35.710 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:28:35.711 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:35.712 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 69.675081157999998, 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:35.713 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.714 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.715 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:28:35.716 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 69.675063288000004, 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:28:35.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:35.719 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 69.675017049999994, 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:28:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:28:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:28:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:35.726 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:35.727 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:35.728 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:35.726+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:35.828 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:35.827+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:35.872 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:35.873 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:35.873 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:35.873 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:35.881 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:35.929 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:35.928+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:35.983 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:35.983 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:35.984 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:35.984 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:35.984 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:35.992 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:36.036 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:36.034+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:36.098 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:36.098 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:36.098 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:36.098 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:36.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.098 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.098 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:36.098 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.999365045, 2024-09-10T10:28:36.098 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.17492211699999999, 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 1 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:36.099 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096318+0000", 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.657e-06 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096326+0000", 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0560000000000005e-06 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096339+0000", 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2965999999999999e-05 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096349+0000", 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0784e-05 2024-09-10T10:28:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096432+0000", 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2657999999999994e-05 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op must wait for map", 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096441+0000", 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4590000000000004e-06 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096935+0000", 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049331899999999998 2024-09-10T10:28:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for peered", 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096944+0000", 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9539999999999993e-06 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267505+0000", 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.170561622 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267700+0000", 2024-09-10T10:28:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019420400000000001 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267923+0000", 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000223451 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.270575+0000", 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026522030000000001 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271131+0000", 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055599000000000002 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271191+0000", 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9947000000000001e-05 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271238+0000", 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6847000000000001e-05 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.610530801, 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078035889999999997, 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 115 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1300000000000001e-07 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485209+0000", 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8628999999999997e-05 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485214+0000", 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0849999999999996e-06 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485218+0000", 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3840000000000001e-06 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490185+0000", 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0049675270000000002 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490231+0000", 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5488000000000002e-05 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490307+0000", 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6038000000000006e-05 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491263+0000", 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095625099999999998 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492908+0000", 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001645355 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492934+0000", 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5431000000000001e-05 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492954+0000", 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0088000000000001e-05 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.610456312, 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077741099999999999, 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 116 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1100000000000002e-07 2024-09-10T10:28:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485227+0000", 2024-09-10T10:28:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1890000000000001e-06 2024-09-10T10:28:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485231+0000", 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4010000000000001e-06 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485233+0000", 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.728e-06 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490628+0000", 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0053952109999999996 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490674+0000", 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5946999999999999e-05 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490750+0000", 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5227000000000003e-05 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491443+0000", 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069346800000000004 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492959+0000", 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015154470000000001 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492983+0000", 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.406e-05 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492999+0000", 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6121e-05 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.610444184, 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078059490000000004, 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 117 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.23e-07 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485240+0000", 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780000000000001e-06 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485243+0000", 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8430000000000003e-06 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485247+0000", 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2279999999999998e-06 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490995+0000", 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057480730000000002 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491043+0000", 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7936000000000003e-05 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.121 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491113+0000", 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0872999999999997e-05 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491636+0000", 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00052242300000000005 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493002+0000", 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001365808 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493025+0000", 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2932999999999999e-05 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493043+0000", 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8031000000000001e-05 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:36.123 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.278089042, 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095639920000000003, 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 131 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1199999999999998e-07 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817663+0000", 2024-09-10T10:28:36.125 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1099000000000005e-05 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817670+0000", 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.8539999999999999e-06 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817674+0000", 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.574e-06 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818465+0000", 2024-09-10T10:28:36.126 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000791445 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818509+0000", 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3569000000000002e-05 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818580+0000", 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1503999999999998e-05 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.127 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821292+0000", 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027120529999999999 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827108+0000", 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0058160759999999999 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827134+0000", 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5859000000000001e-05 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.128 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827156+0000", 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1546999999999999e-05 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.277999834, 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095198769999999995, 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.129 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 132 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2899999999999999e-07 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817683+0000", 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0379999999999998e-06 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.131 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817690+0000", 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3110000000000002e-06 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817699+0000", 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820828+0000", 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031296829999999999 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820873+0000", 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4869e-05 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820943+0000", 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9919999999999995e-05 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823683+0000", 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027395140000000002 2024-09-10T10:28:36.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827161+0000", 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003477857 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827183+0000", 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2612999999999998e-05 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827201+0000", 2024-09-10T10:28:36.134 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7842999999999998e-05 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.277932527, 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0094944009999999995, 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.135 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 133 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817749+0000", 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2399999999999998e-07 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817752+0000", 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1970000000000001e-06 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817760+0000", 2024-09-10T10:28:36.137 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8080000000000005e-06 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817764+0000", 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9419999999999997e-06 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821319+0000", 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035547740000000001 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.138 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821364+0000", 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5398000000000001e-05 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821434+0000", 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0382999999999993e-05 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823717+0000", 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022825089999999998 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.139 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827205+0000", 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034879009999999998 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827225+0000", 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0611999999999998e-05 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827243+0000", 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7353000000000001e-05 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.140 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.27787968600001, 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095863660000000007, 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.141 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 134 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.142 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817802+0000", 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8999999999999996e-07 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817871+0000", 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9572000000000004e-05 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817878+0000", 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2079999999999997e-06 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817882+0000", 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5889999999999996e-06 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821625+0000", 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037431069999999999 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821669+0000", 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3455000000000003e-05 2024-09-10T10:28:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821740+0000", 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1390999999999995e-05 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823826+0000", 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020863399999999999 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827343+0000", 2024-09-10T10:28:36.145 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003516267 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827369+0000", 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.6044999999999999e-05 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827388+0000", 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8802000000000001e-05 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.146 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.277786991, 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095738179999999996, 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 135 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.147 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5999999999999999e-07 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.148 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817897+0000", 2024-09-10T10:28:36.149 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0149999999999999e-06 2024-09-10T10:28:36.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.149 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.149 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817904+0000", 2024-09-10T10:28:36.149 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5089999999999999e-06 2024-09-10T10:28:36.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.149 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817909+0000", 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3120000000000001e-06 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823862+0000", 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059524199999999999 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823911+0000", 2024-09-10T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9323000000000002e-05 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823983+0000", 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1736000000000006e-05 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827101+0000", 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031181009999999999 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827392+0000", 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00029090700000000003 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827426+0000", 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4127e-05 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827468+0000", 2024-09-10T10:28:36.152 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1907999999999999e-05 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.277759975, 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.009594488, 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.153 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 136 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:36.154 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3700000000000001e-07 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817925+0000", 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4659999999999999e-06 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817931+0000", 2024-09-10T10:28:36.155 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5710000000000003e-06 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817936+0000", 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7550000000000002e-06 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824178+0000", 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062415700000000001 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.156 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824225+0000", 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6966999999999999e-05 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824296+0000", 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1284000000000004e-05 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827290+0000", 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002993697 2024-09-10T10:28:36.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.158 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.158 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827473+0000", 2024-09-10T10:28:36.158 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018376699999999999 2024-09-10T10:28:36.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.158 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.158 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827497+0000", 2024-09-10T10:28:36.158 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3445e-05 2024-09-10T10:28:36.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827515+0000", 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8529000000000002e-05 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.277728286, 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096066220000000004, 2024-09-10T10:28:36.159 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 137 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3000000000000001e-07 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818004+0000", 2024-09-10T10:28:36.161 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0902000000000002e-05 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818013+0000", 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5420000000000005e-06 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818019+0000", 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7939999999999997e-06 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.162 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824489+0000", 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0064692580000000003 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824532+0000", 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3631000000000003e-05 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824601+0000", 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9028000000000004e-05 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827334+0000", 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027325409999999998 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827520+0000", 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018642499999999999 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827541+0000", 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1151000000000001e-05 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.164 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827559+0000", 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7819999999999999e-05 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.277648965, 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010100303, 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.165 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 138 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.166 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818033+0000", 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818037+0000", 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0419999999999996e-06 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.167 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818045+0000", 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9759999999999995e-06 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818052+0000", 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5909999999999998e-06 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826594+0000", 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0085421660000000003 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.168 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826671+0000", 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7121999999999998e-05 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826795+0000", 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012315199999999999 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828081+0000", 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0012869139999999999 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.169 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828088+0000", 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9870000000000002e-06 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828112+0000", 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3405e-05 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828132+0000", 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0394000000000002e-05 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.170 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.277618594, 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01125819, 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 139 2024-09-10T10:28:36.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.172 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818063+0000", 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7800000000000002e-07 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818067+0000", 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8500000000000004e-06 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818075+0000", 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:36.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818081+0000", 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1829999999999999e-06 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827594+0000", 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095129340000000007 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827669+0000", 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5239000000000002e-05 2024-09-10T10:28:36.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827789+0000", 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000119757 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829195+0000", 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014064279999999999 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829232+0000", 2024-09-10T10:28:36.175 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7058999999999999e-05 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829281+0000", 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8884000000000002e-05 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829321+0000", 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9527999999999998e-05 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.27758452499999, 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012263886, 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 140 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818097+0000", 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8999999999999997e-07 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818145+0000", 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8040000000000001e-05 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818160+0000", 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4671999999999999e-05 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818169+0000", 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1900000000000001e-06 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828158+0000", 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099891440000000001 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828228+0000", 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0407000000000005e-05 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828343+0000", 2024-09-10T10:28:36.180 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000114821 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829914+0000", 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001570866 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830312+0000", 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00039770499999999999 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.181 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830339+0000", 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.69e-05 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830360+0000", 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1651e-05 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.182 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.27749728400001, 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012742035000000001, 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 141 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.183 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7700000000000004e-07 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818188+0000", 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8659999999999999e-06 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818196+0000", 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9480000000000008e-06 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818205+0000", 2024-09-10T10:28:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8570000000000001e-06 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828883+0000", 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010678600999999999 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828927+0000", 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3887999999999997e-05 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.186 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829001+0000", 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3442000000000006e-05 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830548+0000", 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001547194 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830884+0000", 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00033638100000000002 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.187 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830907+0000", 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3051999999999999e-05 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830926+0000", 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8229000000000001e-05 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:36.188 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 123.26434954699999, 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097270880000000001, 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 145 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831332+0000", 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8900000000000001e-07 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.190 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831379+0000", 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7134e-05 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831387+0000", 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7409999999999992e-06 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831391+0000", 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6190000000000002e-06 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.191 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833155+0000", 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017641969999999999 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833237+0000", 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1957000000000002e-05 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833356+0000", 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011843100000000001 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.192 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.834068+0000", 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00071179000000000001 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841004+0000", 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069359749999999996 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841034+0000", 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.042e-05 2024-09-10T10:28:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841058+0000", 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4535000000000001e-05 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:20 2.9 e26/18 2:92e99a1d:::612.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 87.220153283000002, 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074964979999999999, 2024-09-10T10:28:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875534+0000", 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9109999999999998e-06 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875543+0000", 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8319999999999995e-06 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875549+0000", 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2380000000000002e-06 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875699+0000", 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014945900000000001 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875717+0000", 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8584e-05 2024-09-10T10:28:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882982+0000", 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0072648039999999997 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883024+0000", 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2135000000000002e-05 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:27 2.9 e26/18 2:91bd965c:::501.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 87.220092648999994, 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074993129999999996, 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875589+0000", 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3999999999999997e-07 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875595+0000", 2024-09-10T10:28:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9680000000000003e-06 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875602+0000", 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875608+0000", 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0430000000000004e-06 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875887+0000", 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000279578 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875904+0000", 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6439e-05 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883027+0000", 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0071234480000000001 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883088+0000", 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0424000000000003e-05 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 70.154013571999997, 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01193195, 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:28:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2399999999999999e-07 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941671+0000", 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8329999999999998e-06 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2919999999999998e-06 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941686+0000", 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944874+0000", 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031881639999999998 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945305+0000", 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00043084599999999997 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945438+0000", 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013344600000000001 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950653+0000", 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005214733 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953496+0000", 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028433989999999999 2024-09-10T10:28:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953554+0000", 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7933000000000003e-05 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953599+0000", 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5130999999999999e-05 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 70.153987626000003, 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0093592050000000006, 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941694+0000", 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7899999999999999e-07 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941711+0000", 2024-09-10T10:28:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7530999999999999e-05 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941720+0000", 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4279999999999996e-06 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941725+0000", 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2390000000000002e-06 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942690+0000", 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00096541400000000005 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942807+0000", 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000116777 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942967+0000", 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015997599999999999 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944708+0000", 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001740492 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950956+0000", 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062481580000000002 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951016+0000", 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0299e-05 2024-09-10T10:28:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951052+0000", 2024-09-10T10:28:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6412000000000003e-05 2024-09-10T10:28:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:36.214 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:36.216 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:36.214+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:36.216 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:36.317 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:36.316+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:36.325 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:36.325 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:36.326 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:36.326 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:36.335 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:36.340 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-10T10:28:36.420 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:36.419+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:36.520 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:36.518+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:36.527 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:36.527 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:36.527 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:36.527 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:36.528 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:36.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.529+0000 7fb22d511640 1 Processor -- start 2024-09-10T10:28:36.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.529+0000 7fb22d511640 1 -- start start 2024-09-10T10:28:36.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.530+0000 7fb22d511640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb228154e40 0x7fb22806cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:36.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.530+0000 7fb22d511640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb2280c0760 0x7fb22806ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:36.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.530+0000 7fb22d511640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2280c7b40 con 0x7fb22806efa0 2024-09-10T10:28:36.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.530+0000 7fb22d511640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb2280c8ba0 con 0x7fb228154e40 2024-09-10T10:28:36.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.530+0000 7fb22d511640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb2280c8e80 con 0x7fb2280c0760 2024-09-10T10:28:36.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.531+0000 7fb2267fc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb2280c0760 0x7fb22806ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:36.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.531+0000 7fb2267fc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb2280c0760 0x7fb22806ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:34498/0 (socket says 172.21.15.120:34498) 2024-09-10T10:28:36.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.531+0000 7fb2267fc640 1 -- 172.21.15.120:0/3325096925 learned_addr learned my addr 172.21.15.120:0/3325096925 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:36.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.531+0000 7fb226ffd640 1 --2- 172.21.15.120:0/3325096925 >> v2:172.21.15.120:3300/0 conn(0x7fb228154e40 0x7fb22806cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.531+0000 7fb2267fc640 1 -- 172.21.15.120:0/3325096925 >> v2:172.21.15.120:3300/0 conn(0x7fb228154e40 msgr2=0x7fb22806cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.531+0000 7fb225ffb640 1 -- 172.21.15.120:0/3325096925 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1297889425 0 0) 0x7fb2280c7b40 con 0x7fb22806efa0 2024-09-10T10:28:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.531+0000 7fb2267fc640 1 --2- 172.21.15.120:0/3325096925 >> v2:172.21.15.120:3300/0 conn(0x7fb228154e40 0x7fb22806cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.531+0000 7fb2267fc640 1 -- 172.21.15.120:0/3325096925 >> v1:172.21.15.134:6789/0 conn(0x7fb22806efa0 legacy=0x7fb22815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.531+0000 7fb2267fc640 1 -- 172.21.15.120:0/3325096925 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2280c7e00 con 0x7fb2280c0760 2024-09-10T10:28:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.531+0000 7fb226ffd640 1 --2- 172.21.15.120:0/3325096925 >> v2:172.21.15.120:3300/0 conn(0x7fb228154e40 0x7fb22806cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.532+0000 7fb2267fc640 1 --2- 172.21.15.120:0/3325096925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2280c0760 0x7fb22806ea60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2280caba0 tx=0x7fb21c02a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c0abdc7fa64df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.533+0000 7fb225ffb640 1 -- 172.21.15.120:0/3325096925 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c037070 con 0x7fb2280c0760 2024-09-10T10:28:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.533+0000 7fb225ffb640 1 -- 172.21.15.120:0/3325096925 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21c032440 con 0x7fb2280c0760 2024-09-10T10:28:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.533+0000 7fb225ffb640 1 -- 172.21.15.120:0/3325096925 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c032850 con 0x7fb2280c0760 2024-09-10T10:28:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.534+0000 7fb22d511640 1 -- 172.21.15.120:0/3325096925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2280c0760 msgr2=0x7fb22806ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.534+0000 7fb22d511640 1 --2- 172.21.15.120:0/3325096925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2280c0760 0x7fb22806ea60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2280caba0 tx=0x7fb21c02a2a0 comp rx=0 tx=0).stop 2024-09-10T10:28:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.534+0000 7fb22d511640 1 -- 172.21.15.120:0/3325096925 shutdown_connections 2024-09-10T10:28:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.534+0000 7fb22d511640 1 --2- 172.21.15.120:0/3325096925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2280c0760 0x7fb22806ea60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.534+0000 7fb22d511640 1 --2- 172.21.15.120:0/3325096925 >> v2:172.21.15.120:3300/0 conn(0x7fb228154e40 0x7fb22806cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.534+0000 7fb22d511640 1 -- 172.21.15.120:0/3325096925 >> 172.21.15.120:0/3325096925 conn(0x7fb228067950 msgr2=0x7fb228067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.534+0000 7fb22d511640 1 -- 172.21.15.120:0/3325096925 shutdown_connections 2024-09-10T10:28:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.534+0000 7fb22d511640 1 -- 172.21.15.120:0/3325096925 wait complete. 2024-09-10T10:28:36.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.536+0000 7fb22d511640 1 Processor -- start 2024-09-10T10:28:36.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.536+0000 7fb22d511640 1 -- start start 2024-09-10T10:28:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.537+0000 7fb22d511640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb22806efa0 0x7fb2280bd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.537+0000 7fb22d511640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2280c0760 0x7fb22815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.537+0000 7fb22d511640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb228154e40 0x7fb228151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.537+0000 7fb22d511640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb2280c8620 con 0x7fb2280c0760 2024-09-10T10:28:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.537+0000 7fb22d511640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb22806bd30 con 0x7fb22806efa0 2024-09-10T10:28:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.537+0000 7fb22d511640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb228155220 con 0x7fb228154e40 2024-09-10T10:28:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb2267fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2280c0760 0x7fb22815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:36.540 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb2267fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2280c0760 0x7fb22815cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59364/0 (socket says 172.21.15.120:59364) 2024-09-10T10:28:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb2267fc640 1 -- 172.21.15.120:0/924116470 learned_addr learned my addr 172.21.15.120:0/924116470 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb2277fe640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb228154e40 0x7fb228151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb226ffd640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb22806efa0 0x7fb2280bd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb2267fc640 1 -- 172.21.15.120:0/924116470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb228154e40 msgr2=0x7fb228151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb2267fc640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb228154e40 0x7fb228151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb2267fc640 1 -- 172.21.15.120:0/924116470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb22806efa0 msgr2=0x7fb2280bd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb2267fc640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb22806efa0 0x7fb2280bd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb2267fc640 1 -- 172.21.15.120:0/924116470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22815bc70 con 0x7fb2280c0760 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb2267fc640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2280c0760 0x7fb22815cb80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009960 tx=0x7fb21c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb20f7fe640 1 -- 172.21.15.120:0/924116470 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c037040 con 0x7fb2280c0760 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.538+0000 7fb20f7fe640 1 -- 172.21.15.120:0/924116470 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21c042070 con 0x7fb2280c0760 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.539+0000 7fb22d511640 1 -- 172.21.15.120:0/924116470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22815bf30 con 0x7fb2280c0760 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.539+0000 7fb20f7fe640 1 -- 172.21.15.120:0/924116470 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c03d8a0 con 0x7fb2280c0760 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.539+0000 7fb22d511640 1 -- 172.21.15.120:0/924116470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22815e980 con 0x7fb2280c0760 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.540+0000 7fb22d511640 1 -- 172.21.15.120:0/924116470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22806ea60 con 0x7fb2280c0760 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.541+0000 7fb20f7fe640 1 -- 172.21.15.120:0/924116470 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb21c03dac0 con 0x7fb2280c0760 2024-09-10T10:28:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.541+0000 7fb20f7fe640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb21406be30 0x7fb21406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:36.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.541+0000 7fb20f7fe640 1 -- 172.21.15.120:0/924116470 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7015+0+0 (secure 0 0 0) 0x7fb21c0ab010 con 0x7fb2280c0760 2024-09-10T10:28:36.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.541+0000 7fb226ffd640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb21406be30 0x7fb21406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:36.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.542+0000 7fb226ffd640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb21406be30 0x7fb21406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2281f1340 tx=0x7fb218009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:36.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.546+0000 7fb20f7fe640 1 -- 172.21.15.120:0/924116470 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb21c078e50 con 0x7fb2280c0760 2024-09-10T10:28:36.620 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:36.618+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:36.670 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:36.670 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:36.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:36.670 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:36.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.670 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:1 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 124.399541394, 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028809700000000001, 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:36.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9999999999999999e-07 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268119+0000", 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1810000000000003e-06 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268129+0000", 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9790000000000005e-06 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268137+0000", 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4800000000000001e-06 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268198+0000", 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0918000000000002e-05 2024-09-10T10:28:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268226+0000", 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7267000000000002e-05 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270943+0000", 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027177529999999998 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270996+0000", 2024-09-10T10:28:36.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2191999999999998e-05 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:119 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'81, mlcod=16'78)", 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 124.17369044100001, 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029329880000000001, 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493966+0000", 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1699999999999998e-07 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494040+0000", 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3883999999999999e-05 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494048+0000", 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4880000000000007e-06 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494057+0000", 2024-09-10T10:28:36.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1310000000000006e-06 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494095+0000", 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8470000000000003e-05 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494108+0000", 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2588999999999999e-05 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496869+0000", 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027610400000000002 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496898+0000", 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9869000000000002e-05 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 124.173306805, 2024-09-10T10:28:36.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005336365, 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494350+0000", 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494416+0000", 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6413000000000003e-05 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494424+0000", 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.576e-06 2024-09-10T10:28:36.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494430+0000", 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6579999999999996e-06 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494466+0000", 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5648999999999997e-05 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494478+0000", 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.234e-05 2024-09-10T10:28:36.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499654+0000", 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051755209999999998 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499685+0000", 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.167e-05 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 124.17291199500001, 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049664970000000003, 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494819+0000", 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4690000000000005e-05 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494825+0000", 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7080000000000002e-06 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494828+0000", 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1109999999999999e-06 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494869+0000", 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1146000000000002e-05 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494881+0000", 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2295e-05 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499687+0000", 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048061459999999999 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499710+0000", 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-05 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.687 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.887555792, 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054008370000000003, 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780104+0000", 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3039999999999998e-06 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:28:36.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0738e-05 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780141+0000", 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6317999999999999e-05 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2199999999999992e-06 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780405+0000", 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025557599999999998 2024-09-10T10:28:36.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780498+0000", 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2860000000000002e-05 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780591+0000", 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3094e-05 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:28:36.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024814469999999999 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785441+0000", 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023678280000000002 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785480+0000", 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9511999999999997e-05 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785501+0000", 2024-09-10T10:28:36.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0939999999999999e-05 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.887480958, 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054780480000000001, 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780176+0000", 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6700000000000003e-07 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1289999999999998e-06 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780188+0000", 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.96e-06 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780194+0000", 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9989999999999997e-06 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780882+0000", 2024-09-10T10:28:36.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068832400000000003 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780968+0000", 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5959999999999997e-05 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781098+0000", 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012994100000000001 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783078+0000", 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019803540000000001 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785553+0000", 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002474515 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785608+0000", 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5319999999999999e-05 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785653+0000", 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4879000000000002e-05 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.88744941500001, 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042025200000000004, 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.698 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780217+0000", 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780224+0000", 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2529999999999998e-06 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780230+0000", 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6110000000000001e-06 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780282+0000", 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2148000000000003e-05 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780394+0000", 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111864 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780553+0000", 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015913799999999999 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783054+0000", 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025008809999999999 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784349+0000", 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012947480000000001 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784381+0000", 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2268999999999999e-05 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784409+0000", 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8007000000000001e-05 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:36.703 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.887407063, 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054782659999999999, 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780253+0000", 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4199999999999999e-06 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780260+0000", 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5100000000000001e-06 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780267+0000", 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.443e-06 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781353+0000", 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010859769999999999 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781457+0000", 2024-09-10T10:28:36.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104545 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781561+0000", 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010409200000000001 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783532+0000", 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019702280000000001 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785685+0000", 2024-09-10T10:28:36.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002153313 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785710+0000", 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5024999999999998e-05 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785727+0000", 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7175000000000001e-05 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.887380972, 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005104407, 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780278+0000", 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.959e-06 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780285+0000", 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8040000000000004e-06 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780292+0000", 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4099999999999996e-06 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781345+0000", 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010533579999999999 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781456+0000", 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110624 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781570+0000", 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011417999999999999 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783625+0000", 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020553780000000001 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785303+0000", 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001677827 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785345+0000", 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2364999999999998e-05 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785379+0000", 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4054000000000003e-05 2024-09-10T10:28:36.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 109.88735714400001, 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0060500800000000002, 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:36.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780306+0000", 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780313+0000", 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-06 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1739999999999999e-06 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782543+0000", 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022253419999999999 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010785699999999999 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.786349+0000", 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036978829999999999 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.795655104000005, 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058824769999999997, 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.718 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872002+0000", 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5149999999999999e-06 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872008+0000", 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.66e-06 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872022+0000", 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3932e-05 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872031+0000", 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5259999999999999e-06 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872111+0000", 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9412000000000005e-05 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872231+0000", 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119843 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872340+0000", 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109442 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.875579+0000", 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032383899999999998 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877781+0000", 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022022299999999999 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877835+0000", 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4419000000000002e-05 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877883+0000", 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8108000000000001e-05 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.723 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.794566273000001, 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049968149999999999, 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7300000000000005e-07 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.245e-06 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3898e-05 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873116+0000", 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8050000000000007e-06 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873184+0000", 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7809000000000001e-05 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873257+0000", 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2602999999999996e-05 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873374+0000", 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011744900000000001 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.876282+0000", 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029079459999999998 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878003+0000", 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001720886 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878054+0000", 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0596999999999997e-05 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878086+0000", 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2804e-05 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.720173900999995, 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029964739999999998, 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947484+0000", 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.857e-06 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947490+0000", 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9789999999999998e-06 2024-09-10T10:28:36.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947504+0000", 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3942e-05 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947512+0000", 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5620000000000004e-06 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947599+0000", 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6525000000000003e-05 2024-09-10T10:28:36.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947714+0000", 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114709 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947823+0000", 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109477 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950336+0000", 2024-09-10T10:28:36.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025131509999999999 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950386+0000", 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9974e-05 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950440+0000", 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4011000000000002e-05 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950478+0000", 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8287000000000001e-05 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.718733502999996, 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043004360000000004, 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.734 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948923+0000", 2024-09-10T10:28:36.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8999999999999996e-07 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948928+0000", 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948940+0000", 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1759e-05 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948949+0000", 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5149999999999995e-06 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949028+0000", 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8182999999999993e-05 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949137+0000", 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109065 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949245+0000", 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108047 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950193+0000", 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094856200000000002 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953173+0000", 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029801960000000001 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953200+0000", 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6707999999999999e-05 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953223+0000", 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2758000000000001e-05 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 82.718677991000007, 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046333320000000004, 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:36.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948979+0000", 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-07 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6490000000000001e-06 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948990+0000", 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2560000000000002e-06 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948996+0000", 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9739999999999999e-06 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949496+0000", 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049934299999999997 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949588+0000", 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1762999999999995e-05 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949729+0000", 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014145299999999999 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950984+0000", 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012548419999999999 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953505+0000", 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025208129999999998 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953568+0000", 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3695000000000006e-05 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953611+0000", 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2913000000000001e-05 2024-09-10T10:28:36.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.746626375999995, 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048620219999999997, 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4899999999999995e-07 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921034+0000", 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7050000000000001e-06 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921043+0000", 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1840000000000005e-06 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921049+0000", 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4459999999999998e-06 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921575+0000", 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052602300000000003 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921646+0000", 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0475000000000005e-05 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921769+0000", 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123347 2024-09-10T10:28:36.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923077+0000", 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013078759999999999 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925794+0000", 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027172310000000001 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925853+0000", 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8739999999999997e-05 2024-09-10T10:28:36.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925892+0000", 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8445999999999997e-05 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.746589341999993, 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032737339999999999, 2024-09-10T10:28:36.750 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.751 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.751 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.751 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:36.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9919999999999999e-06 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921080+0000", 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0063e-05 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921086+0000", 2024-09-10T10:28:36.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921142+0000", 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6174999999999997e-05 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013480499999999999 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:36.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921405+0000", 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012819200000000001 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923001+0000", 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015962249999999999 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924243+0000", 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012425820000000001 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924300+0000", 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6901999999999998e-05 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924340+0000", 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9918e-05 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.756 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(2.4 epoch 26/18 rep_tid 19 entries 26'1 (0'0) error 2:218afd09:::203.00000003:head by mds.3.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.744450896999993, 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030589089999999998, 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923206+0000", 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7249999999999997e-06 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923229+0000", 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5905000000000001e-05 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923241+0000", 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1990999999999999e-05 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923317+0000", 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5622000000000002e-05 2024-09-10T10:28:36.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926264+0000", 2024-09-10T10:28:36.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029470170000000001 2024-09-10T10:28:36.760 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.760 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.760 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.760 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.724092897000006, 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050642550000000001, 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943564+0000", 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.13e-07 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943576+0000", 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2187e-05 2024-09-10T10:28:36.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943585+0000", 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9279999999999999e-06 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943592+0000", 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0110000000000003e-06 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943681+0000", 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9487999999999994e-05 2024-09-10T10:28:36.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943767+0000", 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5866999999999996e-05 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943882+0000", 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011443299999999999 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:36.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.947042+0000", 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031600629999999998 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948540+0000", 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001498451 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948581+0000", 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0839e-05 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948627+0000", 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6375000000000002e-05 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 70.724024573999998, 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051386089999999997, 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:36.766 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943632+0000", 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1500000000000004e-07 2024-09-10T10:28:36.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943643+0000", 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0954e-05 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943652+0000", 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4929999999999998e-06 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943657+0000", 2024-09-10T10:28:36.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6509999999999998e-06 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944230+0000", 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057322999999999999 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944340+0000", 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010986 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:36.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948770+0000", 2024-09-10T10:28:36.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044297060000000003 2024-09-10T10:28:36.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.770 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:36.770 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:36.770 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:36.771 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:36.772 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:36.769+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:36.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:36.781+0000 7fb22d511640 1 -- 172.21.15.120:0/924116470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fb22815b0c0 con 0x7fb2280c0760 2024-09-10T10:28:36.872 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:36.872+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:36.887 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:36.887 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:36.887 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:36.887 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:36.896 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:36.981 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:36.980+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:37.042 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:37.042 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:37.042 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:37.042 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:37.042 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:37.051 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:37.075 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:37.073+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:37.176 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:37.175+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:37.201 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.419919713, 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049304329999999997, 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.202 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780094+0000", 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780107+0000", 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3703000000000001e-05 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780116+0000", 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5210000000000002e-06 2024-09-10T10:28:37.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6340000000000007e-06 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001735838 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782000+0000", 2024-09-10T10:28:37.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013984100000000001 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782131+0000", 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000131025 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783737+0000", 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016054179999999999 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784937+0000", 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001200638 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784987+0000", 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9591e-05 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785024+0000", 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6751000000000001e-05 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.41988025800001, 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052929559999999997, 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:37.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780136+0000", 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9950000000000001e-06 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780144+0000", 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.174e-06 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4949999999999999e-06 2024-09-10T10:28:37.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781163+0000", 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001013438 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781279+0000", 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011579200000000001 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781388+0000", 2024-09-10T10:28:37.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108834 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783243+0000", 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00185491 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785355+0000", 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021117990000000001 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785386+0000", 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1612000000000002e-05 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785426+0000", 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9474999999999997e-05 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.213 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:8 2.10 e24/18 2:09401a81:::601.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.417689028, 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005352224, 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4400000000000001e-07 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782467+0000", 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142452 2024-09-10T10:28:37.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782598+0000", 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013078300000000001 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782604+0000", 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.533e-06 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7073999999999997e-05 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782668+0000", 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6861999999999999e-05 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787632+0000", 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049636510000000003 2024-09-10T10:28:37.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787676+0000", 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4425000000000002e-05 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:9 2.10 e24/18 2:09401a81:::601.00000000:head v 24'4, mlcod=0'0)", 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.417052013, 2024-09-10T10:28:37.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047438960000000001, 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:37.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782962+0000", 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4499999999999997e-07 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782967+0000", 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2000000000000002e-06 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782980+0000", 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3685e-05 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782987+0000", 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4690000000000002e-06 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783112+0000", 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012481300000000001 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783130+0000", 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7753000000000001e-05 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787678+0000", 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045487419999999997 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787705+0000", 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6588999999999999e-05 2024-09-10T10:28:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:10 2.10 e24/18 2:0bd6d154:::602.00000000:head v 24'5, mlcod=0'0)", 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.416945079, 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00466629, 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1300000000000001e-07 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6970000000000001e-06 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783309+0000", 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000236209 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783319+0000", 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7180000000000008e-06 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783365+0000", 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6179e-05 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783383+0000", 2024-09-10T10:28:37.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8382000000000001e-05 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787708+0000", 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043246059999999999 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787734+0000", 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6086000000000001e-05 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.327542885, 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054527359999999997, 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872472+0000", 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5409999999999999e-06 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872479+0000", 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9550000000000003e-06 2024-09-10T10:28:37.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872494+0000", 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5452999999999999e-05 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872503+0000", 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1509999999999993e-06 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872571+0000", 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7747e-05 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872704+0000", 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013269900000000001 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872804+0000", 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9934999999999998e-05 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877617+0000", 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048138069999999998 2024-09-10T10:28:37.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877819+0000", 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020164799999999999 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877880+0000", 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1296000000000004e-05 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877923+0000", 2024-09-10T10:28:37.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2503999999999998e-05 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.326521318000005, 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0098302110000000002, 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.233 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873493+0000", 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873498+0000", 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1619999999999999e-06 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873511+0000", 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3319000000000001e-05 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873520+0000", 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3409999999999992e-06 2024-09-10T10:28:37.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873586+0000", 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6115e-05 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873660+0000", 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3103999999999997e-05 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:37.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873775+0000", 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115617 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878285+0000", 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045096049999999999 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883229+0000", 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049443999999999998 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883283+0000", 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4262999999999999e-05 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883322+0000", 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8417000000000002e-05 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.238 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.326032178000006, 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047062800000000002, 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873982+0000", 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2900000000000003e-07 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873986+0000", 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3939999999999998e-06 2024-09-10T10:28:37.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873995+0000", 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0750000000000004e-06 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874002+0000", 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874061+0000", 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8770999999999997e-05 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874178+0000", 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117108 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874290+0000", 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112503 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878269+0000", 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039784870000000002 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878593+0000", 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032385299999999998 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878644+0000", 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1790999999999999e-05 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878687+0000", 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2738999999999999e-05 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.325431898999994, 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089185449999999999, 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.244 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874582+0000", 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1699999999999997e-07 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874592+0000", 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0604999999999999e-05 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874602+0000", 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874610+0000", 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1919999999999999e-06 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874696+0000", 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6033999999999997e-05 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874809+0000", 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113373 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874923+0000", 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114077 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878289+0000", 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033662380000000001 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883420+0000", 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051305760000000004 2024-09-10T10:28:37.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883465+0000", 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5252e-05 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883500+0000", 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4415000000000002e-05 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.249 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.251568578000004, 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073932399999999997, 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948446+0000", 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3400000000000001e-06 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948451+0000", 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6219999999999997e-06 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948464+0000", 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2955e-05 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948472+0000", 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0730000000000005e-06 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948534+0000", 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1727e-05 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948649+0000", 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114991 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:37.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948757+0000", 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107935 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950179+0000", 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014219440000000001 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055817970000000003 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955794+0000", 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3012000000000002e-05 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955838+0000", 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3844000000000002e-05 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.256 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.256 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:37.256 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.251029759000005, 2024-09-10T10:28:37.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069680200000000001, 2024-09-10T10:28:37.256 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.256 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.256 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.256 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948984+0000", 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6100000000000001e-07 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948993+0000", 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9809999999999992e-06 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949002+0000", 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8789999999999992e-06 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949008+0000", 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2759999999999997e-06 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949057+0000", 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9159999999999997e-05 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949128+0000", 2024-09-10T10:28:37.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1292999999999997e-05 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949261+0000", 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132564 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950220+0000", 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00095953300000000004 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955890+0000", 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056692790000000002 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955928+0000", 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8278000000000002e-05 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955951+0000", 2024-09-10T10:28:37.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3215999999999999e-05 2024-09-10T10:28:37.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.262 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.262 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.262 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:37.262 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.279053249, 2024-09-10T10:28:37.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050489669999999997, 2024-09-10T10:28:37.262 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5799999999999999e-07 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920964+0000", 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6720000000000002e-06 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920971+0000", 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5140000000000004e-06 2024-09-10T10:28:37.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920977+0000", 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0100000000000001e-06 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921033+0000", 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5899000000000001e-05 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921172+0000", 2024-09-10T10:28:37.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139011 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921288+0000", 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011605200000000001 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922851+0000", 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001562499 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925915+0000", 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003063727 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925969+0000", 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4083999999999999e-05 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926009+0000", 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9941000000000003e-05 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.267 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.278446392000006, 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045836510000000002, 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:37.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4499999999999996e-07 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921572+0000", 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3239999999999996e-06 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921581+0000", 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9190000000000005e-06 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921589+0000", 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0649999999999999e-06 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921635+0000", 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6863999999999997e-05 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921705+0000", 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9046000000000003e-05 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921818+0000", 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011313499999999999 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923033+0000", 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012153590000000001 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926055+0000", 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030223310000000001 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926104+0000", 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8421999999999998e-05 2024-09-10T10:28:37.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926150+0000", 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6440999999999998e-05 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.278007447999997, 2024-09-10T10:28:37.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051621749999999997, 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3600000000000003e-07 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922011+0000", 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4449999999999998e-06 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922019+0000", 2024-09-10T10:28:37.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0930000000000003e-06 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922025+0000", 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5060000000000001e-06 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922076+0000", 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0671000000000003e-05 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922175+0000", 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9611000000000006e-05 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922278+0000", 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102565 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925217+0000", 2024-09-10T10:28:37.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029384670000000002 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927074+0000", 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018579499999999999 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927129+0000", 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4428000000000001e-05 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927168+0000", 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8803000000000001e-05 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.277971777999994, 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059957930000000001, 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.279 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922042+0000", 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922045+0000", 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4209999999999999e-06 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922052+0000", 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1430000000000001e-06 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922059+0000", 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2139999999999999e-06 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922495+0000", 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043517499999999999 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922563+0000", 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8084999999999996e-05 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922679+0000", 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011609599999999999 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927208+0000", 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045292170000000003 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927952+0000", 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00074421700000000003 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928001+0000", 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9004e-05 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928037+0000", 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.578e-05 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.284 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.277945019000001, 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045268130000000002, 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922078+0000", 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922085+0000", 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3710000000000002e-06 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922091+0000", 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0340000000000002e-06 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922144+0000", 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3106999999999998e-05 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922248+0000", 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010373400000000001 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922399+0000", 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015127200000000001 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925201+0000", 2024-09-10T10:28:37.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028019189999999999 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926514+0000", 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001312279 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926561+0000", 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000002e-05 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926595+0000", 2024-09-10T10:28:37.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3408000000000003e-05 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.277310170999996, 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005322116, 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:37.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999996e-07 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922707+0000", 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.766e-06 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922717+0000", 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7229999999999999e-06 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922724+0000", 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8410000000000002e-06 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922792+0000", 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8557000000000001e-05 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922912+0000", 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119354 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928025+0000", 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051131919999999999 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.276834018000002, 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049271260000000004, 2024-09-10T10:28:37.294 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923180+0000", 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923183+0000", 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4000000000000001e-06 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923196+0000", 2024-09-10T10:28:37.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3127e-05 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923203+0000", 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.939e-06 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923270+0000", 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7087000000000003e-05 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923359+0000", 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9054999999999994e-05 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928106+0000", 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047469239999999996 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:37.298 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.276373376999999, 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045860270000000003, 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2500000000000005e-07 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923644+0000", 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8240000000000001e-06 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923659+0000", 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5358000000000001e-05 2024-09-10T10:28:37.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923666+0000", 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9659999999999999e-06 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923725+0000", 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8674000000000001e-05 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923850+0000", 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000124797 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928226+0000", 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043757830000000003 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.258850941999995, 2024-09-10T10:28:37.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005230859, 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941164+0000", 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.863e-06 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941172+0000", 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2360000000000004e-06 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941190+0000", 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7812000000000002e-05 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941200+0000", 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7599999999999997e-06 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941350+0000", 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015070400000000001 2024-09-10T10:28:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941501+0000", 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015044200000000001 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000167076 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943487+0000", 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018190260000000001 2024-09-10T10:28:37.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946297+0000", 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028102840000000001 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946349+0000", 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1649e-05 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946393+0000", 2024-09-10T10:28:37.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4007000000000001e-05 2024-09-10T10:28:37.308 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.308 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.308 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.308 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.308 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.308 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:37.309 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:37.310 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:37.308+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:37.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.326+0000 7fb20f7fe640 1 -- 172.21.15.120:0/924116470 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v57) ==== 87+0+0 (secure 0 0 0) 0x7fb21c07d150 con 0x7fb2280c0760 2024-09-10T10:28:37.327 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is set 2024-09-10T10:28:37.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.328+0000 7fb22d511640 1 -- 172.21.15.120:0/924116470 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb21406be30 msgr2=0x7fb21406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:37.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.328+0000 7fb22d511640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb21406be30 0x7fb21406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2281f1340 tx=0x7fb218009190 comp rx=0 tx=0).stop 2024-09-10T10:28:37.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.328+0000 7fb22d511640 1 -- 172.21.15.120:0/924116470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2280c0760 msgr2=0x7fb22815cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:37.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.328+0000 7fb22d511640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2280c0760 0x7fb22815cb80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009960 tx=0x7fb21c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.329+0000 7fb22d511640 1 -- 172.21.15.120:0/924116470 shutdown_connections 2024-09-10T10:28:37.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.329+0000 7fb22d511640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb21406be30 0x7fb21406e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.329+0000 7fb22d511640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb228154e40 0x7fb228151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.329+0000 7fb22d511640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2280c0760 0x7fb22815cb80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.329+0000 7fb22d511640 1 --2- 172.21.15.120:0/924116470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb22806efa0 0x7fb2280bd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.329+0000 7fb22d511640 1 -- 172.21.15.120:0/924116470 >> 172.21.15.120:0/924116470 conn(0x7fb228067950 msgr2=0x7fb228152f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:37.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.329+0000 7fb22d511640 1 -- 172.21.15.120:0/924116470 shutdown_connections 2024-09-10T10:28:37.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.329+0000 7fb22d511640 1 -- 172.21.15.120:0/924116470 wait complete. 2024-09-10T10:28:37.410 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:37.409+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:37.438 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:37.438 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:37.438 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:37.438 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:37.447 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:37.531 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:37.530+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:37.594 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:37.594 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:37.594 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:37.594 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:37.594 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:37.603 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:37.612 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:37.611+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:37.618 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:28:37.713 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:37.712+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:37.772 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:37.772 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:37.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:37.772 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:37.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.772 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:37.772 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.990105551, 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078690670000000004, 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2200000000000004e-07 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780048+0000", 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2427999999999999e-05 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780057+0000", 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1090000000000004e-06 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780063+0000", 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9780000000000002e-06 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011213480000000001 2024-09-10T10:28:37.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781281+0000", 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6728e-05 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781382+0000", 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101455 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784820+0000", 2024-09-10T10:28:37.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034376200000000002 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787779+0000", 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002959373 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787852+0000", 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3135999999999999e-05 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787894+0000", 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1369999999999999e-05 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.99005801, 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079289789999999992, 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:37.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:37.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780073+0000", 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780075+0000", 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5129999999999999e-06 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780082+0000", 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9410000000000001e-06 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780087+0000", 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1880000000000001e-06 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781620+0000", 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001532346 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781698+0000", 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8232999999999994e-05 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781816+0000", 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011825800000000001 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784824+0000", 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030076349999999998 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787937+0000", 2024-09-10T10:28:37.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031134829999999998 2024-09-10T10:28:37.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787969+0000", 2024-09-10T10:28:37.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1745999999999997e-05 2024-09-10T10:28:37.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788001+0000", 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2132000000000003e-05 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.990027795, 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079888619999999994, 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.784 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780103+0000", 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780106+0000", 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1379999999999999e-06 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780115+0000", 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4299999999999995e-06 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780122+0000", 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3380000000000001e-06 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782041+0000", 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019193700000000001 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782149+0000", 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107493 2024-09-10T10:28:37.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782246+0000", 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999996e-05 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785205+0000", 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002958535 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788041+0000", 2024-09-10T10:28:37.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028359650000000002 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788069+0000", 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8123000000000002e-05 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788091+0000", 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2506000000000001e-05 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.989999254, 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080531070000000003, 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780134+0000", 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0570000000000001e-06 2024-09-10T10:28:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780143+0000", 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-06 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780156+0000", 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3179e-05 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023278280000000001 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782555+0000", 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0494000000000006e-05 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782675+0000", 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120463 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785209+0000", 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002533455 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788127+0000", 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029181260000000001 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788155+0000", 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8495000000000001e-05 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788184+0000", 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8736999999999999e-05 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.989966737, 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008097524, 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.795 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780164+0000", 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.13e-07 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9780000000000001e-06 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780174+0000", 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7510000000000002e-06 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780179+0000", 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0869999999999997e-06 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782890+0000", 2024-09-10T10:28:37.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027113509999999999 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782985+0000", 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5152e-05 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783090+0000", 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104947 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785222+0000", 2024-09-10T10:28:37.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021320269999999999 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788216+0000", 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029940629999999999 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788242+0000", 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5259999999999999e-05 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788261+0000", 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9395e-05 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 110.989944366, 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081619529999999996, 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.801 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780189+0000", 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.926e-06 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780197+0000", 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2039999999999997e-06 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780202+0000", 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783326+0000", 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003123855 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783402+0000", 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6548e-05 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783558+0000", 2024-09-10T10:28:37.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015539199999999999 2024-09-10T10:28:37.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785225+0000", 2024-09-10T10:28:37.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001666989 2024-09-10T10:28:37.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788290+0000", 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030647890000000001 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788316+0000", 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6662e-05 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788348+0000", 2024-09-10T10:28:37.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1417000000000001e-05 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:3 2.10 2:0f65c9b8:::60a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.898350854, 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066829990000000002, 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871781+0000", 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2129999999999998e-06 2024-09-10T10:28:37.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871790+0000", 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3380000000000005e-06 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871805+0000", 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5353999999999999e-05 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871817+0000", 2024-09-10T10:28:37.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1399999999999999e-05 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871932+0000", 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114982 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872069+0000", 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013713199999999999 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872208+0000", 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013976900000000001 2024-09-10T10:28:37.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874946+0000", 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027378789999999999 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878355+0000", 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034089139999999999 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878413+0000", 2024-09-10T10:28:37.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7540000000000001e-05 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878462+0000", 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9478000000000003e-05 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:37.812 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.822907721999997, 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067332750000000004, 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947223+0000", 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7299999999999995e-07 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947227+0000", 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.895e-06 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947235+0000", 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947246+0000", 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0738e-05 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948505+0000", 2024-09-10T10:28:37.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001259402 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948578+0000", 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2788999999999997e-05 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948718+0000", 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013953699999999999 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.816 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953391+0000", 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046736670000000003 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953881+0000", 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048956900000000003 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953923+0000", 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2153000000000001e-05 2024-09-10T10:28:37.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953956+0000", 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2526e-05 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.822848934000007, 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066654160000000004, 2024-09-10T10:28:37.818 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947282+0000", 2024-09-10T10:28:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947285+0000", 2024-09-10T10:28:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7689999999999998e-06 2024-09-10T10:28:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947292+0000", 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6200000000000001e-06 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947304+0000", 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1702000000000001e-05 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947363+0000", 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9426000000000003e-05 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947485+0000", 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012227599999999999 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947588+0000", 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102641 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950690+0000", 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003102264 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953856+0000", 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031656200000000001 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953907+0000", 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1430000000000001e-05 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953947+0000", 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9164e-05 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.822811760999997, 2024-09-10T10:28:37.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071148679999999999, 2024-09-10T10:28:37.825 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.825 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.825 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.825 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:37.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947319+0000", 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947322+0000", 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.523e-06 2024-09-10T10:28:37.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947331+0000", 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3240000000000003e-06 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947336+0000", 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7189999999999998e-06 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947994+0000", 2024-09-10T10:28:37.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065770599999999998 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948080+0000", 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5649000000000006e-05 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111783 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953965+0000", 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057732089999999996 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954343+0000", 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037846199999999998 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954395+0000", 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.126e-05 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954433+0000", 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8693999999999999e-05 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.822681564999996, 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067554449999999997, 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.830 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6899999999999996e-07 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947454+0000", 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947463+0000", 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0310000000000004e-06 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947477+0000", 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4328000000000001e-05 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3869999999999998e-05 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947631+0000", 2024-09-10T10:28:37.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9874e-05 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947750+0000", 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118483 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953644+0000", 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058946889999999998 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954111+0000", 2024-09-10T10:28:37.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00046712099999999998 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954162+0000", 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000003e-05 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954204+0000", 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1718000000000003e-05 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.822294526999997, 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064551169999999998, 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3e-07 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947841+0000", 2024-09-10T10:28:37.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3460000000000004e-06 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947856+0000", 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947864+0000", 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4939999999999997e-06 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947997+0000", 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013305399999999999 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948076+0000", 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9146000000000004e-05 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115586 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953707+0000", 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055153470000000003 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954215+0000", 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050850400000000001 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954258+0000", 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2932000000000002e-05 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954291+0000", 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2322e-05 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.841 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.822255206999998, 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088919089999999999, 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7199999999999999e-07 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947879+0000", 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8380000000000004e-06 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947887+0000", 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5410000000000003e-06 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947893+0000", 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6930000000000004e-06 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948405+0000", 2024-09-10T10:28:37.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051236000000000001 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948502+0000", 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6916999999999994e-05 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948647+0000", 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000145265 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954479+0000", 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058316970000000003 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956680+0000", 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002201494 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956728+0000", 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8161999999999997e-05 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956767+0000", 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.837e-05 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.821692545999994, 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083279530000000008, 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.847 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8999999999999995e-07 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948443+0000", 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2189999999999998e-06 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948450+0000", 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8879999999999997e-06 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948457+0000", 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0750000000000003e-06 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948501+0000", 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948594+0000", 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3009000000000003e-05 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948696+0000", 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101461 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953756+0000", 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050607129999999997 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956690+0000", 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029334109999999999 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956737+0000", 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7080999999999999e-05 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956766+0000", 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8778000000000001e-05 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.821152871999999, 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0077788589999999999, 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948986+0000", 2024-09-10T10:28:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948998+0000", 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.19e-05 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949007+0000", 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.081e-06 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949062+0000", 2024-09-10T10:28:37.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754999999999997e-05 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949172+0000", 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110128 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949282+0000", 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109289 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955941+0000", 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066594990000000001 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956654+0000", 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071279399999999999 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956706+0000", 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2027000000000002e-05 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956756+0000", 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0012999999999997e-05 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.820614778000007, 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073200349999999999, 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.858 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949516+0000", 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999995e-07 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949522+0000", 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3980000000000004e-06 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949534+0000", 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1562e-05 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949541+0000", 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1380000000000001e-06 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950700+0000", 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011593459999999999 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950786+0000", 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5556999999999994e-05 2024-09-10T10:28:37.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950898+0000", 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111743 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954326+0000", 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034280510000000001 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956774+0000", 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024477430000000001 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956815+0000", 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1752999999999998e-05 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956835+0000", 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0032999999999999e-05 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.863 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.820572476999999, 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073303700000000001, 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:37.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949571+0000", 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2298e-05 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949579+0000", 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5909999999999996e-06 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949585+0000", 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7370000000000003e-06 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953802+0000", 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042170929999999999 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953915+0000", 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011256699999999999 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956888+0000", 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029734639999999999 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:24 2.3 e26/18 2:c216c0d1:::mds2_inotable:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 83.819754476, 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0063254399999999999, 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.868 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.869 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950380+0000", 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2590000000000002e-06 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950387+0000", 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950392+0000", 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950438+0000", 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5592e-05 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950457+0000", 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8569000000000001e-05 2024-09-10T10:28:37.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956657+0000", 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062003409999999998 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956701+0000", 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4140000000000001e-05 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:29 2.10 2:0e4da237:::203.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 77.848093672999994, 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064228799999999997, 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922037+0000", 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922040+0000", 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.694e-06 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922054+0000", 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906e-05 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922065+0000", 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922125+0000", 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.982e-05 2024-09-10T10:28:37.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922233+0000", 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108175 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928459+0000", 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062264570000000003 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.828830042999996, 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081214100000000008, 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941301+0000", 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.062e-06 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941306+0000", 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0919999999999997e-06 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941316+0000", 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5640000000000002e-06 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941323+0000", 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941405+0000", 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1650000000000006e-05 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941549+0000", 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014447599999999999 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012929 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943549+0000", 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018704049999999999 2024-09-10T10:28:37.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949343+0000", 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057943300000000003 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949378+0000", 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4474999999999998e-05 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949421+0000", 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3955999999999997e-05 2024-09-10T10:28:37.881 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.882 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.882 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.882 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:37.882 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:37.882 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:37.883 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:37.888 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:37.882+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:37.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.789+0000 7fa9580ba640 1 Processor -- start 2024-09-10T10:28:37.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.789+0000 7fa9580ba640 1 -- start start 2024-09-10T10:28:37.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.790+0000 7fa9580ba640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa9500cca70 0x7fa95015b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.790+0000 7fa9580ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9500cafa0 0x7fa9500cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.790+0000 7fa9580ba640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95015e0b0 con 0x7fa950154c10 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.790+0000 7fa9580ba640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa95015de60 con 0x7fa9500cca70 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.790+0000 7fa9580ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa95015e390 con 0x7fa9500cafa0 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.790+0000 7fa955655640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa9500cca70 0x7fa95015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.790+0000 7fa955e56640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fa950154c10 0x7fa950073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:42022/0 (socket says 172.21.15.120:42022) 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.790+0000 7fa955e56640 1 -- 172.21.15.120:0/3473404092 learned_addr learned my addr 172.21.15.120:0/3473404092 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.791+0000 7fa956657640 1 --2- 172.21.15.120:0/3473404092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9500cafa0 0x7fa9500cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.791+0000 7fa955655640 1 -- 172.21.15.120:0/3473404092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9500cafa0 msgr2=0x7fa9500cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.791+0000 7fa955655640 1 --2- 172.21.15.120:0/3473404092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9500cafa0 0x7fa9500cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.791+0000 7fa955655640 1 -- 172.21.15.120:0/3473404092 >> v1:172.21.15.120:6789/0 conn(0x7fa950154c10 legacy=0x7fa950073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.791+0000 7fa955655640 1 -- 172.21.15.120:0/3473404092 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95015e670 con 0x7fa9500cca70 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.792+0000 7fa955655640 1 --2- 172.21.15.120:0/3473404092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9500cca70 0x7fa95015b370 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa940009790 tx=0x7fa94002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5904cacf2f2f85e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.792+0000 7fa954e54640 1 -- 172.21.15.120:0/3473404092 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940037070 con 0x7fa9500cca70 2024-09-10T10:28:37.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.792+0000 7fa954e54640 1 -- 172.21.15.120:0/3473404092 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa940032440 con 0x7fa9500cca70 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.792+0000 7fa954e54640 1 -- 172.21.15.120:0/3473404092 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940032850 con 0x7fa9500cca70 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.793+0000 7fa9580ba640 1 -- 172.21.15.120:0/3473404092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9500cca70 msgr2=0x7fa95015b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.793+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3473404092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9500cca70 0x7fa95015b370 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa940009790 tx=0x7fa94002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.793+0000 7fa9580ba640 1 -- 172.21.15.120:0/3473404092 shutdown_connections 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.793+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3473404092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9500cafa0 0x7fa9500cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.793+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3473404092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9500cca70 0x7fa95015b370 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.793+0000 7fa9580ba640 1 -- 172.21.15.120:0/3473404092 >> 172.21.15.120:0/3473404092 conn(0x7fa95006ce80 msgr2=0x7fa95006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.793+0000 7fa9580ba640 1 -- 172.21.15.120:0/3473404092 shutdown_connections 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.793+0000 7fa9580ba640 1 -- 172.21.15.120:0/3473404092 wait complete. 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.795+0000 7fa9580ba640 1 Processor -- start 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.795+0000 7fa9580ba640 1 -- start start 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa9580ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9500cafa0 0x7fa950150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa9580ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9500cca70 0x7fa9501f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa9580ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa950154c10 0x7fa9501f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa9580ba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa950154ff0 con 0x7fa950154c10 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa9580ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa95015ec30 con 0x7fa9500cca70 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa9580ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa950073a50 con 0x7fa9500cafa0 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa956657640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa950154c10 0x7fa9501f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa956657640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa950154c10 0x7fa9501f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60932/0 (socket says 172.21.15.120:60932) 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa956657640 1 -- 172.21.15.120:0/3827655247 learned_addr learned my addr 172.21.15.120:0/3827655247 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa955e56640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9500cafa0 0x7fa950150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.796+0000 7fa955655640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9500cca70 0x7fa9501f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa956657640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9500cafa0 msgr2=0x7fa950150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa956657640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9500cafa0 0x7fa950150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa956657640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9500cca70 msgr2=0x7fa9501f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa956657640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9500cca70 0x7fa9501f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa956657640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9501f4af0 con 0x7fa950154c10 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa956657640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa950154c10 0x7fa9501f44d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa948007e40 tx=0x7fa94800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948014620 con 0x7fa950154c10 2024-09-10T10:28:37.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa948019070 con 0x7fa950154c10 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948014a20 con 0x7fa950154c10 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95015d640 con 0x7fa950154c10 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.797+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95015ca00 con 0x7fa950154c10 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.798+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fa95015db00 con 0x7fa950154c10 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.799+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa948014bc0 con 0x7fa950154c10 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.800+0000 7fa9467fc640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa93406be30 0x7fa93406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.800+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7015+0+0 (secure 0 0 0) 0x7fa94808bb70 con 0x7fa950154c10 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.800+0000 7fa955e56640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa93406be30 0x7fa93406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.800+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v57) ==== 54+0+16 (secure 0 0 0) 0x7fa95015d640 con 0x7fa950154c10 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.800+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fa950151280 0x7fa95014f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.800+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa950153a80 con 0x7fa950151280 2024-09-10T10:28:37.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.801+0000 7fa955e56640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa93406be30 0x7fa93406e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa94c000f60 tx=0x7fa94c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.801+0000 7fa955655640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fa950151280 0x7fa95014f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.801+0000 7fa955655640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fa950151280 0x7fa95014f0c0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.802+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa950153a80 con 0x7fa950151280 2024-09-10T10:28:37.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.817+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa9500cb830 con 0x7fa950151280 2024-09-10T10:28:37.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.818+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa9500cb830 con 0x7fa950151280 2024-09-10T10:28:37.893 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:37.894 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:37.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.818+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fa950151c30 0x7fa95014dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.818+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fa9500cb550 con 0x7fa950151c30 2024-09-10T10:28:37.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.819+0000 7fa955e56640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fa950151c30 0x7fa95014dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.819+0000 7fa955e56640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fa950151c30 0x7fa95014dda0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.820+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9500cb550 con 0x7fa950151c30 2024-09-10T10:28:37.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.835+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa95015cd70 con 0x7fa950151c30 2024-09-10T10:28:37.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.838+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa95015cd70 con 0x7fa950151c30 2024-09-10T10:28:37.895 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:37.896 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:37.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.838+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fa9501f0cc0 0x7fa9501f10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.838+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fa9501f1620 con 0x7fa9501f0cc0 2024-09-10T10:28:37.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.838+0000 7fa955655640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fa9501f0cc0 0x7fa9501f10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.838+0000 7fa955655640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fa9501f0cc0 0x7fa9501f10e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.839+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9501f1620 con 0x7fa9501f0cc0 2024-09-10T10:28:37.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.853+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa950062630 con 0x7fa9501f0cc0 2024-09-10T10:28:37.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.854+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa950062630 con 0x7fa9501f0cc0 2024-09-10T10:28:37.897 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:37.898 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:37.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.854+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fa950063110 0x7fa9501e6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.854+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fa950063530 con 0x7fa950063110 2024-09-10T10:28:37.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.855+0000 7fa956657640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fa950063110 0x7fa9501e6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.855+0000 7fa956657640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fa950063110 0x7fa9501e6ea0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.856+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa950063530 con 0x7fa950063110 2024-09-10T10:28:37.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.882+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa9501e76b0 con 0x7fa950063110 2024-09-10T10:28:37.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.884+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa9501e76b0 con 0x7fa950063110 2024-09-10T10:28:37.899 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.885+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fa95002c640 0x7fa95002ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.885+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fa950063530 con 0x7fa95002c640 2024-09-10T10:28:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.885+0000 7fa955e56640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fa95002c640 0x7fa95002ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.885+0000 7fa955e56640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fa95002c640 0x7fa95002ca80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.886+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa950063530 con 0x7fa95002c640 2024-09-10T10:28:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.913+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa950063530 con 0x7fa95002c640 2024-09-10T10:28:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.915+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa950063530 con 0x7fa95002c640 2024-09-10T10:28:37.916 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.915+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fa95002d4d0 0x7fa9502070b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.915+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fa95002d930 con 0x7fa95002d4d0 2024-09-10T10:28:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.916+0000 7fa955655640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fa95002d4d0 0x7fa9502070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.916+0000 7fa955655640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fa95002d4d0 0x7fa9502070b0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.917+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa95002d930 con 0x7fa95002d4d0 2024-09-10T10:28:37.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.944+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa95002d930 con 0x7fa95002d4d0 2024-09-10T10:28:37.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.946+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa95002d930 con 0x7fa95002d4d0 2024-09-10T10:28:37.949 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:37.950 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:37.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.946+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fa950207b00 0x7fa950209ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.946+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fa95020ccc0 con 0x7fa950207b00 2024-09-10T10:28:37.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.947+0000 7fa956657640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fa950207b00 0x7fa950209ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.947+0000 7fa956657640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fa950207b00 0x7fa950209ef0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.948+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa95020ccc0 con 0x7fa950207b00 2024-09-10T10:28:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.975+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa95020a430 con 0x7fa950207b00 2024-09-10T10:28:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.977+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa95020a430 con 0x7fa950207b00 2024-09-10T10:28:37.978 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.977+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fa95020ab50 0x7fa95020af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.977+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fa950210090 con 0x7fa95020ab50 2024-09-10T10:28:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.978+0000 7fa955e56640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fa95020ab50 0x7fa95020af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.978+0000 7fa955e56640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fa95020ab50 0x7fa95020af70 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:37.979+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa950210090 con 0x7fa95020ab50 2024-09-10T10:28:37.985 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:37.983+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:38.006 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:38.006 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.006+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa95020b5c0 con 0x7fa95020ab50 2024-09-10T10:28:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9467fc640 1 -- 172.21.15.120:0/3827655247 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa95020b5c0 con 0x7fa95020ab50 2024-09-10T10:28:38.009 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fa950151280 msgr2=0x7fa95014f0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fa950151280 0x7fa95014f0c0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fa950151c30 msgr2=0x7fa95014dda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fa950151c30 0x7fa95014dda0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fa9501f0cc0 msgr2=0x7fa9501f10e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fa9501f0cc0 0x7fa9501f10e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa956657640 1 -- 172.21.15.120:0/3827655247 reap_dead start 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fa950063110 msgr2=0x7fa9501e6ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fa950063110 0x7fa9501e6ea0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fa95002c640 msgr2=0x7fa95002ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fa95002c640 0x7fa95002ca80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fa95002d4d0 msgr2=0x7fa9502070b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fa95002d4d0 0x7fa9502070b0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fa950207b00 msgr2=0x7fa950209ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.008+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fa950207b00 0x7fa950209ef0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fa95020ab50 msgr2=0x7fa95020af70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fa95020ab50 0x7fa95020af70 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa956657640 1 -- 172.21.15.120:0/3827655247 reap_dead start 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa93406be30 msgr2=0x7fa93406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa93406be30 0x7fa93406e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa94c000f60 tx=0x7fa94c009040 comp rx=0 tx=0).stop 2024-09-10T10:28:38.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa950154c10 msgr2=0x7fa9501f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa950154c10 0x7fa9501f44d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa948007e40 tx=0x7fa94800e4b0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 shutdown_connections 2024-09-10T10:28:38.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa93406be30 0x7fa93406e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 --2- 172.21.15.120:0/3827655247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa950154c10 0x7fa9501f44d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 >> 172.21.15.120:0/3827655247 conn(0x7fa95006ce80 msgr2=0x7fa9500c55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:38.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 shutdown_connections 2024-09-10T10:28:38.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.009+0000 7fa9580ba640 1 -- 172.21.15.120:0/3827655247 wait complete. 2024-09-10T10:28:38.017 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:38.085 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:38.084+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:38.180 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:38.180 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:38.186 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:38.185+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:38.192 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:38.287 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:38.286+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:38.306 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:38.306 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:38.315 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:38.329 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:28:38.330 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:28:38.330 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-10T10:28:38.330 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-10T10:28:38.388 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:38.386+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:38.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.429+0000 7f38e974b640 1 Processor -- start 2024-09-10T10:28:38.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.429+0000 7f38e974b640 1 -- start start 2024-09-10T10:28:38.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e974b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e4158170 0x7f38e415e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e974b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f38e415ec60 0x7f38e406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e974b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f38e40c0510 0x7f38e406f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e974b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f38e415dfb0 con 0x7f38e415ec60 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e974b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f38e406d1c0 con 0x7f38e40c0510 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e974b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f38e40c79c0 con 0x7f38e4158170 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e27fc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f38e415ec60 0x7f38e406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e27fc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f38e415ec60 0x7f38e406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60936/0 (socket says 172.21.15.120:60936) 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e27fc640 1 -- 172.21.15.120:0/1636864518 learned_addr learned my addr 172.21.15.120:0/1636864518 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e37fe640 1 --2- 172.21.15.120:0/1636864518 >> v2:172.21.15.134:3300/0 conn(0x7f38e40c0510 0x7f38e406f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e2ffd640 1 --2- 172.21.15.120:0/1636864518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e4158170 0x7f38e415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e27fc640 1 -- 172.21.15.120:0/1636864518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e4158170 msgr2=0x7f38e415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:38.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e27fc640 1 --2- 172.21.15.120:0/1636864518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e4158170 0x7f38e415e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e27fc640 1 -- 172.21.15.120:0/1636864518 >> v2:172.21.15.134:3300/0 conn(0x7f38e40c0510 msgr2=0x7f38e406f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:38.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e27fc640 1 --2- 172.21.15.120:0/1636864518 >> v2:172.21.15.134:3300/0 conn(0x7f38e40c0510 0x7f38e406f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.430+0000 7f38e27fc640 1 -- 172.21.15.120:0/1636864518 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e40c7ca0 con 0x7f38e415ec60 2024-09-10T10:28:38.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.431+0000 7f38e27fc640 1 --2- 172.21.15.120:0/1636864518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 0x7f38e406eb00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f38e40c04a0 tx=0x7f38cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24808f11565fb6a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:38.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.431+0000 7f38e1ffb640 1 -- 172.21.15.120:0/1636864518 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc037070 con 0x7f38e415ec60 2024-09-10T10:28:38.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.431+0000 7f38e1ffb640 1 -- 172.21.15.120:0/1636864518 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38cc032440 con 0x7f38e415ec60 2024-09-10T10:28:38.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.431+0000 7f38e1ffb640 1 -- 172.21.15.120:0/1636864518 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc032810 con 0x7f38e415ec60 2024-09-10T10:28:38.435 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:38.435 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:38.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.432+0000 7f38e974b640 1 -- 172.21.15.120:0/1636864518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 msgr2=0x7f38e406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.432+0000 7f38e974b640 1 --2- 172.21.15.120:0/1636864518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 0x7f38e406eb00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f38e40c04a0 tx=0x7f38cc02a270 comp rx=0 tx=0).stop 2024-09-10T10:28:38.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.432+0000 7f38e974b640 1 -- 172.21.15.120:0/1636864518 shutdown_connections 2024-09-10T10:28:38.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.432+0000 7f38e974b640 1 --2- 172.21.15.120:0/1636864518 >> v2:172.21.15.134:3300/0 conn(0x7f38e40c0510 0x7f38e406f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.432+0000 7f38e974b640 1 --2- 172.21.15.120:0/1636864518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 0x7f38e406eb00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.432+0000 7f38e974b640 1 --2- 172.21.15.120:0/1636864518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e4158170 0x7f38e415e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.432+0000 7f38e974b640 1 -- 172.21.15.120:0/1636864518 >> 172.21.15.120:0/1636864518 conn(0x7f38e4067950 msgr2=0x7f38e4067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.432+0000 7f38e974b640 1 -- 172.21.15.120:0/1636864518 shutdown_connections 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.432+0000 7f38e974b640 1 -- 172.21.15.120:0/1636864518 wait complete. 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.433+0000 7f38e974b640 1 Processor -- start 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.433+0000 7f38e974b640 1 -- start start 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e974b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e40c0510 0x7f38e40c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e974b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e4158170 0x7f38e406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e974b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 0x7f38e41f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e974b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38e40c9ac0 con 0x7f38e415ec60 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e974b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38e415e200 con 0x7f38e4158170 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e974b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f38e406bc10 con 0x7f38e40c0510 2024-09-10T10:28:38.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 0x7f38e41f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 0x7f38e41f44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60938/0 (socket says 172.21.15.120:60938) 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e37fe640 1 -- 172.21.15.120:0/445232344 learned_addr learned my addr 172.21.15.120:0/445232344 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e27fc640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e4158170 0x7f38e406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e2ffd640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e40c0510 0x7f38e40c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e37fe640 1 -- 172.21.15.120:0/445232344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e40c0510 msgr2=0x7f38e40c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e37fe640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e40c0510 0x7f38e40c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e37fe640 1 -- 172.21.15.120:0/445232344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e4158170 msgr2=0x7f38e406ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e37fe640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e4158170 0x7f38e406ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.434+0000 7f38e37fe640 1 -- 172.21.15.120:0/445232344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e40c3530 con 0x7f38e415ec60 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.435+0000 7f38e37fe640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 0x7f38e41f44a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00ef00 tx=0x7f38dc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.435+0000 7f38db7fe640 1 -- 172.21.15.120:0/445232344 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc018020 con 0x7f38e415ec60 2024-09-10T10:28:38.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.435+0000 7f38db7fe640 1 -- 172.21.15.120:0/445232344 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38dc014400 con 0x7f38e415ec60 2024-09-10T10:28:38.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.435+0000 7f38e974b640 1 -- 172.21.15.120:0/445232344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e40c2140 con 0x7f38e415ec60 2024-09-10T10:28:38.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.435+0000 7f38db7fe640 1 -- 172.21.15.120:0/445232344 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0147f0 con 0x7f38e415ec60 2024-09-10T10:28:38.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.435+0000 7f38e974b640 1 -- 172.21.15.120:0/445232344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e40c3b00 con 0x7f38e415ec60 2024-09-10T10:28:38.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.436+0000 7f38e974b640 1 -- 172.21.15.120:0/445232344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e406cb00 con 0x7f38e415ec60 2024-09-10T10:28:38.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.436+0000 7f38db7fe640 1 -- 172.21.15.120:0/445232344 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38dc005d10 con 0x7f38e415ec60 2024-09-10T10:28:38.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.437+0000 7f38db7fe640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f38c406be30 0x7f38c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.437+0000 7f38db7fe640 1 -- 172.21.15.120:0/445232344 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7015+0+0 (secure 0 0 0) 0x7f38dc08bbc0 con 0x7f38e415ec60 2024-09-10T10:28:38.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.437+0000 7f38e2ffd640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f38c406be30 0x7f38c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.437+0000 7f38e2ffd640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f38c406be30 0x7f38c406e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009b40 tx=0x7f38d4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:38.442 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:38.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.441+0000 7f38db7fe640 1 -- 172.21.15.120:0/445232344 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f38e406cb00 con 0x7f38e415ec60 2024-09-10T10:28:38.528 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:38.526+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:38.587 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:38.587 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:38.589 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:38.588+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:38.596 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:38.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.610+0000 7f38e974b640 1 -- 172.21.15.120:0/445232344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f38e406ce80 con 0x7f38e415ec60 2024-09-10T10:28:38.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.611+0000 7f38db7fe640 1 -- 172.21.15.120:0/445232344 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v57) ==== 81+0+40353 (secure 0 0 0) 0x7f38dc022030 con 0x7f38e415ec60 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 57, 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: "fsid": "7d0d0830-4370-41c3-9acc-b67b07024f68", 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: "created": "2024-09-10T10:26:18.018818+0000", 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: "modified": "2024-09-10T10:28:37.319151+0000", 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: "last_up_change": "2024-09-10T10:26:29.078139+0000", 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: "last_in_change": "2024-09-10T10:26:20.091201+0000", 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_num": 5801984, 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_set": [ 2024-09-10T10:28:38.613 INFO:teuthology.orchestra.run.smithi120.stdout: "noscrub", 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "pglog_hardlimit", 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "purged_snapdirs", 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_deletes", 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "sortbitwise" 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_version": 5, 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "full_ratio": 0.94999998807907104, 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_snapshot": "", 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_max": 4, 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd": 8, 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "require_min_compat_client": "luminous", 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "min_compat_client": "luminous", 2024-09-10T10:28:38.614 INFO:teuthology.orchestra.run.smithi120.stdout: "require_osd_release": "squid", 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_crimson": false, 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "pools": [ 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 1, 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": ".mgr", 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:30.118649+0000", 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:38.615 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 1, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 1, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 1, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 1, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 1, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:38.616 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "17", 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:38.617 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:38.618 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:38.619 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_max": 32, 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 1 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr": {} 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 8, 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 8, 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 0.25, 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 2, 2024-09-10T10:28:38.620 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 2, 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 2, 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_metadata", 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:38.936606+0000", 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:28:38.621 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:28:38.622 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "20", 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:38.623 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:38.624 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:38.625 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_bias": 4, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 16, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_priority": 5 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:28:38.626 INFO:teuthology.orchestra.run.smithi120.stdout: "metadata": "cephfs" 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.5, 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.5, 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.5, 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.5, 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 3, 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data", 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:40.094780+0000", 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool", 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 1, 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 2, 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 1, 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 0, 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:38.628 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 64, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 64, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 64, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 64, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 64, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:38.629 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "21", 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:38.630 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "", 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:38.631 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 0, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:38.632 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.05000000074505806 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "read_balance": { 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "score_type": "Fair distribution", 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "score_acting": 1.25, 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "score_stable": 1.25, 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "optimal_score": 1, 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_acting": 1.25, 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "raw_score_stable": 1.25, 2024-09-10T10:28:38.633 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity_weighted": 1, 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity": 1, 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "average_primary_affinity_weighted": 1 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "pool": 4, 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": "cephfs_data_ec", 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "create_time": "2024-09-10T10:26:43.814005+0000", 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 5, 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "flags_names": "hashpspool,ec_overwrites", 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "type": 3, 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 4, 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": 3, 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_rule": 1, 2024-09-10T10:28:38.634 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_count": 0, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_target": 0, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "is_stretch_pool": false, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "object_hash": 2, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscale_mode": "off", 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": 32, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num": 32, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_placement_num_target": 32, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_target": 32, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_pending": 32, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "last_pg_merge_meta": { 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "source_pgid": "0.0", 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "ready_epoch": 0, 2024-09-10T10:28:38.635 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_started": 0, 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "last_epoch_clean": 0, 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "source_version": "0'0", 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "target_version": "0'0" 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "last_change": "26", 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend": "0", 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "auid": 0, 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_mode": "selfmanaged", 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_seq": 0, 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "snap_epoch": 0, 2024-09-10T10:28:38.636 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_snaps": [], 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps": "[]", 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_bytes": 0, 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "quota_max_objects": 0, 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "tiers": [], 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "tier_of": -1, 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "read_tier": -1, 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "write_tier": -1, 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_mode": "none", 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_bytes": 0, 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "target_max_objects": 0, 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T10:28:38.637 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_flush_age": 0, 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "cache_min_evict_age": 0, 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profile": "cephfs_data_ec", 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_params": { 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "none" 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_period": 0, 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_count": 0, 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "use_gmt_hitset": true, 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "min_read_recency_for_promote": 0, 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "min_write_recency_for_promote": 0, 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "hit_set_search_last_n": 0, 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "grade_table": [], 2024-09-10T10:28:38.638 INFO:teuthology.orchestra.run.smithi120.stdout: "stripe_width": 8192, 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "expected_num_objects": 0, 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "fast_read": false, 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "options": { 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num_min": 64, 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "application_metadata": { 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs": { 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "data": "cephfs" 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-10T10:28:38.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e3b9c3c3-cf7d-4b33-913e-60e928439d8e", 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 50, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6808", 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6809", 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6810", 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6811", 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.642 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6814", 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6815", 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6812", 2024-09-10T10:28:38.643 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6813", 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3664783490 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6809/3664783490", 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6811/3664783490", 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6815/3664783490", 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6813/3664783490", 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:38.644 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "62c20a9d-e6ab-446d-998c-9ee46e46aa90", 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 49, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:38.645 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.646 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6802", 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6803", 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.647 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6806", 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6807", 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6804", 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6805", 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3972761964 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6801/3972761964", 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6803/3972761964", 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6807/3972761964", 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6805/3972761964", 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "1528f8e0-5e13-4ec7-9d98-0d94b7a505c3", 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:38.650 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 50, 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6816", 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.651 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6817", 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6818", 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.652 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6819", 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6822", 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6823", 2024-09-10T10:28:38.653 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6820", 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6821", 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2916410859 2024-09-10T10:28:38.654 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6817/2916410859", 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6819/2916410859", 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6823/2916410859", 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6821/2916410859", 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.655 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "74942f78-c1fc-4886-abf0-950722c5edc6", 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 50, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:38.656 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6824", 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6825", 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:38.657 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6826", 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6827", 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:38.658 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6830", 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6831", 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:38.659 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6828", 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6829", 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1224278950 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.120:6825/1224278950", 2024-09-10T10:28:38.660 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.120:6827/1224278950", 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.120:6831/1224278950", 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.120:6829/1224278950", 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "e71319d4-5318-4441-84a8-7665ae037596", 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 0.9353485107421875, 2024-09-10T10:28:38.661 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 50, 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6816", 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6817", 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6818", 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.663 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6819", 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6822", 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.664 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6823", 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6820", 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6821", 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1874635282 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6817/1874635282", 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6819/1874635282", 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6823/1874635282", 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6821/1874635282", 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:38.666 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "29e193cd-37af-43e5-bd0c-2f85ce0631e8", 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 50, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:38.667 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6808", 2024-09-10T10:28:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6809", 2024-09-10T10:28:38.668 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6810", 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6811", 2024-09-10T10:28:38.669 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6814", 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6815", 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:38.670 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6812", 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6813", 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 457724087 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.671 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6809/457724087", 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6811/457724087", 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6815/457724087", 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6813/457724087", 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "55523ec2-cf12-44b3-aeea-f1cedabc996a", 2024-09-10T10:28:38.672 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 50, 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.673 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6824", 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6825", 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.674 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6826", 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6827", 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6830", 2024-09-10T10:28:38.675 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6831", 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.676 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6828", 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6829", 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 50133702 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6825/50133702", 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6827/50133702", 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6831/50133702", 2024-09-10T10:28:38.677 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6829/50133702", 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uuid": "097b1149-3e3c-4e5a-890b-d4cc4361d51b", 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "up": 1, 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "in": 1, 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "weight": 1, 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_affinity": 1, 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_begin": 0, 2024-09-10T10:28:38.678 INFO:teuthology.orchestra.run.smithi120.stdout: "last_clean_end": 0, 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "up_from": 13, 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "up_thru": 35, 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "down_at": 0, 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "lost_at": 0, 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addrs": { 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6800", 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6801", 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:38.679 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addrs": { 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6802", 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6803", 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:38.680 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addrs": { 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6806", 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6807", 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:38.681 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addrs": { 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6804", 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.134:6805", 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 251960875 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.682 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "public_addr": "172.21.15.134:6801/251960875", 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_addr": "172.21.15.134:6803/251960875", 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_back_addr": "172.21.15.134:6807/251960875", 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "heartbeat_front_addr": "172.21.15.134:6805/251960875", 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "state": [ 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "exists", 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "up" 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_xinfo": [ 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.683 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 0, 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.143056+0000", 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 1, 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:38.684 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.423968+0000", 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 2, 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.640076+0000", 2024-09-10T10:28:38.685 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 3, 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.536160+0000", 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 4, 2024-09-10T10:28:38.686 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.146033+0000", 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 5, 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:38.687 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.426011+0000", 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 6, 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.598992+0000", 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "osd": 7, 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "down_stamp": "0.000000", 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_probability": 0, 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "laggy_interval": 0, 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "features": 4540701547738038271, 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "old_weight": 0, 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "last_purged_snaps_scrub": "2024-09-10T10:26:26.182929+0000", 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "dead_epoch": 0 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap": [ 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.689 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.2c", 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: 1, 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: 7 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.34", 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: "osds": [ 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: 3, 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: 1 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:38.690 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_items": [ 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "2.1d", 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 6, 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 4 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.7", 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:28:38.691 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 6 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.16", 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 1 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.1d", 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.28", 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.693 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.2d", 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 7, 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 6 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.33", 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "3.36", 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 5, 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.4", 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 6 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 5 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.d", 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 5 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.10", 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:28:38.697 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 5 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.1b", 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 1, 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 0 2024-09-10T10:28:38.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: "pgid": "4.1e", 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: "mappings": [ 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: "from": 3, 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: "to": 2 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_upmap_primaries": [], 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_temp": [], 2024-09-10T10:28:38.699 INFO:teuthology.orchestra.run.smithi120.stdout: "primary_temp": [], 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "blocklist": {}, 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "range_blocklist": {}, 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "erasure_code_profiles": { 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "cephfs_data_ec": { 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-device-class": "", 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-num-failure-domains": "0", 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-osds-per-failure-domain": "0", 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-root": "default", 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "jerasure-per-chunk-alignment": "false", 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "2", 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van", 2024-09-10T10:28:38.700 INFO:teuthology.orchestra.run.smithi120.stdout: "w": "8" 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "default": { 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-failure-domain": "osd", 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "k": "2", 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "m": "1", 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "plugin": "jerasure", 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "technique": "reed_sol_van" 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "removed_snaps_queue": [], 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "new_removed_snaps": [], 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "new_purged_snaps": [], 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_node_flags": {}, 2024-09-10T10:28:38.701 INFO:teuthology.orchestra.run.smithi120.stdout: "device_class_flags": {}, 2024-09-10T10:28:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode": { 2024-09-10T10:28:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_enabled": false, 2024-09-10T10:28:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_bucket_count": 0, 2024-09-10T10:28:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: "degraded_stretch_mode": 0, 2024-09-10T10:28:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: "recovering_stretch_mode": 0, 2024-09-10T10:28:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: "stretch_mode_bucket": 0 2024-09-10T10:28:38.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:38.702 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:38.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 -- 172.21.15.120:0/445232344 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f38c406be30 msgr2=0x7f38c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f38c406be30 0x7f38c406e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009b40 tx=0x7f38d4009190 comp rx=0 tx=0).stop 2024-09-10T10:28:38.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 -- 172.21.15.120:0/445232344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 msgr2=0x7f38e41f44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 0x7f38e41f44a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00ef00 tx=0x7f38dc00b480 comp rx=0 tx=0).stop 2024-09-10T10:28:38.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 -- 172.21.15.120:0/445232344 shutdown_connections 2024-09-10T10:28:38.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f38c406be30 0x7f38c406e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38e415ec60 0x7f38e41f44a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e4158170 0x7f38e406ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 --2- 172.21.15.120:0/445232344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e40c0510 0x7f38e40c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 -- 172.21.15.120:0/445232344 >> 172.21.15.120:0/445232344 conn(0x7f38e4067950 msgr2=0x7f38e4067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:38.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 -- 172.21.15.120:0/445232344 shutdown_connections 2024-09-10T10:28:38.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.613+0000 7f38e974b640 1 -- 172.21.15.120:0/445232344 wait complete. 2024-09-10T10:28:38.706 INFO:tasks.thrashosds.thrasher:j is {'epoch': 57, 'fsid': '7d0d0830-4370-41c3-9acc-b67b07024f68', 'created': '2024-09-10T10:26:18.018818+0000', 'modified': '2024-09-10T10:28:37.319151+0000', 'last_up_change': '2024-09-10T10:26:29.078139+0000', 'last_in_change': '2024-09-10T10:26:20.091201+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:26:30.118649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:26:38.936606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:26:40.094780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T10:26:43.814005+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'e3b9c3c3-cf7d-4b33-913e-60e928439d8e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6808', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6809', 'nonce': 3664783490}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6810', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6811', 'nonce': 3664783490}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6814', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6815', 'nonce': 3664783490}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6812', 'nonce': 3664783490}, {'type': 'v1', 'addr': '172.21.15.120:6813', 'nonce': 3664783490}]}, 'public_addr': '172.21.15.120:6809/3664783490', 'cluster_addr': '172.21.15.120:6811/3664783490', 'heartbeat_back_addr': '172.21.15.120:6815/3664783490', 'heartbeat_front_addr': '172.21.15.120:6813/3664783490', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '62c20a9d-e6ab-446d-998c-9ee46e46aa90', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6800', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6801', 'nonce': 3972761964}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6802', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6803', 'nonce': 3972761964}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6806', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6807', 'nonce': 3972761964}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6804', 'nonce': 3972761964}, {'type': 'v1', 'addr': '172.21.15.120:6805', 'nonce': 3972761964}]}, 'public_addr': '172.21.15.120:6801/3972761964', 'cluster_addr': '172.21.15.120:6803/3972761964', 'heartbeat_back_addr': '172.21.15.120:6807/3972761964', 'heartbeat_front_addr': '172.21.15.120:6805/3972761964', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1528f8e0-5e13-4ec7-9d98-0d94b7a505c3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6816', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6817', 'nonce': 2916410859}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6818', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6819', 'nonce': 2916410859}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6822', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6823', 'nonce': 2916410859}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6820', 'nonce': 2916410859}, {'type': 'v1', 'addr': '172.21.15.120:6821', 'nonce': 2916410859}]}, 'public_addr': '172.21.15.120:6817/2916410859', 'cluster_addr': '172.21.15.120:6819/2916410859', 'heartbeat_back_addr': '172.21.15.120:6823/2916410859', 'heartbeat_front_addr': '172.21.15.120:6821/2916410859', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '74942f78-c1fc-4886-abf0-950722c5edc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6824', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6825', 'nonce': 1224278950}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6826', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6827', 'nonce': 1224278950}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6830', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6831', 'nonce': 1224278950}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6828', 'nonce': 1224278950}, {'type': 'v1', 'addr': '172.21.15.120:6829', 'nonce': 1224278950}]}, 'public_addr': '172.21.15.120:6825/1224278950', 'cluster_addr': '172.21.15.120:6827/1224278950', 'heartbeat_back_addr': '172.21.15.120:6831/1224278950', 'heartbeat_front_addr': '172.21.15.120:6829/1224278950', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e71319d4-5318-4441-84a8-7665ae037596', 'up': 1, 'in': 1, 'weight': 0.9353485107421875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6816', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6817', 'nonce': 1874635282}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6818', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6819', 'nonce': 1874635282}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6822', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6823', 'nonce': 1874635282}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6820', 'nonce': 1874635282}, {'type': 'v1', 'addr': '172.21.15.134:6821', 'nonce': 1874635282}]}, 'public_addr': '172.21.15.134:6817/1874635282', 'cluster_addr': '172.21.15.134:6819/1874635282', 'heartbeat_back_addr': '172.21.15.134:6823/1874635282', 'heartbeat_front_addr': '172.21.15.134:6821/1874635282', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '29e193cd-37af-43e5-bd0c-2f85ce0631e8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6808', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6809', 'nonce': 457724087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6810', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6811', 'nonce': 457724087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6814', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6815', 'nonce': 457724087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6812', 'nonce': 457724087}, {'type': 'v1', 'addr': '172.21.15.134:6813', 'nonce': 457724087}]}, 'public_addr': '172.21.15.134:6809/457724087', 'cluster_addr': '172.21.15.134:6811/457724087', 'heartbeat_back_addr': '172.21.15.134:6815/457724087', 'heartbeat_front_addr': '172.21.15.134:6813/457724087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '55523ec2-cf12-44b3-aeea-f1cedabc996a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6824', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6825', 'nonce': 50133702}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6826', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6827', 'nonce': 50133702}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6830', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6831', 'nonce': 50133702}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6828', 'nonce': 50133702}, {'type': 'v1', 'addr': '172.21.15.134:6829', 'nonce': 50133702}]}, 'public_addr': '172.21.15.134:6825/50133702', 'cluster_addr': '172.21.15.134:6827/50133702', 'heartbeat_back_addr': '172.21.15.134:6831/50133702', 'heartbeat_front_addr': '172.21.15.134:6829/50133702', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '097b1149-3e3c-4e5a-890b-d4cc4361d51b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6800', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6801', 'nonce': 251960875}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6802', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6803', 'nonce': 251960875}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6806', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6807', 'nonce': 251960875}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6804', 'nonce': 251960875}, {'type': 'v1', 'addr': '172.21.15.134:6805', 'nonce': 251960875}]}, 'public_addr': '172.21.15.134:6801/251960875', 'cluster_addr': '172.21.15.134:6803/251960875', 'heartbeat_back_addr': '172.21.15.134:6807/251960875', 'heartbeat_front_addr': '172.21.15.134:6805/251960875', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.143056+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.423968+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.640076+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.536160+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.146033+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.426011+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.598992+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T10:26:26.182929+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2c', 'osds': [1, 7]}, {'pgid': '3.34', 'osds': [3, 1]}], 'pg_upmap_items': [{'pgid': '2.1d', 'mappings': [{'from': 6, 'to': 4}]}, {'pgid': '3.7', 'mappings': [{'from': 3, 'to': 6}]}, {'pgid': '3.16', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.1d', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.28', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.2d', 'mappings': [{'from': 7, 'to': 6}]}, {'pgid': '3.33', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.36', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '4.4', 'mappings': [{'from': 3, 'to': 6}, {'from': 1, 'to': 5}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.10', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 2}, {'from': 3, 'to': 0}]}, {'pgid': '4.1e', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'cephfs_data_ec': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '2', 'plugin': 'jerasure', 'technique': 'reed_sol_van', 'w': '8'}, 'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-10T10:28:38.706 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.36 2024-09-10T10:28:38.706 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.36 2024-09-10T10:28:38.707 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:38.702+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:38.804 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:38.802+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:38.806 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:38.806 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:38.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.808+0000 7f4c8a10a640 1 Processor -- start 2024-09-10T10:28:38.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.809+0000 7f4c8a10a640 1 -- start start 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.809+0000 7f4c8a10a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4c84152a80 0x7f4c840cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.809+0000 7f4c8a10a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4c84158a20 0x7f4c84158e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.809+0000 7f4c8a10a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c841593d0 con 0x7f4c840cd330 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.809+0000 7f4c8a10a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4c840c7560 con 0x7f4c84158a20 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.809+0000 7f4c8a10a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4c840c7e00 con 0x7f4c84152a80 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.809+0000 7f4c83fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4c84158a20 0x7f4c84158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.809+0000 7f4c83fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4c84158a20 0x7f4c84158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60948/0 (socket says 172.21.15.120:60948) 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.809+0000 7f4c83fff640 1 -- 172.21.15.120:0/2384938343 learned_addr learned my addr 172.21.15.120:0/2384938343 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.809+0000 7f4c7bfff640 1 --1- 172.21.15.120:0/2384938343 >> v1:172.21.15.120:6789/0 conn(0x7f4c840cd330 0x7f4c840c56a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:42034/0 (socket says 172.21.15.120:42034) 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.810+0000 7f4c837fe640 1 --2- 172.21.15.120:0/2384938343 >> v2:172.21.15.134:3300/0 conn(0x7f4c84152a80 0x7f4c840cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.810+0000 7f4c83fff640 1 -- 172.21.15.120:0/2384938343 >> v2:172.21.15.134:3300/0 conn(0x7f4c84152a80 msgr2=0x7f4c840cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:38.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.810+0000 7f4c83fff640 1 --2- 172.21.15.120:0/2384938343 >> v2:172.21.15.134:3300/0 conn(0x7f4c84152a80 0x7f4c840cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.810+0000 7f4c83fff640 1 -- 172.21.15.120:0/2384938343 >> v1:172.21.15.120:6789/0 conn(0x7f4c840cd330 legacy=0x7f4c840c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.810+0000 7f4c82ffd640 1 -- 172.21.15.120:0/2384938343 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1471519309 0 0) 0x7f4c841593d0 con 0x7f4c840cd330 2024-09-10T10:28:38.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.810+0000 7f4c83fff640 1 -- 172.21.15.120:0/2384938343 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c840cc340 con 0x7f4c84158a20 2024-09-10T10:28:38.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.811+0000 7f4c83fff640 1 --2- 172.21.15.120:0/2384938343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 0x7f4c84158e90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4c840cf290 tx=0x7f4c7c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733c1e5e92ece143 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:38.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.811+0000 7f4c82ffd640 1 -- 172.21.15.120:0/2384938343 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c036070 con 0x7f4c84158a20 2024-09-10T10:28:38.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.811+0000 7f4c82ffd640 1 -- 172.21.15.120:0/2384938343 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c7c031600 con 0x7f4c84158a20 2024-09-10T10:28:38.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.811+0000 7f4c82ffd640 1 -- 172.21.15.120:0/2384938343 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0319f0 con 0x7f4c84158a20 2024-09-10T10:28:38.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.811+0000 7f4c8a10a640 1 -- 172.21.15.120:0/2384938343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 msgr2=0x7f4c84158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.811+0000 7f4c8a10a640 1 --2- 172.21.15.120:0/2384938343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 0x7f4c84158e90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4c840cf290 tx=0x7f4c7c02a7c0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.811+0000 7f4c8a10a640 1 -- 172.21.15.120:0/2384938343 shutdown_connections 2024-09-10T10:28:38.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.811+0000 7f4c8a10a640 1 --2- 172.21.15.120:0/2384938343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 0x7f4c84158e90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.811+0000 7f4c8a10a640 1 --2- 172.21.15.120:0/2384938343 >> v2:172.21.15.134:3300/0 conn(0x7f4c84152a80 0x7f4c840cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.811+0000 7f4c8a10a640 1 -- 172.21.15.120:0/2384938343 >> 172.21.15.120:0/2384938343 conn(0x7f4c8406bc10 msgr2=0x7f4c8406fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:38.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.812+0000 7f4c8a10a640 1 -- 172.21.15.120:0/2384938343 shutdown_connections 2024-09-10T10:28:38.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.812+0000 7f4c8a10a640 1 -- 172.21.15.120:0/2384938343 wait complete. 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.813+0000 7f4c8a10a640 1 Processor -- start 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.813+0000 7f4c8a10a640 1 -- start start 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.813+0000 7f4c8a10a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c840cd330 0x7f4c8406d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.813+0000 7f4c8a10a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c84152a80 0x7f4c8406e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.813+0000 7f4c8a10a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 0x7f4c840c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.813+0000 7f4c8a10a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c840c7a90 con 0x7f4c84158a20 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.813+0000 7f4c8a10a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4c840c77b0 con 0x7f4c84152a80 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.813+0000 7f4c8a10a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4c840cc000 con 0x7f4c840cd330 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c83fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 0x7f4c840c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c83fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 0x7f4c840c3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60962/0 (socket says 172.21.15.120:60962) 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c83fff640 1 -- 172.21.15.120:0/1693433478 learned_addr learned my addr 172.21.15.120:0/1693433478 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c7bfff640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c84152a80 0x7f4c8406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c837fe640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c840cd330 0x7f4c8406d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c83fff640 1 -- 172.21.15.120:0/1693433478 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c840cd330 msgr2=0x7f4c8406d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c83fff640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c840cd330 0x7f4c8406d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c83fff640 1 -- 172.21.15.120:0/1693433478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c84152a80 msgr2=0x7f4c8406e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c83fff640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c84152a80 0x7f4c8406e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c83fff640 1 -- 172.21.15.120:0/1693433478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c841538a0 con 0x7f4c84158a20 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c83fff640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 0x7f4c840c3cf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c02adb0 tx=0x7f4c7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c80ff9640 1 -- 172.21.15.120:0/1693433478 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c036050 con 0x7f4c84158a20 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c80ff9640 1 -- 172.21.15.120:0/1693433478 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c7c042070 con 0x7f4c84158a20 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c8a10a640 1 -- 172.21.15.120:0/1693433478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8406e830 con 0x7f4c84158a20 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c80ff9640 1 -- 172.21.15.120:0/1693433478 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c009510 con 0x7f4c84158a20 2024-09-10T10:28:38.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.814+0000 7f4c8a10a640 1 -- 172.21.15.120:0/1693433478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8406cec0 con 0x7f4c84158a20 2024-09-10T10:28:38.820 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:38.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.815+0000 7f4c8a10a640 1 -- 172.21.15.120:0/1693433478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c84152e60 con 0x7f4c84158a20 2024-09-10T10:28:38.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.816+0000 7f4c80ff9640 1 -- 172.21.15.120:0/1693433478 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c7c004470 con 0x7f4c84158a20 2024-09-10T10:28:38.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.816+0000 7f4c80ff9640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4c6406be60 0x7f4c6406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:38.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.817+0000 7f4c80ff9640 1 -- 172.21.15.120:0/1693433478 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7015+0+0 (secure 0 0 0) 0x7f4c7c0abf40 con 0x7f4c84158a20 2024-09-10T10:28:38.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.817+0000 7f4c837fe640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4c6406be60 0x7f4c6406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:38.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.817+0000 7f4c837fe640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4c6406be60 0x7f4c6406e320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009aa0 tx=0x7f4c74009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:38.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:38.821+0000 7f4c80ff9640 1 -- 172.21.15.120:0/1693433478 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c841538a0 con 0x7f4c84158a20 2024-09-10T10:28:38.905 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:38.903+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:38.962 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:38.963 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:38.971 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:39.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.005+0000 7f4c8a10a640 1 -- 172.21.15.120:0/1693433478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd rm-pg-upmap-items", "pgid": "3.36"} v 0) -- 0x7f4c8406fe00 con 0x7f4c84158a20 2024-09-10T10:28:39.006 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:39.005+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:39.107 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:39.106+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:39.115 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:39.115 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:39.124 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:39.209 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:39.207+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:39.267 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:39.267 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:39.277 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:39.309 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:39.308+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:39.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.331+0000 7f4c80ff9640 1 -- 172.21.15.120:0/1693433478 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm-pg-upmap-items", "pgid": "3.36"}]=0 clear 3.36 pg_upmap_items mapping v58) ==== 118+0+0 (secure 0 0 0) 0x7f4c7c079f30 con 0x7f4c84158a20 2024-09-10T10:28:39.333 INFO:teuthology.orchestra.run.smithi120.stderr:clear 3.36 pg_upmap_items mapping 2024-09-10T10:28:39.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.334+0000 7f4c8a10a640 1 -- 172.21.15.120:0/1693433478 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4c6406be60 msgr2=0x7f4c6406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:39.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.334+0000 7f4c8a10a640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4c6406be60 0x7f4c6406e320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009aa0 tx=0x7f4c74009190 comp rx=0 tx=0).stop 2024-09-10T10:28:39.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.334+0000 7f4c8a10a640 1 -- 172.21.15.120:0/1693433478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 msgr2=0x7f4c840c3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:39.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.334+0000 7f4c8a10a640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 0x7f4c840c3cf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c02adb0 tx=0x7f4c7c004a40 comp rx=0 tx=0).stop 2024-09-10T10:28:39.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.335+0000 7f4c8a10a640 1 -- 172.21.15.120:0/1693433478 shutdown_connections 2024-09-10T10:28:39.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.335+0000 7f4c8a10a640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4c6406be60 0x7f4c6406e320 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:39.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.335+0000 7f4c8a10a640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c84158a20 0x7f4c840c3cf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:39.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.335+0000 7f4c8a10a640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c84152a80 0x7f4c8406e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:39.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.335+0000 7f4c8a10a640 1 --2- 172.21.15.120:0/1693433478 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c840cd330 0x7f4c8406d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:39.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.335+0000 7f4c8a10a640 1 -- 172.21.15.120:0/1693433478 >> 172.21.15.120:0/1693433478 conn(0x7f4c8406bc10 msgr2=0x7f4c840717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:39.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.335+0000 7f4c8a10a640 1 -- 172.21.15.120:0/1693433478 shutdown_connections 2024-09-10T10:28:39.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.335+0000 7f4c8a10a640 1 -- 172.21.15.120:0/1693433478 wait complete. 2024-09-10T10:28:39.348 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-10T10:28:39.410 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:39.408+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:39.422 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:39.422 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:39.431 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:39.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.485+0000 7f0408d23640 1 Processor -- start 2024-09-10T10:28:39.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.486+0000 7f0408d23640 1 -- start start 2024-09-10T10:28:39.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.486+0000 7f0408d23640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f04040ce160 0x7f040415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:39.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.486+0000 7f0408d23640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f040415ed00 0x7f040406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:39.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.486+0000 7f0408d23640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0404071ed0 con 0x7f0404071af0 2024-09-10T10:28:39.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.486+0000 7f0408d23640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f04040c8b10 con 0x7f040415ed00 2024-09-10T10:28:39.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.486+0000 7f0408d23640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f04040c9320 con 0x7f04040ce160 2024-09-10T10:28:39.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.487+0000 7f0402d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f040415ed00 0x7f040406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:39.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.487+0000 7f0402d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f040415ed00 0x7f040406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60964/0 (socket says 172.21.15.120:60964) 2024-09-10T10:28:39.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.487+0000 7f0402d9d640 1 -- 172.21.15.120:0/1581891935 learned_addr learned my addr 172.21.15.120:0/1581891935 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:39.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.487+0000 7f0401d9b640 1 --2- 172.21.15.120:0/1581891935 >> v2:172.21.15.134:3300/0 conn(0x7f04040ce160 0x7f040415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:39.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.487+0000 7f0402d9d640 1 -- 172.21.15.120:0/1581891935 >> v2:172.21.15.134:3300/0 conn(0x7f04040ce160 msgr2=0x7f040415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:39.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.487+0000 7f0402d9d640 1 --2- 172.21.15.120:0/1581891935 >> v2:172.21.15.134:3300/0 conn(0x7f04040ce160 0x7f040415ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:39.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.487+0000 7f0402d9d640 1 -- 172.21.15.120:0/1581891935 >> v1:172.21.15.134:6789/0 conn(0x7f0404071af0 legacy=0x7f04040cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:39.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.487+0000 7f0402d9d640 1 -- 172.21.15.120:0/1581891935 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04040c8d60 con 0x7f040415ed00 2024-09-10T10:28:39.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f0402d9d640 1 --2- 172.21.15.120:0/1581891935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040415ed00 0x7f040406cd20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f04040704b0 tx=0x7f03f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b515330731405d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:39.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f040159a640 1 -- 172.21.15.120:0/1581891935 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4037070 con 0x7f040415ed00 2024-09-10T10:28:39.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f040159a640 1 -- 172.21.15.120:0/1581891935 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4032440 con 0x7f040415ed00 2024-09-10T10:28:39.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f040159a640 1 -- 172.21.15.120:0/1581891935 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4032810 con 0x7f040415ed00 2024-09-10T10:28:39.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f0408d23640 1 -- 172.21.15.120:0/1581891935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040415ed00 msgr2=0x7f040406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:39.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f0408d23640 1 --2- 172.21.15.120:0/1581891935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040415ed00 0x7f040406cd20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f04040704b0 tx=0x7f03f402a270 comp rx=0 tx=0).stop 2024-09-10T10:28:39.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f0408d23640 1 -- 172.21.15.120:0/1581891935 shutdown_connections 2024-09-10T10:28:39.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f0408d23640 1 --2- 172.21.15.120:0/1581891935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040415ed00 0x7f040406cd20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:39.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f0408d23640 1 --2- 172.21.15.120:0/1581891935 >> v2:172.21.15.134:3300/0 conn(0x7f04040ce160 0x7f040415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:39.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f0408d23640 1 -- 172.21.15.120:0/1581891935 >> 172.21.15.120:0/1581891935 conn(0x7f040406bc10 msgr2=0x7f040406fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:39.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f0408d23640 1 -- 172.21.15.120:0/1581891935 shutdown_connections 2024-09-10T10:28:39.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.488+0000 7f0408d23640 1 -- 172.21.15.120:0/1581891935 wait complete. 2024-09-10T10:28:39.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.490+0000 7f0408d23640 1 Processor -- start 2024-09-10T10:28:39.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.490+0000 7f0408d23640 1 -- start start 2024-09-10T10:28:39.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.490+0000 7f0408d23640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0404071af0 0x7f04040caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:39.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.490+0000 7f0408d23640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04040ce160 0x7f04041556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:39.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.490+0000 7f0408d23640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f040415ed00 0x7f0404154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:39.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.490+0000 7f0408d23640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f040415e500 con 0x7f04040ce160 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.490+0000 7f0408d23640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f040415e220 con 0x7f0404071af0 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.490+0000 7f0408d23640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f040415e7e0 con 0x7f040415ed00 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0401d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04040ce160 0x7f04041556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0401d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04040ce160 0x7f04041556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60976/0 (socket says 172.21.15.120:60976) 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0401d9b640 1 -- 172.21.15.120:0/4201731277 learned_addr learned my addr 172.21.15.120:0/4201731277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0402d9d640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f040415ed00 0x7f0404154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f040259c640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0404071af0 0x7f04040caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0401d9b640 1 -- 172.21.15.120:0/4201731277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f040415ed00 msgr2=0x7f0404154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0401d9b640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f040415ed00 0x7f0404154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0401d9b640 1 -- 172.21.15.120:0/4201731277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0404071af0 msgr2=0x7f04040caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0401d9b640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0404071af0 0x7f04040caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:39.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0401d9b640 1 -- 172.21.15.120:0/4201731277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04041f0910 con 0x7f04040ce160 2024-09-10T10:28:39.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0401d9b640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04040ce160 0x7f04041556e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009d10 tx=0x7f03f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:39.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f03f2ffd640 1 -- 172.21.15.120:0/4201731277 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8018020 con 0x7f04040ce160 2024-09-10T10:28:39.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f03f2ffd640 1 -- 172.21.15.120:0/4201731277 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f8014480 con 0x7f04040ce160 2024-09-10T10:28:39.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0408d23640 1 -- 172.21.15.120:0/4201731277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04041f4650 con 0x7f04040ce160 2024-09-10T10:28:39.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f03f2ffd640 1 -- 172.21.15.120:0/4201731277 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8014830 con 0x7f04040ce160 2024-09-10T10:28:39.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.491+0000 7f0408d23640 1 -- 172.21.15.120:0/4201731277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04041f0bb0 con 0x7f04040ce160 2024-09-10T10:28:39.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.492+0000 7f0408d23640 1 -- 172.21.15.120:0/4201731277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04040ce8a0 con 0x7f04040ce160 2024-09-10T10:28:39.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.493+0000 7f03f2ffd640 1 -- 172.21.15.120:0/4201731277 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03f8005d10 con 0x7f04040ce160 2024-09-10T10:28:39.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.493+0000 7f03f2ffd640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03e406be30 0x7f03e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:39.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.493+0000 7f03f2ffd640 1 -- 172.21.15.120:0/4201731277 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6986+0+0 (secure 0 0 0) 0x7f03f808bd20 con 0x7f04040ce160 2024-09-10T10:28:39.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.493+0000 7f040259c640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03e406be30 0x7f03e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:39.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.493+0000 7f040259c640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03e406be30 0x7f03e406e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0016e0 tx=0x7f03fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:39.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.497+0000 7f03f2ffd640 1 -- 172.21.15.120:0/4201731277 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f03f8022070 con 0x7f04040ce160 2024-09-10T10:28:39.515 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:39.513+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:39.574 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:39.574 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:39.584 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:39.611 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:39.610+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:39.691+0000 7f0408d23640 1 -- 172.21.15.120:0/4201731277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f04040ca820 con 0x7f04040ce160 2024-09-10T10:28:39.712 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:39.711+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:39.728 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:39.728 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:39.737 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:39.814 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:39.813+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:39.840 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:39.840 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:39.849 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:39.915 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:39.914+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:39.951 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:39.951 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:39.960 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:40.026 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:28:40.044 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:40.043+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:40.103 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:40.104 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:40.112 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:40.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.129+0000 7f527e586640 1 Processor -- start 2024-09-10T10:28:40.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.129+0000 7f527e586640 1 -- start start 2024-09-10T10:28:40.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f527e586640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5278154c10 0x7f527815b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f527e586640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f527815e0b0 con 0x7f52780cca70 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f527e586640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f527815e390 con 0x7f52780cafa0 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f527e586640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f527815de60 con 0x7f5278154c10 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f5277fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5278154c10 0x7f527815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f5277fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5278154c10 0x7f527815b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60986/0 (socket says 172.21.15.120:60986) 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f5277fff640 1 -- 172.21.15.120:0/1150873794 learned_addr learned my addr 172.21.15.120:0/1150873794 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f52777fe640 1 --1- 172.21.15.120:0/1150873794 >> v1:172.21.15.120:6789/0 conn(0x7f52780cca70 0x7f5278073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:42050/0 (socket says 172.21.15.120:42050) 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f5277fff640 1 -- 172.21.15.120:0/1150873794 >> v1:172.21.15.134:6789/0 conn(0x7f52780cafa0 legacy=0x7f52780cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f5277fff640 1 -- 172.21.15.120:0/1150873794 >> v1:172.21.15.120:6789/0 conn(0x7f52780cca70 legacy=0x7f5278073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f5277fff640 1 -- 172.21.15.120:0/1150873794 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527815e670 con 0x7f5278154c10 2024-09-10T10:28:40.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.130+0000 7f5276ffd640 1 -- 172.21.15.120:0/1150873794 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4260397619 0 0) 0x7f527815e0b0 con 0x7f52780cca70 2024-09-10T10:28:40.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.131+0000 7f5277fff640 1 --2- 172.21.15.120:0/1150873794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5278154c10 0x7f527815b370 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5268009870 tx=0x7f526802a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa45fcd928a2363a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.131+0000 7f5276ffd640 1 -- 172.21.15.120:0/1150873794 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268037070 con 0x7f5278154c10 2024-09-10T10:28:40.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.131+0000 7f5276ffd640 1 -- 172.21.15.120:0/1150873794 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5268032640 con 0x7f5278154c10 2024-09-10T10:28:40.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.131+0000 7f5276ffd640 1 -- 172.21.15.120:0/1150873794 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268032a10 con 0x7f5278154c10 2024-09-10T10:28:40.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.132+0000 7f527e586640 1 -- 172.21.15.120:0/1150873794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5278154c10 msgr2=0x7f527815b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.132+0000 7f527e586640 1 --2- 172.21.15.120:0/1150873794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5278154c10 0x7f527815b370 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5268009870 tx=0x7f526802a3c0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.132+0000 7f527e586640 1 -- 172.21.15.120:0/1150873794 shutdown_connections 2024-09-10T10:28:40.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.132+0000 7f527e586640 1 --2- 172.21.15.120:0/1150873794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5278154c10 0x7f527815b370 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.132+0000 7f527e586640 1 -- 172.21.15.120:0/1150873794 >> 172.21.15.120:0/1150873794 conn(0x7f527806ce80 msgr2=0x7f527806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:40.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.132+0000 7f527e586640 1 -- 172.21.15.120:0/1150873794 shutdown_connections 2024-09-10T10:28:40.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.132+0000 7f527e586640 1 -- 172.21.15.120:0/1150873794 wait complete. 2024-09-10T10:28:40.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.133+0000 7f527e586640 1 Processor -- start 2024-09-10T10:28:40.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.133+0000 7f527e586640 1 -- start start 2024-09-10T10:28:40.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f527e586640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52780cafa0 0x7f5278156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f527e586640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52780cca70 0x7f52780c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f527e586640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5278154c10 0x7f52780c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f527e586640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5278154ff0 con 0x7f5278154c10 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f527e586640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f527815ec30 con 0x7f52780cafa0 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f527e586640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5278073a50 con 0x7f52780cca70 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f527cb23640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5278154c10 0x7f52780c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f5277fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52780cafa0 0x7f5278156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f527cb23640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5278154c10 0x7f52780c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60992/0 (socket says 172.21.15.120:60992) 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f527cb23640 1 -- 172.21.15.120:0/2979758906 learned_addr learned my addr 172.21.15.120:0/2979758906 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f52777fe640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52780cca70 0x7f52780c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.134+0000 7f527cb23640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52780cca70 msgr2=0x7f52780c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f527cb23640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52780cca70 0x7f52780c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f527cb23640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52780cafa0 msgr2=0x7f5278156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f527cb23640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52780cafa0 0x7f5278156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f527cb23640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527815a860 con 0x7f5278154c10 2024-09-10T10:28:40.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f5277fff640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52780cafa0 0x7f5278156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:40.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f52777fe640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52780cca70 0x7f52780c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:40.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f527cb23640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5278154c10 0x7f52780c9000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f527000c7d0 tx=0x7f527000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270003b70 con 0x7f5278154c10 2024-09-10T10:28:40.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5270003d30 con 0x7f5278154c10 2024-09-10T10:28:40.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5278155400 con 0x7f5278154c10 2024-09-10T10:28:40.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527000aa90 con 0x7f5278154c10 2024-09-10T10:28:40.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.135+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52780ca320 con 0x7f5278154c10 2024-09-10T10:28:40.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.136+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f527815b370 con 0x7f5278154c10 2024-09-10T10:28:40.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.137+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f527000ac30 con 0x7f5278154c10 2024-09-10T10:28:40.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.137+0000 7f5274ff9640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f526406be30 0x7f526406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.137+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6986+0+0 (secure 0 0 0) 0x7f527008b1a0 con 0x7f5278154c10 2024-09-10T10:28:40.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.137+0000 7f5277fff640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f526406be30 0x7f526406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.137+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v58) ==== 54+0+16 (secure 0 0 0) 0x7f527008d740 con 0x7f5278154c10 2024-09-10T10:28:40.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.138+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f52780ca520 0x7f52781f0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.138+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52780ce440 con 0x7f52780ca520 2024-09-10T10:28:40.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.138+0000 7f5277fff640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f526406be30 0x7f526406e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5268034000 tx=0x7f526802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.138+0000 7f52777fe640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f52780ca520 0x7f52781f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.138+0000 7f52777fe640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f52780ca520 0x7f52781f0390 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.139+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52780ce440 con 0x7f52780ca520 2024-09-10T10:28:40.154 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:40.153+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:40.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.153+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f52780c8360 con 0x7f52780ca520 2024-09-10T10:28:40.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.155+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f52780c8360 con 0x7f52780ca520 2024-09-10T10:28:40.156 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:40.157 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:40.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.155+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f52781f3670 0x7f52781f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.155+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5278075880 con 0x7f52781f3670 2024-09-10T10:28:40.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.155+0000 7f5277fff640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f52781f3670 0x7f52781f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.155+0000 7f5277fff640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f52781f3670 0x7f52781f3a50 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.156+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5278075880 con 0x7f52781f3670 2024-09-10T10:28:40.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.170+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f52781f3f90 con 0x7f52781f3670 2024-09-10T10:28:40.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.172+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f52781f3f90 con 0x7f52781f3670 2024-09-10T10:28:40.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:40.174 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:40.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.172+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f527814e0b0 0x7f527814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.172+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f527814e9d0 con 0x7f527814e0b0 2024-09-10T10:28:40.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.173+0000 7f52777fe640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f527814e0b0 0x7f527814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.173+0000 7f52777fe640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f527814e0b0 0x7f527814e490 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.173+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f527814e9d0 con 0x7f527814e0b0 2024-09-10T10:28:40.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.187+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5278062630 con 0x7f527814e0b0 2024-09-10T10:28:40.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.189+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5278062630 con 0x7f527814e0b0 2024-09-10T10:28:40.190 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:40.191 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:40.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.189+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f5278063110 0x7f527815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.189+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f5278063570 con 0x7f5278063110 2024-09-10T10:28:40.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.189+0000 7f527cb23640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f5278063110 0x7f527815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.190+0000 7f527cb23640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f5278063110 0x7f527815ba50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.190+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5278063570 con 0x7f5278063110 2024-09-10T10:28:40.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.204+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f527815c260 con 0x7f5278063110 2024-09-10T10:28:40.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.205+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f527815c260 con 0x7f5278063110 2024-09-10T10:28:40.207 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:40.207 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:40.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.206+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f527802c640 0x7f527802ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.206+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f527815c8e0 con 0x7f527802c640 2024-09-10T10:28:40.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.206+0000 7f5277fff640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f527802c640 0x7f527802ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.206+0000 7f5277fff640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f527802c640 0x7f527802ca80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.207+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f527815c8e0 con 0x7f527802c640 2024-09-10T10:28:40.218 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:40.217+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:40.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.220+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f527815c8e0 con 0x7f527802c640 2024-09-10T10:28:40.222 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:40.222 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:40.222 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:40.222 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:40.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.222+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f527815c8e0 con 0x7f527802c640 2024-09-10T10:28:40.223 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:40.224 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:40.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.222+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f527802d4d0 0x7f5278206c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.222+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f527802d9c0 con 0x7f527802d4d0 2024-09-10T10:28:40.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.223+0000 7f52777fe640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f527802d4d0 0x7f5278206c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.223+0000 7f52777fe640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f527802d4d0 0x7f5278206c80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.224+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f527802d9c0 con 0x7f527802d4d0 2024-09-10T10:28:40.231 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:40.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.237+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f527802d910 con 0x7f527802d4d0 2024-09-10T10:28:40.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.239+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f527802d910 con 0x7f527802d4d0 2024-09-10T10:28:40.241 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:40.242 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:40.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.239+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f52782076d0 0x7f5278209ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.239+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f527820ca60 con 0x7f52782076d0 2024-09-10T10:28:40.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.240+0000 7f527cb23640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f52782076d0 0x7f5278209ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.240+0000 7f527cb23640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f52782076d0 0x7f5278209ac0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.241+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f527820ca60 con 0x7f52782076d0 2024-09-10T10:28:40.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.255+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5278000950 con 0x7f52782076d0 2024-09-10T10:28:40.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.256+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5278000950 con 0x7f52782076d0 2024-09-10T10:28:40.258 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:40.258 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:40.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.257+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f527820bcc0 0x7f527820c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:40.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.257+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f527820fe80 con 0x7f527820bcc0 2024-09-10T10:28:40.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.257+0000 7f5277fff640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f527820bcc0 0x7f527820c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:40.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.257+0000 7f5277fff640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f527820bcc0 0x7f527820c120 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:40.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.258+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f527820fe80 con 0x7f527820bcc0 2024-09-10T10:28:40.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.272+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f527820a070 con 0x7f527820bcc0 2024-09-10T10:28:40.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.273+0000 7f5274ff9640 1 -- 172.21.15.120:0/2979758906 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f527820a070 con 0x7f527820bcc0 2024-09-10T10:28:40.275 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:40.276 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:40.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f52780ca520 msgr2=0x7f52781f0390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f52780ca520 0x7f52781f0390 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f52781f3670 msgr2=0x7f52781f3a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f52781f3670 0x7f52781f3a50 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f527814e0b0 msgr2=0x7f527814e490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f527814e0b0 0x7f527814e490 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f5278063110 msgr2=0x7f527815ba50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f5278063110 0x7f527815ba50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f527802c640 msgr2=0x7f527802ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527cb23640 1 -- 172.21.15.120:0/2979758906 reap_dead start 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f527802c640 0x7f527802ca80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527cb23640 1 -- 172.21.15.120:0/2979758906 reap_dead start 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f527802d4d0 msgr2=0x7f5278206c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f527802d4d0 0x7f5278206c80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f52782076d0 msgr2=0x7f5278209ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f52782076d0 0x7f5278209ac0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f527820bcc0 msgr2=0x7f527820c120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f527820bcc0 0x7f527820c120 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f526406be30 msgr2=0x7f526406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f526406be30 0x7f526406e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5268034000 tx=0x7f526802e000 comp rx=0 tx=0).stop 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5278154c10 msgr2=0x7f52780c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 --2- 172.21.15.120:0/2979758906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5278154c10 0x7f52780c9000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f527000c7d0 tx=0x7f527000cc90 comp rx=0 tx=0).stop 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527cb23640 1 -- 172.21.15.120:0/2979758906 reap_dead start 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 shutdown_connections 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 >> 172.21.15.120:0/2979758906 conn(0x7f527806ce80 msgr2=0x7f52780cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 shutdown_connections 2024-09-10T10:28:40.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.274+0000 7f527e586640 1 -- 172.21.15.120:0/2979758906 wait complete. 2024-09-10T10:28:40.328 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:40.327+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:40.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.333+0000 7f03f2ffd640 1 -- 172.21.15.120:0/4201731277 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v59) ==== 97+0+0 (secure 0 0 0) 0x7f03f8010040 con 0x7f04040ce160 2024-09-10T10:28:40.334 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is set 2024-09-10T10:28:40.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 -- 172.21.15.120:0/4201731277 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03e406be30 msgr2=0x7f03e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03e406be30 0x7f03e406e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0016e0 tx=0x7f03fc009040 comp rx=0 tx=0).stop 2024-09-10T10:28:40.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 -- 172.21.15.120:0/4201731277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04040ce160 msgr2=0x7f04041556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:40.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04040ce160 0x7f04041556e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009d10 tx=0x7f03f800c490 comp rx=0 tx=0).stop 2024-09-10T10:28:40.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 -- 172.21.15.120:0/4201731277 shutdown_connections 2024-09-10T10:28:40.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03e406be30 0x7f03e406e2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f040415ed00 0x7f0404154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04040ce160 0x7f04041556e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 --2- 172.21.15.120:0/4201731277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0404071af0 0x7f04040caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:40.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 -- 172.21.15.120:0/4201731277 >> 172.21.15.120:0/4201731277 conn(0x7f040406bc10 msgr2=0x7f04040cc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:40.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 -- 172.21.15.120:0/4201731277 shutdown_connections 2024-09-10T10:28:40.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:40.337+0000 7f0408d23640 1 -- 172.21.15.120:0/4201731277 wait complete. 2024-09-10T10:28:40.347 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:40.347 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:40.348 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:40.348 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:40.357 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:40.420 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:40.419+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:40.507 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:40.507 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:40.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:40.507 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:40.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.507 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:1 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-09-10T10:28:40.507 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 128.236959928, 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028809700000000001, 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:40.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.509 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268115+0000", 2024-09-10T10:28:40.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9999999999999999e-07 2024-09-10T10:28:40.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268119+0000", 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1810000000000003e-06 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268129+0000", 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9790000000000005e-06 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268137+0000", 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4800000000000001e-06 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268198+0000", 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0918000000000002e-05 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.268226+0000", 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7267000000000002e-05 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270943+0000", 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027177529999999998 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.270996+0000", 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2191999999999998e-05 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:119 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'81, mlcod=16'78)", 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 128.01110897500001, 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029329880000000001, 2024-09-10T10:28:40.512 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493965+0000", 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.493966+0000", 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1699999999999998e-07 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494040+0000", 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3883999999999999e-05 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494048+0000", 2024-09-10T10:28:40.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4880000000000007e-06 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494057+0000", 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1310000000000006e-06 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494095+0000", 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8470000000000003e-05 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494108+0000", 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2588999999999999e-05 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496869+0000", 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027610400000000002 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.496898+0000", 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9869000000000002e-05 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:120 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'82, mlcod=16'78)", 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 128.010725339, 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005336365, 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494349+0000", 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494350+0000", 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:28:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494416+0000", 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6413000000000003e-05 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494424+0000", 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.576e-06 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494430+0000", 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6579999999999996e-06 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494466+0000", 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5648999999999997e-05 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494478+0000", 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.234e-05 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499654+0000", 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051755209999999998 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499685+0000", 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.167e-05 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(client.4206.0:121 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'83, mlcod=16'78)", 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 128.01033052899999, 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049664970000000003, 2024-09-10T10:28:40.521 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494744+0000", 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3099999999999999e-07 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494819+0000", 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4690000000000005e-05 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494825+0000", 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7080000000000002e-06 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494828+0000", 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1109999999999999e-06 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494869+0000", 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1146000000000002e-05 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.494881+0000", 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2295e-05 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499687+0000", 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048061459999999999 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:32.499710+0000", 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3070000000000001e-05 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.525 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 113.72497432599999, 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054008370000000003, 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780100+0000", 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780104+0000", 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3039999999999998e-06 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0738e-05 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780141+0000", 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6317999999999999e-05 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2199999999999992e-06 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780405+0000", 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025557599999999998 2024-09-10T10:28:40.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780498+0000", 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2860000000000002e-05 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780591+0000", 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3094e-05 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024814469999999999 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785441+0000", 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023678280000000002 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785480+0000", 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9511999999999997e-05 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785501+0000", 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0939999999999999e-05 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 113.72489949200001, 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054780480000000001, 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.531 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780175+0000", 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780176+0000", 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6700000000000003e-07 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1289999999999998e-06 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780188+0000", 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.96e-06 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780194+0000", 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9989999999999997e-06 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780882+0000", 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068832400000000003 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780968+0000", 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5959999999999997e-05 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781098+0000", 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012994100000000001 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783078+0000", 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019803540000000001 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785553+0000", 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002474515 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785608+0000", 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5319999999999999e-05 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785653+0000", 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4879000000000002e-05 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:40.536 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 113.724867949, 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042025200000000004, 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780207+0000", 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780217+0000", 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780224+0000", 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2529999999999998e-06 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780230+0000", 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6110000000000001e-06 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780282+0000", 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2148000000000003e-05 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780394+0000", 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111864 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780553+0000", 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015913799999999999 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783054+0000", 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025008809999999999 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784349+0000", 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012947480000000001 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784381+0000", 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2268999999999999e-05 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784409+0000", 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8007000000000001e-05 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 113.72482559700001, 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054782659999999999, 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780249+0000", 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3799999999999997e-07 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780253+0000", 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4199999999999999e-06 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780260+0000", 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5100000000000001e-06 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780267+0000", 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.443e-06 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781353+0000", 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010859769999999999 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781457+0000", 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104545 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781561+0000", 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010409200000000001 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783532+0000", 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019702280000000001 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785685+0000", 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002153313 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785710+0000", 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5024999999999998e-05 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785727+0000", 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7175000000000001e-05 2024-09-10T10:28:40.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 113.724799506, 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005104407, 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780275+0000", 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:40.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780278+0000", 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.959e-06 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780285+0000", 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8040000000000004e-06 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780292+0000", 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4099999999999996e-06 2024-09-10T10:28:40.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781345+0000", 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010533579999999999 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781456+0000", 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110624 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781570+0000", 2024-09-10T10:28:40.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011417999999999999 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783625+0000", 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020553780000000001 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785303+0000", 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001677827 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785345+0000", 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2364999999999998e-05 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785379+0000", 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4054000000000003e-05 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 113.724775678, 2024-09-10T10:28:40.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0060500800000000002, 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780299+0000", 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780306+0000", 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2740000000000004e-06 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780313+0000", 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9809999999999997e-06 2024-09-10T10:28:40.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1739999999999999e-06 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782543+0000", 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022253419999999999 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010785699999999999 2024-09-10T10:28:40.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.786349+0000", 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036978829999999999 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:17 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.633073637999999, 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058824769999999997, 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872001+0000", 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872002+0000", 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5149999999999999e-06 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872008+0000", 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.66e-06 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872022+0000", 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3932e-05 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872031+0000", 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5259999999999999e-06 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872111+0000", 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9412000000000005e-05 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872231+0000", 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119843 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872340+0000", 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109442 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.875579+0000", 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032383899999999998 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877781+0000", 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022022299999999999 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877835+0000", 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4419000000000002e-05 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877883+0000", 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8108000000000001e-05 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:40.561 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.631984806999995, 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049968149999999999, 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:40.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873090+0000", 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7300000000000005e-07 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.245e-06 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:40.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3898e-05 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873116+0000", 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8050000000000007e-06 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873184+0000", 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7809000000000001e-05 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873257+0000", 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2602999999999996e-05 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873374+0000", 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011744900000000001 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.876282+0000", 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029079459999999998 2024-09-10T10:28:40.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878003+0000", 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001720886 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878054+0000", 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0596999999999997e-05 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878086+0000", 2024-09-10T10:28:40.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2804e-05 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:5 2.1f 2:fa28013d:::615.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.557592435000004, 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029964739999999998, 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.567 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947482+0000", 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947484+0000", 2024-09-10T10:28:40.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.857e-06 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947490+0000", 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9789999999999998e-06 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947504+0000", 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3942e-05 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947512+0000", 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5620000000000004e-06 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947599+0000", 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6525000000000003e-05 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947714+0000", 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114709 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947823+0000", 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109477 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950336+0000", 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025131509999999999 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950386+0000", 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9974e-05 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950440+0000", 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4011000000000002e-05 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950478+0000", 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8287000000000001e-05 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.556152037000004, 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043004360000000004, 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948922+0000", 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948923+0000", 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8999999999999996e-07 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948928+0000", 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9529999999999998e-06 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948940+0000", 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1759e-05 2024-09-10T10:28:40.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948949+0000", 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5149999999999995e-06 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949028+0000", 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8182999999999993e-05 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949137+0000", 2024-09-10T10:28:40.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109065 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949245+0000", 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108047 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950193+0000", 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094856200000000002 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953173+0000", 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029801960000000001 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953200+0000", 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6707999999999999e-05 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953223+0000", 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2758000000000001e-05 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.577 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 86.556096525000001, 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046333320000000004, 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:40.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948979+0000", 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3099999999999997e-07 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6490000000000001e-06 2024-09-10T10:28:40.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948990+0000", 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2560000000000002e-06 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948996+0000", 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9739999999999999e-06 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949496+0000", 2024-09-10T10:28:40.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049934299999999997 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949588+0000", 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1762999999999995e-05 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949729+0000", 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014145299999999999 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950984+0000", 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012548419999999999 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953505+0000", 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025208129999999998 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953568+0000", 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3695000000000006e-05 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953611+0000", 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2913000000000001e-05 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.584044910000003, 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048620219999999997, 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921030+0000", 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4899999999999995e-07 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921034+0000", 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7050000000000001e-06 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921043+0000", 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1840000000000005e-06 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921049+0000", 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4459999999999998e-06 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921575+0000", 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052602300000000003 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921646+0000", 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0475000000000005e-05 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921769+0000", 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123347 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923077+0000", 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013078759999999999 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925794+0000", 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027172310000000001 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925853+0000", 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8739999999999997e-05 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925892+0000", 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8445999999999997e-05 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.584007876000001, 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032737339999999999, 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.588 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921067+0000", 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6899999999999997e-07 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9919999999999999e-06 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921080+0000", 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0063e-05 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921086+0000", 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3109999999999996e-06 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921142+0000", 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6174999999999997e-05 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013480499999999999 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:40.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921405+0000", 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012819200000000001 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923001+0000", 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015962249999999999 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924243+0000", 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012425820000000001 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924300+0000", 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6901999999999998e-05 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.924340+0000", 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9918e-05 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "pg_update_log_missing(2.4 epoch 26/18 rep_tid 19 entries 26'1 (0'0) error 2:218afd09:::203.00000003:head by mds.3.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:40.593 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 80.581869431000001, 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030589089999999998, 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923205+0000", 2024-09-10T10:28:40.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923206+0000", 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7249999999999997e-06 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923229+0000", 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5905000000000001e-05 2024-09-10T10:28:40.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923241+0000", 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1990999999999999e-05 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923317+0000", 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5622000000000002e-05 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926264+0000", 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029470170000000001 2024-09-10T10:28:40.596 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 74.561511431, 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050642550000000001, 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943563+0000", 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943564+0000", 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.13e-07 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943576+0000", 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2187e-05 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943585+0000", 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9279999999999999e-06 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943592+0000", 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0110000000000003e-06 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943681+0000", 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9487999999999994e-05 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943767+0000", 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5866999999999996e-05 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943882+0000", 2024-09-10T10:28:40.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011443299999999999 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.947042+0000", 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031600629999999998 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948540+0000", 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001498451 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948581+0000", 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0839e-05 2024-09-10T10:28:40.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948627+0000", 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6375000000000002e-05 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 74.561443108000006, 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051386089999999997, 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943631+0000", 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943632+0000", 2024-09-10T10:28:40.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1500000000000004e-07 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943643+0000", 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0954e-05 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943652+0000", 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4929999999999998e-06 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943657+0000", 2024-09-10T10:28:40.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6509999999999998e-06 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944230+0000", 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057322999999999999 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944340+0000", 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010986 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.948770+0000", 2024-09-10T10:28:40.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044297060000000003 2024-09-10T10:28:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:40.606 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:40.606 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:40.607 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:40.608 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:40.606+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:40.708 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:40.707+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:40.713 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:40.713 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:40.713 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:40.713 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:40.722 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:40.809 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:40.808+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:40.870 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:40.870 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:40.870 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:40.870 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:40.870 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:40.879 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:40.910 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:40.909+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:41.011 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:41.010+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:41.026 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.244769397, 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049304329999999997, 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.027 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780093+0000", 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780094+0000", 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780107+0000", 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3703000000000001e-05 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780116+0000", 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5210000000000002e-06 2024-09-10T10:28:41.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780125+0000", 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6340000000000007e-06 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001735838 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782000+0000", 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013984100000000001 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782131+0000", 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000131025 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783737+0000", 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016054179999999999 2024-09-10T10:28:41.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784937+0000", 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001200638 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784987+0000", 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9591e-05 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785024+0000", 2024-09-10T10:28:41.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6751000000000001e-05 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.24472994200001, 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052929559999999997, 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.033 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780133+0000", 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.32e-07 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780136+0000", 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9950000000000001e-06 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780144+0000", 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.174e-06 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780150+0000", 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4949999999999999e-06 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781163+0000", 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001013438 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781279+0000", 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011579200000000001 2024-09-10T10:28:41.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781388+0000", 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108834 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783243+0000", 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00185491 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785355+0000", 2024-09-10T10:28:41.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021117990000000001 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785386+0000", 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1612000000000002e-05 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785426+0000", 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9474999999999997e-05 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:8 2.10 e24/18 2:09401a81:::601.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.242538712, 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005352224, 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:41.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782324+0000", 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4400000000000001e-07 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782467+0000", 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142452 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782598+0000", 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013078300000000001 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782604+0000", 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.533e-06 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782651+0000", 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7073999999999997e-05 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782668+0000", 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6861999999999999e-05 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787632+0000", 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049636510000000003 2024-09-10T10:28:41.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787676+0000", 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4425000000000002e-05 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:9 2.10 e24/18 2:09401a81:::601.00000000:head v 24'4, mlcod=0'0)", 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.241901697, 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047438960000000001, 2024-09-10T10:28:41.043 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782961+0000", 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782962+0000", 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4499999999999997e-07 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782967+0000", 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2000000000000002e-06 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782980+0000", 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3685e-05 2024-09-10T10:28:41.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782987+0000", 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4690000000000002e-06 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783112+0000", 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012481300000000001 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783130+0000", 2024-09-10T10:28:41.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7753000000000001e-05 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787678+0000", 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045487419999999997 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787705+0000", 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6588999999999999e-05 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:10 2.10 e24/18 2:0bd6d154:::602.00000000:head v 24'5, mlcod=0'0)", 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.241794763, 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00466629, 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:41.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783068+0000", 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1300000000000001e-07 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783073+0000", 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6970000000000001e-06 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783309+0000", 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000236209 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783319+0000", 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7180000000000008e-06 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783365+0000", 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6179e-05 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783383+0000", 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8382000000000001e-05 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787708+0000", 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043246059999999999 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787734+0000", 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6086000000000001e-05 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:19 2.9 2:92e99a1d:::612.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.152392569, 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054527359999999997, 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.052 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872470+0000", 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872472+0000", 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5409999999999999e-06 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872479+0000", 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9550000000000003e-06 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872494+0000", 2024-09-10T10:28:41.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5452999999999999e-05 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872503+0000", 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1509999999999993e-06 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872571+0000", 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7747e-05 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872704+0000", 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013269900000000001 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872804+0000", 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9934999999999998e-05 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877617+0000", 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048138069999999998 2024-09-10T10:28:41.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877819+0000", 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020164799999999999 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877880+0000", 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1296000000000004e-05 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877923+0000", 2024-09-10T10:28:41.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2503999999999998e-05 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.151371002000005, 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0098302110000000002, 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873493+0000", 2024-09-10T10:28:41.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6799999999999999e-07 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873498+0000", 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1619999999999999e-06 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873511+0000", 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3319000000000001e-05 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873520+0000", 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3409999999999992e-06 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873586+0000", 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6115e-05 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873660+0000", 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3103999999999997e-05 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873775+0000", 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115617 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878285+0000", 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045096049999999999 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883229+0000", 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049443999999999998 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883283+0000", 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4262999999999999e-05 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883322+0000", 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8417000000000002e-05 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:23 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.150881862000006, 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047062800000000002, 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873981+0000", 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873982+0000", 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2900000000000003e-07 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873986+0000", 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3939999999999998e-06 2024-09-10T10:28:41.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.873995+0000", 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0750000000000004e-06 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874002+0000", 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874061+0000", 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8770999999999997e-05 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874178+0000", 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117108 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874290+0000", 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000112503 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878269+0000", 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039784870000000002 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878593+0000", 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032385299999999998 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878644+0000", 2024-09-10T10:28:41.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1790999999999999e-05 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878687+0000", 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2738999999999999e-05 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.150281582999995, 2024-09-10T10:28:41.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0089185449999999999, 2024-09-10T10:28:41.070 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.070 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.070 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.070 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:41.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874582+0000", 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1699999999999997e-07 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874592+0000", 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0604999999999999e-05 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874602+0000", 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0066e-05 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874610+0000", 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1919999999999999e-06 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874696+0000", 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6033999999999997e-05 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874809+0000", 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000113373 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874923+0000", 2024-09-10T10:28:41.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114077 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878289+0000", 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033662380000000001 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883420+0000", 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051305760000000004 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883465+0000", 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5252e-05 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883500+0000", 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4415000000000002e-05 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:41.075 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.076418262000004, 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073932399999999997, 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948444+0000", 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948446+0000", 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3400000000000001e-06 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948451+0000", 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6219999999999997e-06 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948464+0000", 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2955e-05 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948472+0000", 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0730000000000005e-06 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948534+0000", 2024-09-10T10:28:41.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1727e-05 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948649+0000", 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114991 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948757+0000", 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107935 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950179+0000", 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014219440000000001 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055817970000000003 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955794+0000", 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3012000000000002e-05 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955838+0000", 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3844000000000002e-05 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.075879443000005, 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069680200000000001, 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.081 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948984+0000", 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6100000000000001e-07 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948993+0000", 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9809999999999992e-06 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949002+0000", 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8789999999999992e-06 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949008+0000", 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2759999999999997e-06 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949057+0000", 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9159999999999997e-05 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949128+0000", 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1292999999999997e-05 2024-09-10T10:28:41.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949261+0000", 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132564 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950220+0000", 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00095953300000000004 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955890+0000", 2024-09-10T10:28:41.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056692790000000002 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955928+0000", 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8278000000000002e-05 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955951+0000", 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3215999999999999e-05 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.103902933000001, 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050489669999999997, 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:41.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920960+0000", 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5799999999999999e-07 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920964+0000", 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6720000000000002e-06 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920971+0000", 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5140000000000004e-06 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.920977+0000", 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0100000000000001e-06 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921033+0000", 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5899000000000001e-05 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921172+0000", 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000139011 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921288+0000", 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011605200000000001 2024-09-10T10:28:41.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922851+0000", 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001562499 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925915+0000", 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003063727 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925969+0000", 2024-09-10T10:28:41.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4083999999999999e-05 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926009+0000", 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9941000000000003e-05 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:41.092 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.103296076000007, 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045836510000000002, 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921567+0000", 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4499999999999996e-07 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921572+0000", 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3239999999999996e-06 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921581+0000", 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9190000000000005e-06 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921589+0000", 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0649999999999999e-06 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921635+0000", 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6863999999999997e-05 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921705+0000", 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9046000000000003e-05 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.921818+0000", 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011313499999999999 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923033+0000", 2024-09-10T10:28:41.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012153590000000001 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926055+0000", 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030223310000000001 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926104+0000", 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8421999999999998e-05 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926150+0000", 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6440999999999998e-05 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.102857131999997, 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051621749999999997, 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922006+0000", 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3600000000000003e-07 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922011+0000", 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4449999999999998e-06 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922019+0000", 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0930000000000003e-06 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922025+0000", 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5060000000000001e-06 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922076+0000", 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0671000000000003e-05 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922175+0000", 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9611000000000006e-05 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922278+0000", 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102565 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925217+0000", 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029384670000000002 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927074+0000", 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018579499999999999 2024-09-10T10:28:41.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927129+0000", 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4428000000000001e-05 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927168+0000", 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8803000000000001e-05 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.102821461999994, 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059957930000000001, 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922041+0000", 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922042+0000", 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4099999999999999e-07 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922045+0000", 2024-09-10T10:28:41.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4209999999999999e-06 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922052+0000", 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1430000000000001e-06 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922059+0000", 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2139999999999999e-06 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922495+0000", 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043517499999999999 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922563+0000", 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8084999999999996e-05 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922679+0000", 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011609599999999999 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927208+0000", 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045292170000000003 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.927952+0000", 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00074421700000000003 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928001+0000", 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9004e-05 2024-09-10T10:28:41.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928037+0000", 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.578e-05 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.102794703000001, 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045268130000000002, 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.109 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922068+0000", 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9499999999999998e-07 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922078+0000", 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5040000000000008e-06 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922085+0000", 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3710000000000002e-06 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922091+0000", 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0340000000000002e-06 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922144+0000", 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3106999999999998e-05 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922248+0000", 2024-09-10T10:28:41.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010373400000000001 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922399+0000", 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015127200000000001 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.925201+0000", 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028019189999999999 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926514+0000", 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001312279 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926561+0000", 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7790000000000002e-05 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.926595+0000", 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3408000000000003e-05 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.114 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.102159854999996, 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005322116, 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922703+0000", 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999996e-07 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922707+0000", 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.766e-06 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922717+0000", 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7229999999999999e-06 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922724+0000", 2024-09-10T10:28:41.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8410000000000002e-06 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922792+0000", 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8557000000000001e-05 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922912+0000", 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119354 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928025+0000", 2024-09-10T10:28:41.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051131919999999999 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.101683702000003, 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049271260000000004, 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923179+0000", 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923180+0000", 2024-09-10T10:28:41.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923183+0000", 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4000000000000001e-06 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923196+0000", 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3127e-05 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923203+0000", 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.939e-06 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923270+0000", 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7087000000000003e-05 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923359+0000", 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9054999999999994e-05 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928106+0000", 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047469239999999996 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.101223060999999, 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045860270000000003, 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:41.123 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923640+0000", 2024-09-10T10:28:41.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2500000000000005e-07 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923644+0000", 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8240000000000001e-06 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923659+0000", 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5358000000000001e-05 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923666+0000", 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9659999999999999e-06 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923725+0000", 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8674000000000001e-05 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.923850+0000", 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000124797 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928226+0000", 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0043757830000000003 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.083700625999995, 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005230859, 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941162+0000", 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941164+0000", 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.863e-06 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941172+0000", 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2360000000000004e-06 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941190+0000", 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7812000000000002e-05 2024-09-10T10:28:41.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941200+0000", 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7599999999999997e-06 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941350+0000", 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015070400000000001 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941501+0000", 2024-09-10T10:28:41.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015044200000000001 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000167076 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943487+0000", 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018190260000000001 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946297+0000", 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028102840000000001 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946349+0000", 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1649e-05 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946393+0000", 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4007000000000001e-05 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.132 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.133 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.133 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.133 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.133 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:41.134 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:41.135 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:41.133+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:41.235 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:41.234+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:41.282 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:41.282 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:41.282 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:41.282 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:41.291 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:41.336 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:41.335+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:41.420 INFO:tasks.workunit.client.0.smithi120.stderr:Updating files: 95% (12142/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T10:28:41.436 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:41.437 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:41.437 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:41.436+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:41.446 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:41.538 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:41.537+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:41.594 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:41.594 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:41.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.811617179, 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078690670000000004, 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2200000000000004e-07 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780048+0000", 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2427999999999999e-05 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780057+0000", 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1090000000000004e-06 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780063+0000", 2024-09-10T10:28:41.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9780000000000002e-06 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011213480000000001 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781281+0000", 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6728e-05 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781382+0000", 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101455 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784820+0000", 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034376200000000002 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787779+0000", 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002959373 2024-09-10T10:28:41.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787852+0000", 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3135999999999999e-05 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787894+0000", 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1369999999999999e-05 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.81156963799999, 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079289789999999992, 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780073+0000", 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780075+0000", 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5129999999999999e-06 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780082+0000", 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9410000000000001e-06 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780087+0000", 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1880000000000001e-06 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781620+0000", 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001532346 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781698+0000", 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8232999999999994e-05 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781816+0000", 2024-09-10T10:28:41.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011825800000000001 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784824+0000", 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030076349999999998 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787937+0000", 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031134829999999998 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787969+0000", 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1745999999999997e-05 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788001+0000", 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2132000000000003e-05 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:41.606 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.811539423, 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079888619999999994, 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780103+0000", 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780106+0000", 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1379999999999999e-06 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780115+0000", 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4299999999999995e-06 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780122+0000", 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3380000000000001e-06 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782041+0000", 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019193700000000001 2024-09-10T10:28:41.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782149+0000", 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107493 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782246+0000", 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999996e-05 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785205+0000", 2024-09-10T10:28:41.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002958535 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788041+0000", 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028359650000000002 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788069+0000", 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8123000000000002e-05 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788091+0000", 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2506000000000001e-05 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.81151088199999, 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080531070000000003, 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.612 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780134+0000", 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0570000000000001e-06 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780143+0000", 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-06 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780156+0000", 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3179e-05 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023278280000000001 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782555+0000", 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0494000000000006e-05 2024-09-10T10:28:41.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782675+0000", 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120463 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785209+0000", 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002533455 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788127+0000", 2024-09-10T10:28:41.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029181260000000001 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788155+0000", 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8495000000000001e-05 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788184+0000", 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8736999999999999e-05 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.811478365, 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008097524, 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780164+0000", 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.13e-07 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9780000000000001e-06 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780174+0000", 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7510000000000002e-06 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780179+0000", 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0869999999999997e-06 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782890+0000", 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027113509999999999 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782985+0000", 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5152e-05 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783090+0000", 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104947 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785222+0000", 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021320269999999999 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788216+0000", 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029940629999999999 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788242+0000", 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5259999999999999e-05 2024-09-10T10:28:41.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788261+0000", 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9395e-05 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 114.811455994, 2024-09-10T10:28:41.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081619529999999996, 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780189+0000", 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.926e-06 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780197+0000", 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2039999999999997e-06 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780202+0000", 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783326+0000", 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003123855 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783402+0000", 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6548e-05 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783558+0000", 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015539199999999999 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785225+0000", 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001666989 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788290+0000", 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030647890000000001 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788316+0000", 2024-09-10T10:28:41.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6662e-05 2024-09-10T10:28:41.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788348+0000", 2024-09-10T10:28:41.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1417000000000001e-05 2024-09-10T10:28:41.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.629 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.633 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:3 2.10 2:0f65c9b8:::60a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.633 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:41.633 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.719862481999996, 2024-09-10T10:28:41.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066829990000000002, 2024-09-10T10:28:41.633 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.633 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.633 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.633 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871781+0000", 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2129999999999998e-06 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871790+0000", 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3380000000000005e-06 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871805+0000", 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5353999999999999e-05 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871817+0000", 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1399999999999999e-05 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871932+0000", 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114982 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872069+0000", 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013713199999999999 2024-09-10T10:28:41.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872208+0000", 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013976900000000001 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874946+0000", 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027378789999999999 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878355+0000", 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034089139999999999 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878413+0000", 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7540000000000001e-05 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878462+0000", 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9478000000000003e-05 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.638 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.644419350000007, 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067332750000000004, 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947223+0000", 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7299999999999995e-07 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947227+0000", 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.895e-06 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947235+0000", 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947246+0000", 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0738e-05 2024-09-10T10:28:41.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948505+0000", 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001259402 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948578+0000", 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2788999999999997e-05 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948718+0000", 2024-09-10T10:28:41.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013953699999999999 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953391+0000", 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046736670000000003 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953881+0000", 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048956900000000003 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953923+0000", 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2153000000000001e-05 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953956+0000", 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2526e-05 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.644 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.644360562000003, 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066654160000000004, 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.645 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947282+0000", 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947285+0000", 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7689999999999998e-06 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947292+0000", 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6200000000000001e-06 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947304+0000", 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1702000000000001e-05 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947363+0000", 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9426000000000003e-05 2024-09-10T10:28:41.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947485+0000", 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012227599999999999 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947588+0000", 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102641 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950690+0000", 2024-09-10T10:28:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003102264 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953856+0000", 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031656200000000001 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953907+0000", 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1430000000000001e-05 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953947+0000", 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9164e-05 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.644323388999993, 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071148679999999999, 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.650 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947319+0000", 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947322+0000", 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.523e-06 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947331+0000", 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3240000000000003e-06 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947336+0000", 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7189999999999998e-06 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947994+0000", 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065770599999999998 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948080+0000", 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5649000000000006e-05 2024-09-10T10:28:41.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111783 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953965+0000", 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057732089999999996 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954343+0000", 2024-09-10T10:28:41.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037846199999999998 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954395+0000", 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.126e-05 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954433+0000", 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8693999999999999e-05 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.644193193000007, 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067554449999999997, 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:41.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6899999999999996e-07 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947454+0000", 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947463+0000", 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0310000000000004e-06 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947477+0000", 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4328000000000001e-05 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3869999999999998e-05 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947631+0000", 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9874e-05 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947750+0000", 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118483 2024-09-10T10:28:41.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953644+0000", 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058946889999999998 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954111+0000", 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00046712099999999998 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954162+0000", 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000003e-05 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954204+0000", 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1718000000000003e-05 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:41.661 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.643806154999993, 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064551169999999998, 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:41.662 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3e-07 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947841+0000", 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3460000000000004e-06 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.663 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947856+0000", 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947864+0000", 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4939999999999997e-06 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947997+0000", 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013305399999999999 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.664 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948076+0000", 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9146000000000004e-05 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115586 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953707+0000", 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055153470000000003 2024-09-10T10:28:41.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954215+0000", 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050850400000000001 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954258+0000", 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2932000000000002e-05 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954291+0000", 2024-09-10T10:28:41.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2322e-05 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.643766834999994, 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088919089999999999, 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:41.668 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7199999999999999e-07 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947879+0000", 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8380000000000004e-06 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947887+0000", 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5410000000000003e-06 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.669 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947893+0000", 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6930000000000004e-06 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948405+0000", 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051236000000000001 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948502+0000", 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6916999999999994e-05 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.670 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948647+0000", 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000145265 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954479+0000", 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058316970000000003 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956680+0000", 2024-09-10T10:28:41.671 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002201494 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956728+0000", 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8161999999999997e-05 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956767+0000", 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.837e-05 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.672 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.673 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.673 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:41.673 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.643204174000005, 2024-09-10T10:28:41.673 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083279530000000008, 2024-09-10T10:28:41.673 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.673 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.673 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.673 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.674 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8999999999999995e-07 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948443+0000", 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2189999999999998e-06 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948450+0000", 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8879999999999997e-06 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.675 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948457+0000", 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0750000000000003e-06 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948501+0000", 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948594+0000", 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3009000000000003e-05 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948696+0000", 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101461 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953756+0000", 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050607129999999997 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956690+0000", 2024-09-10T10:28:41.677 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029334109999999999 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956737+0000", 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7080999999999999e-05 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956766+0000", 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8778000000000001e-05 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.642664499999995, 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0077788589999999999, 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:41.679 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.680 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948986+0000", 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948998+0000", 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.19e-05 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949007+0000", 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.081e-06 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.681 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949062+0000", 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754999999999997e-05 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949172+0000", 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110128 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949282+0000", 2024-09-10T10:28:41.682 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109289 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955941+0000", 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066594990000000001 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956654+0000", 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071279399999999999 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.683 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956706+0000", 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2027000000000002e-05 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956756+0000", 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0012999999999997e-05 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.684 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.642126406000003, 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073200349999999999, 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.685 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949516+0000", 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999995e-07 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949522+0000", 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3980000000000004e-06 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.686 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949534+0000", 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1562e-05 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949541+0000", 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1380000000000001e-06 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950700+0000", 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011593459999999999 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.687 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950786+0000", 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5556999999999994e-05 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950898+0000", 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111743 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954326+0000", 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034280510000000001 2024-09-10T10:28:41.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956774+0000", 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024477430000000001 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956815+0000", 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1752999999999998e-05 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956835+0000", 2024-09-10T10:28:41.689 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0032999999999999e-05 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.642084104999995, 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073303700000000001, 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.690 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:41.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949571+0000", 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2298e-05 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949579+0000", 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5909999999999996e-06 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949585+0000", 2024-09-10T10:28:41.692 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7370000000000003e-06 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953802+0000", 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042170929999999999 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953915+0000", 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011256699999999999 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.693 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956888+0000", 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029734639999999999 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:24 2.3 e26/18 2:c216c0d1:::mds2_inotable:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 87.641266103999996, 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0063254399999999999, 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.694 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.695 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950380+0000", 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2590000000000002e-06 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950387+0000", 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.696 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950392+0000", 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950438+0000", 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5592e-05 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950457+0000", 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8569000000000001e-05 2024-09-10T10:28:41.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956657+0000", 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062003409999999998 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956701+0000", 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4140000000000001e-05 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:29 2.10 2:0e4da237:::203.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 81.669605301000004, 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064228799999999997, 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922037+0000", 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.700 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922040+0000", 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.694e-06 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922054+0000", 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906e-05 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922065+0000", 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.701 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922125+0000", 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.982e-05 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922233+0000", 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108175 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928459+0000", 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062264570000000003 2024-09-10T10:28:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 75.650341671000007, 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081214100000000008, 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.703 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941301+0000", 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.062e-06 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941306+0000", 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0919999999999997e-06 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941316+0000", 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5640000000000002e-06 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941323+0000", 2024-09-10T10:28:41.705 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941405+0000", 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1650000000000006e-05 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941549+0000", 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014447599999999999 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:41.706 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012929 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943549+0000", 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018704049999999999 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949343+0000", 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057943300000000003 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949378+0000", 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4474999999999998e-05 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949421+0000", 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3955999999999997e-05 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:41.708 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:41.709 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:41.710 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:41.710 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:41.709+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:41.811 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:41.810+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:41.857 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:41.857 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:41.857 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:41.857 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:41.866 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:41.911 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:41.911+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:41.972 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:41.973 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:41.973 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:41.977 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:28:41.980 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:28:41.982 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:42.024 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:42.022+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:42.113 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:42.112+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr: git switch -c 2024-09-10T10:28:42.120 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T10:28:42.121 INFO:tasks.workunit.client.0.smithi120.stderr:Or undo this operation with: 2024-09-10T10:28:42.121 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T10:28:42.121 INFO:tasks.workunit.client.0.smithi120.stderr: git switch - 2024-09-10T10:28:42.121 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T10:28:42.121 INFO:tasks.workunit.client.0.smithi120.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T10:28:42.121 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T10:28:42.121 INFO:tasks.workunit.client.0.smithi120.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:28:42.125 DEBUG:teuthology.orchestra.run.smithi120:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T10:28:42.132 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:42.132 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:42.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:42.132 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:42.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.132 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:42.132 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:42.132 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 115.349936262, 2024-09-10T10:28:42.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054521709999999996, 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:42.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780407+0000", 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.517e-06 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780414+0000", 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3910000000000003e-06 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780429+0000", 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5379999999999998e-05 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780438+0000", 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9230000000000008e-06 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780885+0000", 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044711300000000001 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780979+0000", 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4312000000000001e-05 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785856+0000", 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048765350000000004 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 115.349861535, 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028177609999999998, 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:42.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780479+0000", 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9800000000000003e-07 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780488+0000", 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3349999999999996e-06 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780512+0000", 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3156999999999999e-05 2024-09-10T10:28:42.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780518+0000", 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.764e-06 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780576+0000", 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7813999999999999e-05 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780700+0000", 2024-09-10T10:28:42.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123733 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783296+0000", 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025963599999999998 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:19 2.14 e24/18 2:2f55791f:::606.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 115.348742375, 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003505083, 2024-09-10T10:28:42.141 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9400000000000005e-07 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781606+0000", 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6559999999999992e-06 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781624+0000", 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7575e-05 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781630+0000", 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8630000000000002e-06 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781683+0000", 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475000000000002e-05 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781700+0000", 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7393000000000001e-05 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785058+0000", 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033575660000000002 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785103+0000", 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:1 2.14 e24/18 2:292cf221:::200.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 115.34857386900001, 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033666960000000002, 2024-09-10T10:28:42.145 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781767+0000", 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781772+0000", 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5539999999999997e-06 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781781+0000", 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6149999999999997e-06 2024-09-10T10:28:42.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781787+0000", 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1589999999999998e-06 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2859000000000002e-05 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781876+0000", 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5916000000000001e-05 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785108+0000", 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003232146 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785133+0000", 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5015999999999999e-05 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.149 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:23 2.e e24/18 2:717a0223:::608.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 115.34795933300001, 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027543580000000002, 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1699999999999998e-07 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782387+0000", 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3229999999999997e-06 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782399+0000", 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2466000000000001e-05 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782406+0000", 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9460000000000001e-06 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782468+0000", 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2347999999999993e-05 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6016e-05 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785096+0000", 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026118629999999999 2024-09-10T10:28:42.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785135+0000", 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8779000000000003e-05 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:24 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.252777627, 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062416909999999997, 2024-09-10T10:28:42.154 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877564+0000", 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877576+0000", 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2411e-05 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877592+0000", 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5812e-05 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877600+0000", 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3590000000000002e-06 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877661+0000", 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1749999999999997e-05 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877785+0000", 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123807 2024-09-10T10:28:42.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877936+0000", 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015084300000000001 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.879317+0000", 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001381415 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883686+0000", 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004368701 2024-09-10T10:28:42.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883755+0000", 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9140999999999994e-05 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883804+0000", 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8772e-05 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.251951663, 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056063609999999998, 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878389+0000", 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699999999999999e-07 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878394+0000", 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6120000000000001e-06 2024-09-10T10:28:42.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878407+0000", 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3704e-05 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878415+0000", 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7100000000000007e-06 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878475+0000", 2024-09-10T10:28:42.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878603+0000", 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127525 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883995+0000", 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053920460000000002 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 93.251901297000003, 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057312819999999999, 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5000000000000003e-07 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878442+0000", 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.632e-06 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878450+0000", 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0509999999999997e-06 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878456+0000", 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7200000000000003e-06 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878847+0000", 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039182099999999999 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878949+0000", 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101458 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.884170+0000", 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052210499999999996 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:42.167 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.183235979000003, 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033173069999999998, 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947105+0000", 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4670000000000001e-06 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947111+0000", 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6230000000000002e-06 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947123+0000", 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1883e-05 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947132+0000", 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8740000000000001e-06 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947220+0000", 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7800000000000006e-05 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947375+0000", 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015566399999999999 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947501+0000", 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125609 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948393+0000", 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00089165399999999997 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950323+0000", 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019308769999999999 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950366+0000", 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2688999999999997e-05 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950421+0000", 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5167000000000002e-05 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.172 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.183180426999996, 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0070341989999999997, 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:42.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:42.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947163+0000", 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3440000000000001e-06 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947174+0000", 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0248000000000001e-05 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947180+0000", 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0639999999999999e-06 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947958+0000", 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077789700000000005 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948031+0000", 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3503000000000004e-05 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948151+0000", 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119976 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953625+0000", 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054739360000000004 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954124+0000", 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049858900000000002 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954157+0000", 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3652999999999998e-05 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954194+0000", 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6455000000000002e-05 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.183088584000004, 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032449549999999999, 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:42.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947252+0000", 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000003e-07 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947256+0000", 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3760000000000001e-06 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947267+0000", 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0259e-05 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947273+0000", 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2190000000000001e-06 2024-09-10T10:28:42.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947332+0000", 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9086000000000003e-05 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947452+0000", 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120146 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950496+0000", 2024-09-10T10:28:42.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003044171 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:13 2.1 e26/18 2:81735765:::619.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.182678452999994, 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006029375, 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947662+0000", 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7199999999999997e-07 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6990000000000003e-06 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947681+0000", 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0682999999999999e-05 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947687+0000", 2024-09-10T10:28:42.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7479999999999996e-06 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947742+0000", 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4487000000000002e-05 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947761+0000", 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9497999999999999e-05 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953653+0000", 2024-09-10T10:28:42.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058916530000000002 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953691+0000", 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7734999999999997e-05 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:14 2.1 e26/18 2:81735765:::619.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.181358115999998, 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052525849999999997, 2024-09-10T10:28:42.187 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5700000000000002e-07 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948988+0000", 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7490000000000004e-06 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949003+0000", 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4214e-05 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949010+0000", 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3499999999999999e-06 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949059+0000", 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8758000000000003e-05 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949078+0000", 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9806e-05 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954198+0000", 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051191509999999997 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954234+0000", 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.68e-05 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.191 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:1 2.1 e26/18 2:85bbe569:::202.00000000:head v 26'4, mlcod=0'0)", 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 88.181129752000004, 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005049258, 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949211+0000", 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4999999999999998e-07 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949228+0000", 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7808999999999998e-05 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949240+0000", 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1226000000000001e-05 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949246+0000", 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0689999999999998e-06 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949292+0000", 2024-09-10T10:28:42.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6211999999999997e-05 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949311+0000", 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8873e-05 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954236+0000", 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049253980000000001 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954259+0000", 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3221e-05 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:26 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.188267670000002, 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039596270000000003, 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942075+0000", 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2680000000000001e-06 2024-09-10T10:28:42.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942086+0000", 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1361e-05 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942098+0000", 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2221e-05 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942106+0000", 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0960000000000001e-06 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942209+0000", 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102944 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942346+0000", 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136976 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:42.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942516+0000", 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017015699999999999 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945195+0000", 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026790920000000001 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945919+0000", 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00072374500000000003 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945983+0000", 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3418999999999997e-05 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946032+0000", 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9348000000000003e-05 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:2 2.1 e28/18 2:82971f97:::404.00000000:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:42.201 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.186608514, 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054707489999999996, 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943732+0000", 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1999999999999998e-07 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943740+0000", 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1910000000000003e-06 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943755+0000", 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5014000000000001e-05 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943763+0000", 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9519999999999994e-06 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5569999999999997e-05 2024-09-10T10:28:42.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943862+0000", 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4150000000000003e-05 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949157+0000", 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052949340000000003 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949202+0000", 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5018000000000002e-05 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:6 2.e e28/18 2:75ee5fd8:::629.00000000:head v 28'5, mlcod=24'2)", 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.186546686, 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027501769999999999, 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943794+0000", 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943799+0000", 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5620000000000003e-06 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943807+0000", 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.486e-06 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943813+0000", 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-10T10:28:42.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6419999999999999e-05 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943888+0000", 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9309000000000001e-05 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946498+0000", 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026094669999999999 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946543+0000", 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:17 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'12, mlcod=26'10)", 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:42.210 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.186502924999999, 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027214219999999998, 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:42.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943849+0000", 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2023e-05 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943857+0000", 2024-09-10T10:28:42.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6009999999999997e-06 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943863+0000", 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2899999999999999e-06 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943927+0000", 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3496000000000003e-05 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943949+0000", 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1801e-05 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946490+0000", 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025409579999999998 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946558+0000", 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8830000000000003e-05 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.214 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:18 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'13, mlcod=26'10)", 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.186456723000006, 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027193059999999999, 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943884+0000", 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943889+0000", 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.95e-06 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943896+0000", 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3819999999999998e-06 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943901+0000", 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1719999999999998e-06 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944666+0000", 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00076445300000000001 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944686+0000", 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-05 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018748879999999999 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946603+0000", 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1947999999999999e-05 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:25 2.12 e28/18 2:4dad84a4:::mds4_sessionmap:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.183542759999995, 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002978994, 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:42.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946798+0000", 2024-09-10T10:28:42.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0100000000000001e-06 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946807+0000", 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9700000000000005e-06 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946820+0000", 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2458e-05 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:42.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946856+0000", 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6427999999999997e-05 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946938+0000", 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2106000000000003e-05 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946958+0000", 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0162000000000001e-05 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:42.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949723+0000", 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027645899999999999 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949776+0000", 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3270000000000003e-05 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:42.223 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:42.225 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:42.225 INFO:tasks.workunit.client.0.smithi120.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T10:28:42.225 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T10:28:42.226 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T10:28:42.226 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:42.223+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:42.287 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T10:28:42.325 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:42.325+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:42.342 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:42.342 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:42.342 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:42.342 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:42.351 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-10T10:28:42.352 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:42.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.385+0000 7fb4837cf640 1 Processor -- start 2024-09-10T10:28:42.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.385+0000 7fb4837cf640 1 -- start start 2024-09-10T10:28:42.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.386+0000 7fb4837cf640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb47c155df0 0x7fb47c158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.386+0000 7fb4837cf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c3f50 0x7fb47c06e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.386+0000 7fb4837cf640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb47c06bc10 con 0x7fb47c0ca7c0 2024-09-10T10:28:42.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.386+0000 7fb4837cf640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb47c073690 con 0x7fb47c155df0 2024-09-10T10:28:42.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.386+0000 7fb4837cf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb47c15c6f0 con 0x7fb47c0c3f50 2024-09-10T10:28:42.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.386+0000 7fb480d6a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c3f50 0x7fb47c06e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.386+0000 7fb48156b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb47c155df0 0x7fb47c158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.386+0000 7fb481d6c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb47c0ca7c0 0x7fb47c0ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:46666/0 (socket says 172.21.15.120:46666) 2024-09-10T10:28:42.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.386+0000 7fb481d6c640 1 -- 172.21.15.120:0/872151099 learned_addr learned my addr 172.21.15.120:0/872151099 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.387+0000 7fb46ffff640 1 -- 172.21.15.120:0/872151099 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670069711 0 0) 0x7fb47c06bc10 con 0x7fb47c0ca7c0 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.387+0000 7fb46ffff640 1 -- 172.21.15.120:0/872151099 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb468003520 con 0x7fb47c0ca7c0 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.387+0000 7fb48156b640 1 -- 172.21.15.120:0/872151099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c3f50 msgr2=0x7fb47c06e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.387+0000 7fb48156b640 1 --2- 172.21.15.120:0/872151099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c3f50 0x7fb47c06e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.387+0000 7fb48156b640 1 -- 172.21.15.120:0/872151099 >> v1:172.21.15.134:6789/0 conn(0x7fb47c0ca7c0 legacy=0x7fb47c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.387+0000 7fb48156b640 1 -- 172.21.15.120:0/872151099 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c06bed0 con 0x7fb47c155df0 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.387+0000 7fb480d6a640 1 --2- 172.21.15.120:0/872151099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c3f50 0x7fb47c06e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb48156b640 1 --2- 172.21.15.120:0/872151099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155df0 0x7fb47c158920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb4780097a0 tx=0x7fb47802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf0d5c7fb035603a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb46ffff640 1 -- 172.21.15.120:0/872151099 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478037070 con 0x7fb47c155df0 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb46ffff640 1 -- 172.21.15.120:0/872151099 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb478032440 con 0x7fb47c155df0 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb46ffff640 1 -- 172.21.15.120:0/872151099 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4780327f0 con 0x7fb47c155df0 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb4837cf640 1 -- 172.21.15.120:0/872151099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155df0 msgr2=0x7fb47c158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb4837cf640 1 --2- 172.21.15.120:0/872151099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155df0 0x7fb47c158920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb4780097a0 tx=0x7fb47802a210 comp rx=0 tx=0).stop 2024-09-10T10:28:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb4837cf640 1 -- 172.21.15.120:0/872151099 shutdown_connections 2024-09-10T10:28:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb4837cf640 1 --2- 172.21.15.120:0/872151099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c3f50 0x7fb47c06e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb4837cf640 1 --2- 172.21.15.120:0/872151099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155df0 0x7fb47c158920 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb4837cf640 1 -- 172.21.15.120:0/872151099 >> 172.21.15.120:0/872151099 conn(0x7fb47c0728b0 msgr2=0x7fb47c072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb4837cf640 1 -- 172.21.15.120:0/872151099 shutdown_connections 2024-09-10T10:28:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.388+0000 7fb4837cf640 1 -- 172.21.15.120:0/872151099 wait complete. 2024-09-10T10:28:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb4837cf640 1 Processor -- start 2024-09-10T10:28:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb4837cf640 1 -- start start 2024-09-10T10:28:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb4837cf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb47c0c3f50 0x7fb47c1eef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb4837cf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0ca7c0 0x7fb47c1e5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb4837cf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c155df0 0x7fb47c0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb4837cf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c15c380 con 0x7fb47c0c3f50 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb4837cf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c15c0a0 con 0x7fb47c0ca7c0 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb4837cf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb47c158e60 con 0x7fb47c155df0 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb48156b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb47c0c3f50 0x7fb47c1eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb48156b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb47c0c3f50 0x7fb47c1eef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32774/0 (socket says 172.21.15.120:32774) 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb48156b640 1 -- 172.21.15.120:0/3974968133 learned_addr learned my addr 172.21.15.120:0/3974968133 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb480d6a640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0ca7c0 0x7fb47c1e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.390+0000 7fb481d6c640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c155df0 0x7fb47c0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb48156b640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c155df0 msgr2=0x7fb47c0c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb48156b640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c155df0 0x7fb47c0c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb48156b640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0ca7c0 msgr2=0x7fb47c1e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb48156b640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0ca7c0 0x7fb47c1e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb48156b640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c0cbf00 con 0x7fb47c0c3f50 2024-09-10T10:28:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb480d6a640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0ca7c0 0x7fb47c1e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb48156b640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb47c0c3f50 0x7fb47c1eef90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb478035040 tx=0x7fb4780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478037040 con 0x7fb47c0c3f50 2024-09-10T10:28:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb478042070 con 0x7fb47c0c3f50 2024-09-10T10:28:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c1e1a30 con 0x7fb47c0c3f50 2024-09-10T10:28:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478006e10 con 0x7fb47c0c3f50 2024-09-10T10:28:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.391+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c0cc1a0 con 0x7fb47c0c3f50 2024-09-10T10:28:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.392+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb47c15bce0 con 0x7fb47c0c3f50 2024-09-10T10:28:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.393+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb478032830 con 0x7fb47c0c3f50 2024-09-10T10:28:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.393+0000 7fb46dffb640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb46006be30 0x7fb46006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.393+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6986+0+0 (secure 0 0 0) 0x7fb4780ac540 con 0x7fb47c0c3f50 2024-09-10T10:28:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.393+0000 7fb480d6a640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb46006be30 0x7fb46006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.394+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v59) ==== 54+0+16 (secure 0 0 0) 0x7fb47c15bce0 con 0x7fb47c0c3f50 2024-09-10T10:28:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.394+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fb47c0cdd50 0x7fb47c0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.394+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb47c1561d0 con 0x7fb47c0cdd50 2024-09-10T10:28:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.394+0000 7fb480d6a640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb46006be30 0x7fb46006e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0c0760 tx=0x7fb470009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.394+0000 7fb481d6c640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fb47c0cdd50 0x7fb47c0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.394+0000 7fb481d6c640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fb47c0cdd50 0x7fb47c0ccbd0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.395+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb47c1561d0 con 0x7fb47c0cdd50 2024-09-10T10:28:42.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.409+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb47c0ca5a0 con 0x7fb47c0cdd50 2024-09-10T10:28:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.410+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb47c0ca5a0 con 0x7fb47c0cdd50 2024-09-10T10:28:42.411 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.410+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fb47c153520 0x7fb47c1f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.410+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb47c154220 con 0x7fb47c153520 2024-09-10T10:28:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.411+0000 7fb48156b640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fb47c153520 0x7fb47c1f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.411+0000 7fb48156b640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fb47c153520 0x7fb47c1f1a80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.411+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb47c154220 con 0x7fb47c153520 2024-09-10T10:28:42.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.426+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb47c159f20 con 0x7fb47c153520 2024-09-10T10:28:42.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.427+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb47c159f20 con 0x7fb47c153520 2024-09-10T10:28:42.429 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:42.429 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:42.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.428+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fb47c1f24d0 0x7fb47c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.428+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb47c0c8a10 con 0x7fb47c1f24d0 2024-09-10T10:28:42.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.428+0000 7fb480d6a640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fb47c1f24d0 0x7fb47c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.428+0000 7fb480d6a640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fb47c1f24d0 0x7fb47c062630 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.429+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb47c0c8a10 con 0x7fb47c1f24d0 2024-09-10T10:28:42.439 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:42.438+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:42.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.442+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb47c0c8f20 con 0x7fb47c1f24d0 2024-09-10T10:28:42.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.444+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb47c0c8f20 con 0x7fb47c1f24d0 2024-09-10T10:28:42.445 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.444+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fb47c0c7890 0x7fb47c0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.444+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb47c0c8ea0 con 0x7fb47c0c7890 2024-09-10T10:28:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.444+0000 7fb481d6c640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fb47c0c7890 0x7fb47c0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.444+0000 7fb481d6c640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fb47c0c7890 0x7fb47c0c7cf0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.445+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb47c0c8ea0 con 0x7fb47c0c7890 2024-09-10T10:28:42.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.453+0000 7f98bc677640 1 Processor -- start 2024-09-10T10:28:42.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.453+0000 7f98bc677640 1 -- start start 2024-09-10T10:28:42.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98bc677640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f98b4158170 0x7f98b415e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98bc677640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98b415ec60 0x7f98b406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98bc677640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b406d1c0 con 0x7f98b40c0510 2024-09-10T10:28:42.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98bc677640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f98b415dfb0 con 0x7f98b4158170 2024-09-10T10:28:42.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98bc677640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f98b40c79c0 con 0x7f98b415ec60 2024-09-10T10:28:42.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98ba413640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f98b4158170 0x7f98b415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98ba413640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f98b4158170 0x7f98b415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32784/0 (socket says 172.21.15.120:32784) 2024-09-10T10:28:42.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98ba413640 1 -- 172.21.15.120:0/1217868993 learned_addr learned my addr 172.21.15.120:0/1217868993 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:42.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98b9c12640 1 --2- 172.21.15.120:0/1217868993 >> v2:172.21.15.134:3300/0 conn(0x7f98b415ec60 0x7f98b406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98ba413640 1 -- 172.21.15.120:0/1217868993 >> v2:172.21.15.134:3300/0 conn(0x7f98b415ec60 msgr2=0x7f98b406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:42.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98ba413640 1 --2- 172.21.15.120:0/1217868993 >> v2:172.21.15.134:3300/0 conn(0x7f98b415ec60 0x7f98b406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98ba413640 1 -- 172.21.15.120:0/1217868993 >> v1:172.21.15.134:6789/0 conn(0x7f98b40c0510 legacy=0x7f98b40c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98ba413640 1 -- 172.21.15.120:0/1217868993 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b40c7ca0 con 0x7f98b4158170 2024-09-10T10:28:42.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.454+0000 7f98b9c12640 1 --2- 172.21.15.120:0/1217868993 >> v2:172.21.15.134:3300/0 conn(0x7f98b415ec60 0x7f98b406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:42.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.455+0000 7f98ba413640 1 --2- 172.21.15.120:0/1217868993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b4158170 0x7f98b415e720 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f98b406d730 tx=0x7f98b002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8d14361e218135 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.455+0000 7f98b9411640 1 -- 172.21.15.120:0/1217868993 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0037070 con 0x7f98b4158170 2024-09-10T10:28:42.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.455+0000 7f98b9411640 1 -- 172.21.15.120:0/1217868993 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b002ae50 con 0x7f98b4158170 2024-09-10T10:28:42.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.455+0000 7f98b9411640 1 -- 172.21.15.120:0/1217868993 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0032860 con 0x7f98b4158170 2024-09-10T10:28:42.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.456+0000 7f98bc677640 1 -- 172.21.15.120:0/1217868993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b4158170 msgr2=0x7f98b415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.456+0000 7f98bc677640 1 --2- 172.21.15.120:0/1217868993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b4158170 0x7f98b415e720 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f98b406d730 tx=0x7f98b002a300 comp rx=0 tx=0).stop 2024-09-10T10:28:42.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.456+0000 7f98bc677640 1 -- 172.21.15.120:0/1217868993 shutdown_connections 2024-09-10T10:28:42.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.456+0000 7f98bc677640 1 --2- 172.21.15.120:0/1217868993 >> v2:172.21.15.134:3300/0 conn(0x7f98b415ec60 0x7f98b406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.456+0000 7f98bc677640 1 --2- 172.21.15.120:0/1217868993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b4158170 0x7f98b415e720 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.456+0000 7f98bc677640 1 -- 172.21.15.120:0/1217868993 >> 172.21.15.120:0/1217868993 conn(0x7f98b4067950 msgr2=0x7f98b4067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:42.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.456+0000 7f98bc677640 1 -- 172.21.15.120:0/1217868993 shutdown_connections 2024-09-10T10:28:42.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.456+0000 7f98bc677640 1 -- 172.21.15.120:0/1217868993 wait complete. 2024-09-10T10:28:42.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.457+0000 7f98bc677640 1 Processor -- start 2024-09-10T10:28:42.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.457+0000 7f98bc677640 1 -- start start 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bc677640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98b40c0510 0x7f98b4152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bc677640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98b4158170 0x7f98b40c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bc677640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b415ec60 0x7f98b40c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bc677640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98b40c9ac0 con 0x7f98b415ec60 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bc677640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98b415e200 con 0x7f98b40c0510 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bc677640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98b406bc10 con 0x7f98b4158170 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bac14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b415ec60 0x7f98b40c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bac14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b415ec60 0x7f98b40c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32790/0 (socket says 172.21.15.120:32790) 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98ba413640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98b40c0510 0x7f98b4152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bac14640 1 -- 172.21.15.120:0/576881024 learned_addr learned my addr 172.21.15.120:0/576881024 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:42.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98b9c12640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98b4158170 0x7f98b40c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bac14640 1 -- 172.21.15.120:0/576881024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98b4158170 msgr2=0x7f98b40c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bac14640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98b4158170 0x7f98b40c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bac14640 1 -- 172.21.15.120:0/576881024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98b40c0510 msgr2=0x7f98b4152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bac14640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98b40c0510 0x7f98b4152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.458+0000 7f98bac14640 1 -- 172.21.15.120:0/576881024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b40c0dc0 con 0x7f98b415ec60 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.459+0000 7f98bac14640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b415ec60 0x7f98b40c33f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00c7d0 tx=0x7f98ac00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.459+0000 7f98aaffd640 1 -- 172.21.15.120:0/576881024 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac003b70 con 0x7f98b415ec60 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.459+0000 7f98aaffd640 1 -- 172.21.15.120:0/576881024 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac003d30 con 0x7f98b415ec60 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.459+0000 7f98aaffd640 1 -- 172.21.15.120:0/576881024 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00aa90 con 0x7f98b415ec60 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.459+0000 7f98bc677640 1 -- 172.21.15.120:0/576881024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b40c10a0 con 0x7f98b415ec60 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.459+0000 7f98bc677640 1 -- 172.21.15.120:0/576881024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b40c1660 con 0x7f98b415ec60 2024-09-10T10:28:42.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.459+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb47c0c8ea0 con 0x7fb47c0c7890 2024-09-10T10:28:42.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.460+0000 7f98bc677640 1 -- 172.21.15.120:0/576881024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b41e50f0 con 0x7f98b415ec60 2024-09-10T10:28:42.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.461+0000 7f98aaffd640 1 -- 172.21.15.120:0/576881024 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98ac00ac30 con 0x7f98b415ec60 2024-09-10T10:28:42.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.461+0000 7f98aaffd640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f989806be60 0x7f989806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.461+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb47c0c8ea0 con 0x7fb47c0c7890 2024-09-10T10:28:42.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.461+0000 7f98aaffd640 1 -- 172.21.15.120:0/576881024 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6986+0+0 (secure 0 0 0) 0x7f98ac08b0c0 con 0x7f98b415ec60 2024-09-10T10:28:42.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.461+0000 7f98ba413640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f989806be60 0x7f989806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.464 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:42.464 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:42.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.461+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fb47c0c8740 0x7fb47c02c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.461+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb47c0c8ea0 con 0x7fb47c0c8740 2024-09-10T10:28:42.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.462+0000 7f98ba413640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f989806be60 0x7f989806e320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98b0034000 tx=0x7f98b002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.462+0000 7fb48156b640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fb47c0c8740 0x7fb47c02c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.462+0000 7fb48156b640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fb47c0c8740 0x7fb47c02c640 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.463+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb47c0c8ea0 con 0x7fb47c0c8740 2024-09-10T10:28:42.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.466+0000 7f98aaffd640 1 -- 172.21.15.120:0/576881024 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f98b41e50f0 con 0x7f98b415ec60 2024-09-10T10:28:42.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.487+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb47c0c8ea0 con 0x7fb47c0c8740 2024-09-10T10:28:42.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.489+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb47c0c8ea0 con 0x7fb47c0c8740 2024-09-10T10:28:42.490 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:42.490 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:42.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.489+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fb47c02d090 0x7fb47c02d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.489+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb47c02d9f0 con 0x7fb47c02d090 2024-09-10T10:28:42.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.489+0000 7fb480d6a640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fb47c02d090 0x7fb47c02d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.490+0000 7fb480d6a640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fb47c02d090 0x7fb47c02d4b0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.490+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb47c02d9f0 con 0x7fb47c02d090 2024-09-10T10:28:42.500 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:42.500 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:42.509 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:42.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.509+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb47c02d9f0 con 0x7fb47c02d090 2024-09-10T10:28:42.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.511+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb47c02d9f0 con 0x7fb47c02d090 2024-09-10T10:28:42.512 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:42.512 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:42.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.511+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fb47c205320 0x7fb47c207710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.511+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb47c20a560 con 0x7fb47c205320 2024-09-10T10:28:42.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.511+0000 7fb481d6c640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fb47c205320 0x7fb47c207710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.512+0000 7fb481d6c640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fb47c205320 0x7fb47c207710 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.512+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb47c20a560 con 0x7fb47c205320 2024-09-10T10:28:42.526 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:42.526+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:42.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.527+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb47c207cd0 con 0x7fb47c205320 2024-09-10T10:28:42.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.529+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb47c207cd0 con 0x7fb47c205320 2024-09-10T10:28:42.530 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:42.531 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:42.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.529+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb47c209b50 0x7fb47c209f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:42.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.529+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb47c20d900 con 0x7fb47c209b50 2024-09-10T10:28:42.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.530+0000 7fb48156b640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb47c209b50 0x7fb47c209f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:42.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.530+0000 7fb48156b640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb47c209b50 0x7fb47c209f90 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:42.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.531+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb47c20d900 con 0x7fb47c209b50 2024-09-10T10:28:42.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.546+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb47c20c800 con 0x7fb47c209b50 2024-09-10T10:28:42.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb46dffb640 1 -- 172.21.15.120:0/3974968133 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb47c20c800 con 0x7fb47c209b50 2024-09-10T10:28:42.549 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:42.551 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:42.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fb47c0cdd50 msgr2=0x7fb47c0ccbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fb47c0cdd50 0x7fb47c0ccbd0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fb47c153520 msgr2=0x7fb47c1f1a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7fb47c153520 0x7fb47c1f1a80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fb47c1f24d0 msgr2=0x7fb47c062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7fb47c1f24d0 0x7fb47c062630 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fb47c0c7890 msgr2=0x7fb47c0c7cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7fb47c0c7890 0x7fb47c0c7cf0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fb47c0c8740 msgr2=0x7fb47c02c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fb47c0c8740 0x7fb47c02c640 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fb47c02d090 msgr2=0x7fb47c02d4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fb47c02d090 0x7fb47c02d4b0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fb47c205320 msgr2=0x7fb47c207710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7fb47c205320 0x7fb47c207710 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb481d6c640 1 -- 172.21.15.120:0/3974968133 reap_dead start 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb47c209b50 msgr2=0x7fb47c209f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb47c209b50 0x7fb47c209f90 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb481d6c640 1 -- 172.21.15.120:0/3974968133 reap_dead start 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb481d6c640 1 -- 172.21.15.120:0/3974968133 reap_dead start 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.548+0000 7fb481d6c640 1 -- 172.21.15.120:0/3974968133 reap_dead start 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb46006be30 msgr2=0x7fb46006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb46006be30 0x7fb46006e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0c0760 tx=0x7fb470009110 comp rx=0 tx=0).stop 2024-09-10T10:28:42.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb47c0c3f50 msgr2=0x7fb47c1eef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:42.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb47c0c3f50 0x7fb47c1eef90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb478035040 tx=0x7fb4780046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 shutdown_connections 2024-09-10T10:28:42.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb46006be30 0x7fb46006e2f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7fb47c209b50 0x7fb47c209f90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 --2- 172.21.15.120:0/3974968133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb47c0c3f50 0x7fb47c1eef90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:42.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 >> 172.21.15.120:0/3974968133 conn(0x7fb47c0728b0 msgr2=0x7fb47c157b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:42.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 shutdown_connections 2024-09-10T10:28:42.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.549+0000 7fb4837cf640 1 -- 172.21.15.120:0/3974968133 wait complete. 2024-09-10T10:28:42.627 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:42.626+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:42.656 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:42.656 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:42.660+0000 7f98bc677640 1 -- 172.21.15.120:0/576881024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f98b406ce80 con 0x7f98b415ec60 2024-09-10T10:28:42.665 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:42.749 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:42.748+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:42.754 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T10:28:42.808 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:42.809 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:42.816 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T10:28:42.817 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:42.828 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:42.828+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:42.856 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T10:28:42.857 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T10:28:42.858 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T10:28:42.896 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T10:28:42.899 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:28:42.899 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T10:28:42.929 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:42.929+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:42.955 INFO:tasks.workunit:Running workunits matching fs/misc/trivial_sync.sh on client.0... 2024-09-10T10:28:42.957 INFO:tasks.workunit:Running workunit fs/misc/trivial_sync.sh... 2024-09-10T10:28:42.957 DEBUG:teuthology.orchestra.run.smithi120:workunit test fs/misc/trivial_sync.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh 2024-09-10T10:28:42.961 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:42.961 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:42.970 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:43.040 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:43.039+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:43.114 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:43.114 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:43.123 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:43.131 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:43.130+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:43.226 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:43.227 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:43.231 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:43.230+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:43.235 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:43.332 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:43.331+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:43.338 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:43.338 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:43.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.344+0000 7f98aaffd640 1 -- 172.21.15.120:0/576881024 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v60) ==== 91+0+0 (secure 0 0 0) 0x7f98ac058fb0 con 0x7f98b415ec60 2024-09-10T10:28:43.345 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-10T10:28:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.346+0000 7f98bc677640 1 -- 172.21.15.120:0/576881024 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f989806be60 msgr2=0x7f989806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.346+0000 7f98bc677640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f989806be60 0x7f989806e320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98b0034000 tx=0x7f98b002e000 comp rx=0 tx=0).stop 2024-09-10T10:28:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.346+0000 7f98bc677640 1 -- 172.21.15.120:0/576881024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b415ec60 msgr2=0x7f98b40c33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.346+0000 7f98bc677640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b415ec60 0x7f98b40c33f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00c7d0 tx=0x7f98ac00cc90 comp rx=0 tx=0).stop 2024-09-10T10:28:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.346+0000 7f98bc677640 1 -- 172.21.15.120:0/576881024 shutdown_connections 2024-09-10T10:28:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.346+0000 7f98bc677640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f989806be60 0x7f989806e320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.346+0000 7f98bc677640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98b415ec60 0x7f98b40c33f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.346+0000 7f98bc677640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98b4158170 0x7f98b40c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.346+0000 7f98bc677640 1 --2- 172.21.15.120:0/576881024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98b40c0510 0x7f98b4152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.346+0000 7f98bc677640 1 -- 172.21.15.120:0/576881024 >> 172.21.15.120:0/576881024 conn(0x7f98b4067950 msgr2=0x7f98b406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.347+0000 7f98bc677640 1 -- 172.21.15.120:0/576881024 shutdown_connections 2024-09-10T10:28:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:43.347+0000 7f98bc677640 1 -- 172.21.15.120:0/576881024 wait complete. 2024-09-10T10:28:43.349 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:43.433 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:43.432+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:43.453 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:43.453 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:43.462 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:43.534 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:43.533+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:43.565 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:43.565 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:43.574 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:43.635 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:43.634+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:43.677 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:43.677 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:43.686 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:43.736 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:43.735+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:43.789 INFO:teuthology.orchestra.run.smithi134.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:43.790 INFO:teuthology.orchestra.run.smithi134.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:43.798 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T10:28:43.837 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:43.836+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:43.912 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:43.913 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:43.921 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-10T10:28:43.938 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:43.938+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:44.027 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:44.027 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:44.037 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-10T10:28:44.040 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:44.039+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:44.140 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:44.139+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:44.141 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:44.141 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:44.151 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T10:28:44.243 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:44.240+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:44.255 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:44.255 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:44.264 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-10T10:28:44.343 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:44.342+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:44.347 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T10:28:44.347 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T10:28:44.347 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-10T10:28:44.348 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-10T10:28:44.348 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-10T10:28:44.348 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-10T10:28:44.368 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:44.368 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:44.378 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-10T10:28:44.453 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-10T10:28:44.453 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:44.453 INFO:tasks.ceph.osd.0.smithi120.stderr:2024-09-10T10:28:44.452+0000 7f8d9ffc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36314) UID: 0 2024-09-10T10:28:44.523 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:44.523 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:44.532 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T10:28:44.545 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:44.544+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:44.560 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T10:28:44.637 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-09-10T10:28:44.637 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T10:28:44.646 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-10T10:28:44.647 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:44.645+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:44.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.671+0000 7f27f4913640 1 Processor -- start 2024-09-10T10:28:44.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.671+0000 7f27f4913640 1 -- start start 2024-09-10T10:28:44.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27f4913640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27f0067d20 0x7f27f0072d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27f4913640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27f00c9fc0 con 0x7f27f0154dc0 2024-09-10T10:28:44.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27f4913640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27f00c3aa0 con 0x7f27f00732a0 2024-09-10T10:28:44.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27f4913640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27f00ca230 con 0x7f27f0067d20 2024-09-10T10:28:44.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27ee59c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f27f0154dc0 0x7f27f006cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:42054/0 (socket says 172.21.15.120:42054) 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27ee59c640 1 -- 172.21.15.120:0/671325185 learned_addr learned my addr 172.21.15.120:0/671325185 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27edd9b640 1 --2- 172.21.15.120:0/671325185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27f0067d20 0x7f27f0072d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27ed59a640 1 -- 172.21.15.120:0/671325185 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2954782148 0 0) 0x7f27f00c9fc0 con 0x7f27f0154dc0 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27ed59a640 1 -- 172.21.15.120:0/671325185 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27d8003520 con 0x7f27f0154dc0 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27ed59a640 1 -- 172.21.15.120:0/671325185 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394339729 0 0) 0x7f27f00c3aa0 con 0x7f27f00732a0 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27ed59a640 1 -- 172.21.15.120:0/671325185 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27f00c9fc0 con 0x7f27f00732a0 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27edd9b640 1 -- 172.21.15.120:0/671325185 >> v1:172.21.15.134:6789/0 conn(0x7f27f00732a0 legacy=0x7f27f0165310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27edd9b640 1 -- 172.21.15.120:0/671325185 >> v1:172.21.15.120:6789/0 conn(0x7f27f0154dc0 legacy=0x7f27f006cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.672+0000 7f27edd9b640 1 -- 172.21.15.120:0/671325185 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f01551a0 con 0x7f27f0067d20 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.673+0000 7f27edd9b640 1 --2- 172.21.15.120:0/671325185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27f0067d20 0x7f27f0072d60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27e40097d0 tx=0x7f27e402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fcf49c9c4b021dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.673+0000 7f27ed59a640 1 -- 172.21.15.120:0/671325185 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4037070 con 0x7f27f0067d20 2024-09-10T10:28:44.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.673+0000 7f27ed59a640 1 -- 172.21.15.120:0/671325185 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e4032440 con 0x7f27f0067d20 2024-09-10T10:28:44.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.673+0000 7f27ed59a640 1 -- 172.21.15.120:0/671325185 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4032810 con 0x7f27f0067d20 2024-09-10T10:28:44.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.674+0000 7f27f4913640 1 -- 172.21.15.120:0/671325185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27f0067d20 msgr2=0x7f27f0072d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.674+0000 7f27f4913640 1 --2- 172.21.15.120:0/671325185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27f0067d20 0x7f27f0072d60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27e40097d0 tx=0x7f27e402a180 comp rx=0 tx=0).stop 2024-09-10T10:28:44.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.674+0000 7f27f4913640 1 -- 172.21.15.120:0/671325185 shutdown_connections 2024-09-10T10:28:44.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.674+0000 7f27f4913640 1 --2- 172.21.15.120:0/671325185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27f0067d20 0x7f27f0072d60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.674+0000 7f27f4913640 1 -- 172.21.15.120:0/671325185 >> 172.21.15.120:0/671325185 conn(0x7f27f00c9400 msgr2=0x7f27f00c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:44.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.674+0000 7f27f4913640 1 -- 172.21.15.120:0/671325185 shutdown_connections 2024-09-10T10:28:44.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.674+0000 7f27f4913640 1 -- 172.21.15.120:0/671325185 wait complete. 2024-09-10T10:28:44.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.675+0000 7f27f4913640 1 Processor -- start 2024-09-10T10:28:44.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.675+0000 7f27f4913640 1 -- start start 2024-09-10T10:28:44.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27f4913640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27f0067d20 0x7f27f01e4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27f4913640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27f00732a0 0x7f27f00c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27f4913640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27f0154dc0 0x7f27f00c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27f4913640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0072840 con 0x7f27f0067d20 2024-09-10T10:28:44.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27f4913640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0072560 con 0x7f27f00732a0 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27f4913640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27f0072010 con 0x7f27f0154dc0 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27ee59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27f0067d20 0x7f27f01e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27ee59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27f0067d20 0x7f27f01e4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32800/0 (socket says 172.21.15.120:32800) 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27ee59c640 1 -- 172.21.15.120:0/1952314734 learned_addr learned my addr 172.21.15.120:0/1952314734 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27edd9b640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27f00732a0 0x7f27f00c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27eed9d640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27f0154dc0 0x7f27f00c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27ee59c640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27f0154dc0 msgr2=0x7f27f00c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27ee59c640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27f0154dc0 0x7f27f00c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27ee59c640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27f00732a0 msgr2=0x7f27f00c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.676+0000 7f27ee59c640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27f00732a0 0x7f27f00c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.677+0000 7f27ee59c640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f01f45f0 con 0x7f27f0067d20 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.677+0000 7f27edd9b640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27f00732a0 0x7f27f00c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:44.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.677+0000 7f27ee59c640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27f0067d20 0x7f27f01e4860 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f27e000c530 tx=0x7f27e000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.677+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0017070 con 0x7f27f0067d20 2024-09-10T10:28:44.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.677+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e0005ce0 con 0x7f27f0067d20 2024-09-10T10:28:44.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.677+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f01f48d0 con 0x7f27f0067d20 2024-09-10T10:28:44.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.677+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0007270 con 0x7f27f0067d20 2024-09-10T10:28:44.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.677+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f01e4da0 con 0x7f27f0067d20 2024-09-10T10:28:44.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.678+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f27f015b900 con 0x7f27f0067d20 2024-09-10T10:28:44.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.678+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f27e001a070 con 0x7f27f0067d20 2024-09-10T10:28:44.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.679+0000 7f27deffd640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f27d006be30 0x7f27d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.679+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6986+0+0 (secure 0 0 0) 0x7f27e008a670 con 0x7f27f0067d20 2024-09-10T10:28:44.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.679+0000 7f27edd9b640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f27d006be30 0x7f27d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.679+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v60) ==== 54+0+16 (secure 0 0 0) 0x7f27e0090170 con 0x7f27f0067d20 2024-09-10T10:28:44.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.679+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f27f00c5b80 0x7f27f00c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.679+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27f00c3670 con 0x7f27f00c5b80 2024-09-10T10:28:44.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.679+0000 7f27eed9d640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f27f00c5b80 0x7f27f00c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.679+0000 7f27edd9b640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f27d006be30 0x7f27d006e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f27e4034000 tx=0x7f27e402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.680+0000 7f27eed9d640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f27f00c5b80 0x7f27f00c5f60 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.680+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f00c3670 con 0x7f27f00c5b80 2024-09-10T10:28:44.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.695+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f27f0067b00 con 0x7f27f00c5b80 2024-09-10T10:28:44.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.696+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f27f0067b00 con 0x7f27f00c5b80 2024-09-10T10:28:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0: {} 2024-09-10T10:28:44.698 INFO:teuthology.orchestra.run.smithi120.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T10:28:44.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.696+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f27f01500b0 0x7f27f01504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.696+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f27f00c9980 con 0x7f27f01500b0 2024-09-10T10:28:44.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.697+0000 7f27ee59c640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f27f01500b0 0x7f27f01504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.697+0000 7f27ee59c640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f27f01500b0 0x7f27f01504d0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.697+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f00c9980 con 0x7f27f01500b0 2024-09-10T10:28:44.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.712+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f27f01e6eb0 con 0x7f27f01500b0 2024-09-10T10:28:44.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.713+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f27f01e6eb0 con 0x7f27f01500b0 2024-09-10T10:28:44.715 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1: {} 2024-09-10T10:28:44.715 INFO:teuthology.orchestra.run.smithi120.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T10:28:44.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.714+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f27f014d390 0x7f27f014d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.714+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f27f014dcb0 con 0x7f27f014d390 2024-09-10T10:28:44.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.714+0000 7f27edd9b640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f27f014d390 0x7f27f014d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.714+0000 7f27edd9b640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f27f014d390 0x7f27f014d770 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.715+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f014dcb0 con 0x7f27f014d390 2024-09-10T10:28:44.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.728+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f27f0062630 con 0x7f27f014d390 2024-09-10T10:28:44.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.730+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f27f0062630 con 0x7f27f014d390 2024-09-10T10:28:44.731 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2: {} 2024-09-10T10:28:44.732 INFO:teuthology.orchestra.run.smithi120.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T10:28:44.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.730+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f27f0063110 0x7f27f006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.730+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f27f0063510 con 0x7f27f0063110 2024-09-10T10:28:44.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.730+0000 7f27eed9d640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f27f0063110 0x7f27f006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.731+0000 7f27eed9d640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f27f0063110 0x7f27f006bc10 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.732+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f0063510 con 0x7f27f0063110 2024-09-10T10:28:44.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.746+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f27f006c150 con 0x7f27f0063110 2024-09-10T10:28:44.747 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:44.746+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:44.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.748+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f27f006c150 con 0x7f27f0063110 2024-09-10T10:28:44.749 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3: {} 2024-09-10T10:28:44.750 INFO:teuthology.orchestra.run.smithi120.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T10:28:44.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.748+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f27f006c870 0x7f27f002c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.748+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f27f006c150 con 0x7f27f006c870 2024-09-10T10:28:44.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.749+0000 7f27ee59c640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f27f006c870 0x7f27f002c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.749+0000 7f27ee59c640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f27f006c870 0x7f27f002c640 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.750+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f006c150 con 0x7f27f006c870 2024-09-10T10:28:44.755 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:44.755 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:44.755 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:44.764 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-10T10:28:44.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.763+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f27f006c150 con 0x7f27f006c870 2024-09-10T10:28:44.767 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4: {} 2024-09-10T10:28:44.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.765+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f27f006c150 con 0x7f27f006c870 2024-09-10T10:28:44.767 INFO:teuthology.orchestra.run.smithi120.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T10:28:44.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.765+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f27f002d090 0x7f27f002d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.765+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f27f0209970 con 0x7f27f002d090 2024-09-10T10:28:44.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.766+0000 7f27edd9b640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f27f002d090 0x7f27f002d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.766+0000 7f27edd9b640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f27f002d090 0x7f27f002d510 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.767+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f0209970 con 0x7f27f002d090 2024-09-10T10:28:44.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.780+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f27f02070e0 con 0x7f27f002d090 2024-09-10T10:28:44.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.783+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f27f02070e0 con 0x7f27f002d090 2024-09-10T10:28:44.784 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5: {} 2024-09-10T10:28:44.784 INFO:teuthology.orchestra.run.smithi120.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T10:28:44.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.783+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f27f0207800 0x7f27f0209bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.783+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f27f020cac0 con 0x7f27f0207800 2024-09-10T10:28:44.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.783+0000 7f27eed9d640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f27f0207800 0x7f27f0209bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.784+0000 7f27eed9d640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f27f0207800 0x7f27f0209bf0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.784+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f020cac0 con 0x7f27f0207800 2024-09-10T10:28:44.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.798+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f27f020ca90 con 0x7f27f0207800 2024-09-10T10:28:44.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.800+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f27f020ca90 con 0x7f27f0207800 2024-09-10T10:28:44.801 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6: {} 2024-09-10T10:28:44.802 INFO:teuthology.orchestra.run.smithi120.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T10:28:44.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.800+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f27f020a640 0x7f27f020aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:44.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.800+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f27f020fe90 con 0x7f27f020a640 2024-09-10T10:28:44.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.801+0000 7f27ee59c640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f27f020a640 0x7f27f020aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:44.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.801+0000 7f27ee59c640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f27f020a640 0x7f27f020aa80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:44.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.802+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f020fe90 con 0x7f27f020a640 2024-09-10T10:28:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.815+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f27f020b0d0 con 0x7f27f020a640 2024-09-10T10:28:44.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27deffd640 1 -- 172.21.15.120:0/1952314734 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f27f020b0d0 con 0x7f27f020a640 2024-09-10T10:28:44.818 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7: {} 2024-09-10T10:28:44.820 INFO:teuthology.orchestra.run.smithi120.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T10:28:44.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f27f00c5b80 msgr2=0x7f27f00c5f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f27f00c5b80 0x7f27f00c5f60 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f27f01500b0 msgr2=0x7f27f01504d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f27f01500b0 0x7f27f01504d0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f27f014d390 msgr2=0x7f27f014d770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f27f014d390 0x7f27f014d770 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f27f0063110 msgr2=0x7f27f006bc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f27f0063110 0x7f27f006bc10 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f27f006c870 msgr2=0x7f27f002c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7f27f006c870 0x7f27f002c640 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f27f002d090 msgr2=0x7f27f002d510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7f27f002d090 0x7f27f002d510 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f27f0207800 msgr2=0x7f27f0209bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f27f0207800 0x7f27f0209bf0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f27f020a640 msgr2=0x7f27f020aa80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f27f020a640 0x7f27f020aa80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27eed9d640 1 -- 172.21.15.120:0/1952314734 reap_dead start 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27eed9d640 1 -- 172.21.15.120:0/1952314734 reap_dead start 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f27d006be30 msgr2=0x7f27d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.818+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f27d006be30 0x7f27d006e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f27e4034000 tx=0x7f27e402e000 comp rx=0 tx=0).stop 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.817+0000 7f27eed9d640 1 -- 172.21.15.120:0/1952314734 reap_dead start 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.818+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27f0067d20 msgr2=0x7f27f01e4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:44.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.818+0000 7f27f4913640 1 --2- 172.21.15.120:0/1952314734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27f0067d20 0x7f27f01e4860 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f27e000c530 tx=0x7f27e000c9f0 comp rx=0 tx=0).stop 2024-09-10T10:28:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.818+0000 7f27eed9d640 1 -- 172.21.15.120:0/1952314734 reap_dead start 2024-09-10T10:28:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.818+0000 7f27eed9d640 1 -- 172.21.15.120:0/1952314734 reap_dead start 2024-09-10T10:28:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.818+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 shutdown_connections 2024-09-10T10:28:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.818+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 >> 172.21.15.120:0/1952314734 conn(0x7f27f00c9400 msgr2=0x7f27f01519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.818+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 shutdown_connections 2024-09-10T10:28:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:44.818+0000 7f27f4913640 1 -- 172.21.15.120:0/1952314734 wait complete. 2024-09-10T10:28:44.870 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:44.870 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.0888052, 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078690670000000004, 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:44.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780025+0000", 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2200000000000004e-07 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780048+0000", 2024-09-10T10:28:44.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2427999999999999e-05 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780057+0000", 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1090000000000004e-06 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780063+0000", 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9780000000000002e-06 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011213480000000001 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781281+0000", 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6728e-05 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781382+0000", 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101455 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784820+0000", 2024-09-10T10:28:44.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034376200000000002 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787779+0000", 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002959373 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787852+0000", 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3135999999999999e-05 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787894+0000", 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1369999999999999e-05 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.088757659, 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079289789999999992, 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780072+0000", 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780073+0000", 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780075+0000", 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5129999999999999e-06 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780082+0000", 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9410000000000001e-06 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780087+0000", 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1880000000000001e-06 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781620+0000", 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001532346 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781698+0000", 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8232999999999994e-05 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781816+0000", 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011825800000000001 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.784824+0000", 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030076349999999998 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787937+0000", 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031134829999999998 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.787969+0000", 2024-09-10T10:28:44.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1745999999999997e-05 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788001+0000", 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2132000000000003e-05 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.088727444, 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0079888619999999994, 2024-09-10T10:28:44.881 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780102+0000", 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780103+0000", 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780106+0000", 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1379999999999999e-06 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780115+0000", 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4299999999999995e-06 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780122+0000", 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3380000000000001e-06 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782041+0000", 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019193700000000001 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782149+0000", 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107493 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782246+0000", 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7399999999999996e-05 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785205+0000", 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002958535 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788041+0000", 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028359650000000002 2024-09-10T10:28:44.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788069+0000", 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8123000000000002e-05 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788091+0000", 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2506000000000001e-05 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.886 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.08869890299999, 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080531070000000003, 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:44.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780131+0000", 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7300000000000001e-07 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780134+0000", 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0570000000000001e-06 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780143+0000", 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-06 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780156+0000", 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3179e-05 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023278280000000001 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.889 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782555+0000", 2024-09-10T10:28:44.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0494000000000006e-05 2024-09-10T10:28:44.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782675+0000", 2024-09-10T10:28:44.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120463 2024-09-10T10:28:44.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785209+0000", 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002533455 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788127+0000", 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029181260000000001 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788155+0000", 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8495000000000001e-05 2024-09-10T10:28:44.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788184+0000", 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8736999999999999e-05 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.088666386, 2024-09-10T10:28:44.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.008097524, 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780163+0000", 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780164+0000", 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.13e-07 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9780000000000001e-06 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780174+0000", 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7510000000000002e-06 2024-09-10T10:28:44.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780179+0000", 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0869999999999997e-06 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782890+0000", 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027113509999999999 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782985+0000", 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5152e-05 2024-09-10T10:28:44.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783090+0000", 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104947 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785222+0000", 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021320269999999999 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788216+0000", 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029940629999999999 2024-09-10T10:28:44.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788242+0000", 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5259999999999999e-05 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788261+0000", 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9395e-05 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.088644015, 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081619529999999996, 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780186+0000", 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780189+0000", 2024-09-10T10:28:44.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.926e-06 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780197+0000", 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2039999999999997e-06 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780202+0000", 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783326+0000", 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003123855 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783402+0000", 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6548e-05 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783558+0000", 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015539199999999999 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785225+0000", 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001666989 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788290+0000", 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030647890000000001 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788316+0000", 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6662e-05 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.788348+0000", 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1417000000000001e-05 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:3 2.10 2:0f65c9b8:::60a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 95.997050502999997, 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066829990000000002, 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.903 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871779+0000", 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871781+0000", 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2129999999999998e-06 2024-09-10T10:28:44.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871790+0000", 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3380000000000005e-06 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871805+0000", 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5353999999999999e-05 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871817+0000", 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1399999999999999e-05 2024-09-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.871932+0000", 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000114982 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872069+0000", 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013713199999999999 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.872208+0000", 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013976900000000001 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.874946+0000", 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027378789999999999 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878355+0000", 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034089139999999999 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878413+0000", 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7540000000000001e-05 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878462+0000", 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9478000000000003e-05 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.921607370999993, 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067332750000000004, 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947222+0000", 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947223+0000", 2024-09-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7299999999999995e-07 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947227+0000", 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.895e-06 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947235+0000", 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947246+0000", 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0738e-05 2024-09-10T10:28:44.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948505+0000", 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001259402 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948578+0000", 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2788999999999997e-05 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948718+0000", 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013953699999999999 2024-09-10T10:28:44.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953391+0000", 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046736670000000003 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953881+0000", 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048956900000000003 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953923+0000", 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2153000000000001e-05 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953956+0000", 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2526e-05 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.921548583000003, 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066654160000000004, 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.913 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947281+0000", 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947282+0000", 2024-09-10T10:28:44.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0399999999999996e-07 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947285+0000", 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7689999999999998e-06 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947292+0000", 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6200000000000001e-06 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947304+0000", 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1702000000000001e-05 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947363+0000", 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9426000000000003e-05 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947485+0000", 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012227599999999999 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947588+0000", 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102641 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950690+0000", 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003102264 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953856+0000", 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031656200000000001 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953907+0000", 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1430000000000001e-05 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953947+0000", 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9164e-05 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.921511409999994, 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071148679999999999, 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947318+0000", 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947319+0000", 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3900000000000005e-07 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947322+0000", 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.523e-06 2024-09-10T10:28:44.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947331+0000", 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3240000000000003e-06 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947336+0000", 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7189999999999998e-06 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947994+0000", 2024-09-10T10:28:44.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065770599999999998 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948080+0000", 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5649000000000006e-05 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111783 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953965+0000", 2024-09-10T10:28:44.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057732089999999996 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954343+0000", 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037846199999999998 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954395+0000", 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.126e-05 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954433+0000", 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8693999999999999e-05 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.921381213999993, 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067554449999999997, 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.924 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947449+0000", 2024-09-10T10:28:44.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6899999999999996e-07 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947454+0000", 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7520000000000004e-06 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947463+0000", 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0310000000000004e-06 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947477+0000", 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4328000000000001e-05 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3869999999999998e-05 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947631+0000", 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.9874e-05 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947750+0000", 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118483 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953644+0000", 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058946889999999998 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954111+0000", 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00046712099999999998 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954162+0000", 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000003e-05 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954204+0000", 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1718000000000003e-05 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.920994175999994, 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064551169999999998, 2024-09-10T10:28:44.929 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947836+0000", 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3e-07 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947841+0000", 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3460000000000004e-06 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947856+0000", 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947864+0000", 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4939999999999997e-06 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947997+0000", 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013305399999999999 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948076+0000", 2024-09-10T10:28:44.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9146000000000004e-05 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948192+0000", 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000115586 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953707+0000", 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0055153470000000003 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954215+0000", 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00050850400000000001 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954258+0000", 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2932000000000002e-05 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954291+0000", 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2322e-05 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.934 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.920954855999994, 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088919089999999999, 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:44.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947875+0000", 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7199999999999999e-07 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947879+0000", 2024-09-10T10:28:44.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8380000000000004e-06 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947887+0000", 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5410000000000003e-06 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947893+0000", 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6930000000000004e-06 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948405+0000", 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051236000000000001 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948502+0000", 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6916999999999994e-05 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948647+0000", 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000145265 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954479+0000", 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058316970000000003 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956680+0000", 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002201494 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956728+0000", 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8161999999999997e-05 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956767+0000", 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.837e-05 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.920392195000005, 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0083279530000000008, 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948438+0000", 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8999999999999995e-07 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948443+0000", 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2189999999999998e-06 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948450+0000", 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8879999999999997e-06 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948457+0000", 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0750000000000003e-06 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948501+0000", 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4527999999999998e-05 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948594+0000", 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3009000000000003e-05 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948696+0000", 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101461 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953756+0000", 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050607129999999997 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956690+0000", 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029334109999999999 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956737+0000", 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7080999999999999e-05 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956766+0000", 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8778000000000001e-05 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.919852520999996, 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0077788589999999999, 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.945 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948977+0000", 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948978+0000", 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948986+0000", 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948998+0000", 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.19e-05 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949007+0000", 2024-09-10T10:28:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.081e-06 2024-09-10T10:28:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949062+0000", 2024-09-10T10:28:44.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5754999999999997e-05 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949172+0000", 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110128 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949282+0000", 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109289 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.955941+0000", 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066594990000000001 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956654+0000", 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071279399999999999 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956706+0000", 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2027000000000002e-05 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956756+0000", 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0012999999999997e-05 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.919314427000003, 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073200349999999999, 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.951 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949515+0000", 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949516+0000", 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999995e-07 2024-09-10T10:28:44.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949522+0000", 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3980000000000004e-06 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949534+0000", 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1562e-05 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949541+0000", 2024-09-10T10:28:44.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1380000000000001e-06 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950700+0000", 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011593459999999999 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950786+0000", 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5556999999999994e-05 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950898+0000", 2024-09-10T10:28:44.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111743 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954326+0000", 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0034280510000000001 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956774+0000", 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024477430000000001 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956815+0000", 2024-09-10T10:28:44.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1752999999999998e-05 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956835+0000", 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0032999999999999e-05 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.919272125999996, 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073303700000000001, 2024-09-10T10:28:44.956 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949558+0000", 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949571+0000", 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2298e-05 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949579+0000", 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5909999999999996e-06 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949585+0000", 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7370000000000003e-06 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953802+0000", 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042170929999999999 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953915+0000", 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011256699999999999 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956888+0000", 2024-09-10T10:28:44.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0029734639999999999 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:24 2.3 e26/18 2:c216c0d1:::mds2_inotable:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 90.918454124999997, 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0063254399999999999, 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950376+0000", 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8500000000000002e-07 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950380+0000", 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2590000000000002e-06 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950387+0000", 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950392+0000", 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0810000000000001e-06 2024-09-10T10:28:44.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950438+0000", 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5592e-05 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950457+0000", 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8569000000000001e-05 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956657+0000", 2024-09-10T10:28:44.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062003409999999998 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.956701+0000", 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4140000000000001e-05 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.3.15:29 2.10 2:0e4da237:::203.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:44.964 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 84.946793322000005, 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0064228799999999997, 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922036+0000", 2024-09-10T10:28:44.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922037+0000", 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922040+0000", 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.694e-06 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922054+0000", 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906e-05 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922065+0000", 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1294000000000001e-05 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922125+0000", 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.982e-05 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.922233+0000", 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000108175 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:19.928459+0000", 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062264570000000003 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:44.968 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 78.927529691999993, 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0081214100000000008, 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941300+0000", 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941301+0000", 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.062e-06 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941306+0000", 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0919999999999997e-06 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941316+0000", 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5640000000000002e-06 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941323+0000", 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1099999999999997e-06 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941405+0000", 2024-09-10T10:28:44.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1650000000000006e-05 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941549+0000", 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014447599999999999 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012929 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:44.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943549+0000", 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018704049999999999 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949343+0000", 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057943300000000003 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949378+0000", 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4474999999999998e-05 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:44.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949421+0000", 2024-09-10T10:28:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3955999999999997e-05 2024-09-10T10:28:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:44.974 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:44.974 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:44.974 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:44.869+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:44.975 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-10T10:28:44.977 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-10T10:28:44.976+0000 7fa40c533640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37195) UID: 0 2024-09-10T10:28:45.078 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:45.077+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:45.120 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:45.120 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:45.121 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-09-10T10:28:45.121 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:45.129 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-10T10:28:45.179 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:45.178+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:45.233 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-09-10T10:28:45.233 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 180, 2024-09-10T10:28:45.233 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-09-10T10:28:45.233 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:45.242 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-10T10:28:45.279 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:45.279+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:45.361 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-10T10:28:45.381 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-10T10:28:45.380+0000 7fe7083d5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37196) UID: 0 2024-09-10T10:28:45.389 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:28:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-09-10T10:28:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-09-10T10:28:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-09-10T10:28:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.606987932, 2024-09-10T10:28:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054521709999999996, 2024-09-10T10:28:45.389 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780404+0000", 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780407+0000", 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.517e-06 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780414+0000", 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3910000000000003e-06 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780429+0000", 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5379999999999998e-05 2024-09-10T10:28:45.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780438+0000", 2024-09-10T10:28:45.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9230000000000008e-06 2024-09-10T10:28:45.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780885+0000", 2024-09-10T10:28:45.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044711300000000001 2024-09-10T10:28:45.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780979+0000", 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4312000000000001e-05 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785856+0000", 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0048765350000000004 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.606913205, 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028177609999999998, 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780478+0000", 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780479+0000", 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9800000000000003e-07 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780488+0000", 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3349999999999996e-06 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780512+0000", 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3156999999999999e-05 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780518+0000", 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.764e-06 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780576+0000", 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7813999999999999e-05 2024-09-10T10:28:45.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.780700+0000", 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123733 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.783296+0000", 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025963599999999998 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:19 2.14 e24/18 2:2f55791f:::606.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.605794045, 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003505083, 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781598+0000", 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9400000000000005e-07 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781606+0000", 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6559999999999992e-06 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781624+0000", 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7575e-05 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781630+0000", 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8630000000000002e-06 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781683+0000", 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2475000000000002e-05 2024-09-10T10:28:45.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781700+0000", 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7393000000000001e-05 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785058+0000", 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033575660000000002 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785103+0000", 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4861000000000003e-05 2024-09-10T10:28:45.401 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:1 2.14 e24/18 2:292cf221:::200.00000000:head v 24'3, mlcod=0'0)", 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.605625539, 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033666960000000002, 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781766+0000", 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781767+0000", 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3099999999999998e-07 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781772+0000", 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5539999999999997e-06 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781781+0000", 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6149999999999997e-06 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781787+0000", 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1589999999999998e-06 2024-09-10T10:28:45.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781860+0000", 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2859000000000002e-05 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.781876+0000", 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5916000000000001e-05 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785108+0000", 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003232146 2024-09-10T10:28:45.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785133+0000", 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5015999999999999e-05 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.0.4:23 2.e e24/18 2:717a0223:::608.00000000:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 118.605011003, 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027543580000000002, 2024-09-10T10:28:45.406 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782381+0000", 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1699999999999998e-07 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782387+0000", 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3229999999999997e-06 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782399+0000", 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2466000000000001e-05 2024-09-10T10:28:45.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782406+0000", 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9460000000000001e-06 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782468+0000", 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2347999999999993e-05 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.782484+0000", 2024-09-10T10:28:45.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6016e-05 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785096+0000", 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026118629999999999 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:26:46.785135+0000", 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8779000000000003e-05 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:24 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.509829296999996, 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062416909999999997, 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877562+0000", 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877564+0000", 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.68e-06 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877576+0000", 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2411e-05 2024-09-10T10:28:45.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877592+0000", 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5812e-05 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877600+0000", 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3590000000000002e-06 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877661+0000", 2024-09-10T10:28:45.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1749999999999997e-05 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877785+0000", 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000123807 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.877936+0000", 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015084300000000001 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.879317+0000", 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001381415 2024-09-10T10:28:45.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883686+0000", 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004368701 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883755+0000", 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9140999999999994e-05 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883804+0000", 2024-09-10T10:28:45.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8772e-05 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.509003332999995, 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0056063609999999998, 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:45.416 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878388+0000", 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878389+0000", 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2699999999999999e-07 2024-09-10T10:28:45.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878394+0000", 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6120000000000001e-06 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878407+0000", 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3704e-05 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878415+0000", 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7100000000000007e-06 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878475+0000", 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878603+0000", 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127525 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.883995+0000", 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053920460000000002 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.1.9:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 96.508952966999999, 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0057312819999999999, 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.420 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878439+0000", 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5000000000000003e-07 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878442+0000", 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.632e-06 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878450+0000", 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0509999999999997e-06 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878456+0000", 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7200000000000003e-06 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878847+0000", 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039182099999999999 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.878949+0000", 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000101458 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:08.884170+0000", 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052210499999999996 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.440287648999998, 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033173069999999998, 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.424 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947104+0000", 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947105+0000", 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4670000000000001e-06 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947111+0000", 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6230000000000002e-06 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947123+0000", 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1883e-05 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947132+0000", 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8740000000000001e-06 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947220+0000", 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7800000000000006e-05 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947375+0000", 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015566399999999999 2024-09-10T10:28:45.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947501+0000", 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125609 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948393+0000", 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00089165399999999997 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950323+0000", 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019308769999999999 2024-09-10T10:28:45.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950366+0000", 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2688999999999997e-05 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950421+0000", 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5167000000000002e-05 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.429 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.440232097000006, 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0070341989999999997, 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947160+0000", 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947163+0000", 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3440000000000001e-06 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947174+0000", 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0248000000000001e-05 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947180+0000", 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0639999999999999e-06 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947958+0000", 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077789700000000005 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948031+0000", 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3503000000000004e-05 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948151+0000", 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119976 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953625+0000", 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054739360000000004 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:45.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954124+0000", 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049858900000000002 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954157+0000", 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3652999999999998e-05 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954194+0000", 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6455000000000002e-05 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.2.13:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.440140253999999, 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032449549999999999, 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947251+0000", 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947252+0000", 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9800000000000003e-07 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947256+0000", 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3760000000000001e-06 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947267+0000", 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0259e-05 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947273+0000", 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2190000000000001e-06 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947332+0000", 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9086000000000003e-05 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947452+0000", 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120146 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.950496+0000", 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003044171 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:13 2.1 e26/18 2:81735765:::619.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.439730123000004, 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.006029375, 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947661+0000", 2024-09-10T10:28:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947662+0000", 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7199999999999997e-07 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6990000000000003e-06 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947681+0000", 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0682999999999999e-05 2024-09-10T10:28:45.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947687+0000", 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7479999999999996e-06 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947742+0000", 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4487000000000002e-05 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.947761+0000", 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9497999999999999e-05 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953653+0000", 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058916530000000002 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.953691+0000", 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7734999999999997e-05 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:14 2.1 e26/18 2:81735765:::619.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.438409785999994, 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052525849999999997, 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948982+0000", 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948983+0000", 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5700000000000002e-07 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.948988+0000", 2024-09-10T10:28:45.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7490000000000004e-06 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949003+0000", 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4214e-05 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949010+0000", 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3499999999999999e-06 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949059+0000", 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8758000000000003e-05 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949078+0000", 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9806e-05 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954198+0000", 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051191509999999997 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954234+0000", 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.68e-05 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.2.13:1 2.1 e26/18 2:85bbe569:::202.00000000:head v 26'4, mlcod=0'0)", 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 91.438181422, 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005049258, 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.447 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949210+0000", 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949211+0000", 2024-09-10T10:28:45.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4999999999999998e-07 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949228+0000", 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7808999999999998e-05 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949240+0000", 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1226000000000001e-05 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949246+0000", 2024-09-10T10:28:45.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0689999999999998e-06 2024-09-10T10:28:45.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949292+0000", 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6211999999999997e-05 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.949311+0000", 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8873e-05 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954236+0000", 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049253980000000001 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:13.954259+0000", 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3221e-05 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(mds.4.17:26 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.445319339999998, 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039596270000000003, 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942072+0000", 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942075+0000", 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2680000000000001e-06 2024-09-10T10:28:45.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942086+0000", 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1361e-05 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942098+0000", 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2221e-05 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942106+0000", 2024-09-10T10:28:45.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0960000000000001e-06 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942209+0000", 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102944 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942346+0000", 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000136976 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:45.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.942516+0000", 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017015699999999999 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945195+0000", 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026790920000000001 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945919+0000", 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00072374500000000003 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.945983+0000", 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3418999999999997e-05 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946032+0000", 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9348000000000003e-05 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.457 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:2 2.1 e28/18 2:82971f97:::404.00000000:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.443660183999995, 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054707489999999996, 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943731+0000", 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943732+0000", 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1999999999999998e-07 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943740+0000", 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1910000000000003e-06 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943755+0000", 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5014000000000001e-05 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943763+0000", 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9519999999999994e-06 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5569999999999997e-05 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943862+0000", 2024-09-10T10:28:45.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4150000000000003e-05 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949157+0000", 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0052949340000000003 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949202+0000", 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5018000000000002e-05 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:6 2.e e28/18 2:75ee5fd8:::629.00000000:head v 28'5, mlcod=24'2)", 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.443598355999995, 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027501769999999999, 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943793+0000", 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943794+0000", 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4799999999999999e-07 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943799+0000", 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5620000000000003e-06 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943807+0000", 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.486e-06 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943813+0000", 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8810000000000001e-06 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6419999999999999e-05 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943888+0000", 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9309000000000001e-05 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946498+0000", 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026094669999999999 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946543+0000", 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.465 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:17 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'12, mlcod=26'10)", 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.443554594999995, 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027214219999999998, 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:45.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943837+0000", 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2300000000000002e-07 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943849+0000", 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2023e-05 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943857+0000", 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6009999999999997e-06 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943863+0000", 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2899999999999999e-06 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943927+0000", 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3496000000000003e-05 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943949+0000", 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1801e-05 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946490+0000", 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025409579999999998 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946558+0000", 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8830000000000003e-05 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:18 2.14 e28/18 2:2eebc8f4:::62f.00000000:head v 28'13, mlcod=26'10)", 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.443508393000002, 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027193059999999999, 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.470 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943883+0000", 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943884+0000", 2024-09-10T10:28:45.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9299999999999999e-07 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943889+0000", 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.95e-06 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943896+0000", 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3819999999999998e-06 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.943901+0000", 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1719999999999998e-06 2024-09-10T10:28:45.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944666+0000", 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00076445300000000001 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.944686+0000", 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.012e-05 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018748879999999999 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946603+0000", 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1947999999999999e-05 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_repop(mds.4.17:25 2.12 e28/18 2:4dad84a4:::mds4_sessionmap:head v 28'5, mlcod=26'3)", 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 79.440594430000004, 2024-09-10T10:28:45.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002978994, 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946797+0000", 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946798+0000", 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0100000000000001e-06 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946807+0000", 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9700000000000005e-06 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946820+0000", 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2458e-05 2024-09-10T10:28:45.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946856+0000", 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6427999999999997e-05 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946938+0000", 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2106000000000003e-05 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.946958+0000", 2024-09-10T10:28:45.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0162000000000001e-05 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949723+0000", 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0027645899999999999 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-09-10T10:27:25.949776+0000", 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3270000000000003e-05 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T10:28:45.479 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T10:28:45.479 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:28:45.480 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-10T10:28:45.481 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T10:28:45.481 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T10:28:45.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.461+0000 7f7d90aac640 1 Processor -- start 2024-09-10T10:28:45.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.461+0000 7f7d90aac640 1 -- start start 2024-09-10T10:28:45.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d90aac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 0x7f7d8c158670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:45.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d90aac640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7d8c158bb0 0x7f7d8c0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:45.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d90aac640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8c072f50 con 0x7f7d8c0c6bb0 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d90aac640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8c073210 con 0x7f7d8c158bb0 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d90aac640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c15cb50 con 0x7f7d8c152850 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d81d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7d8c158bb0 0x7f7d8c0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d8a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 0x7f7d8c158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d8ad9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f7d8c0c6bb0 0x7f7d8c0c86f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:46684/0 (socket says 172.21.15.120:46684) 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d8a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 0x7f7d8c158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:38004/0 (socket says 172.21.15.120:38004) 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d81d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7d8c158bb0 0x7f7d8c0c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:52564/0 (socket says 172.21.15.120:52564) 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d8ad9d640 1 -- 172.21.15.120:0/3205097035 learned_addr learned my addr 172.21.15.120:0/3205097035 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d8a59c640 1 -- 172.21.15.120:0/3205097035 >> v2:172.21.15.134:3300/0 conn(0x7f7d8c158bb0 msgr2=0x7f7d8c0c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d8a59c640 1 --2- 172.21.15.120:0/3205097035 >> v2:172.21.15.134:3300/0 conn(0x7f7d8c158bb0 0x7f7d8c0c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d8a59c640 1 -- 172.21.15.120:0/3205097035 >> v1:172.21.15.134:6789/0 conn(0x7f7d8c0c6bb0 legacy=0x7f7d8c0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d8a59c640 1 -- 172.21.15.120:0/3205097035 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c15bd90 con 0x7f7d8c152850 2024-09-10T10:28:45.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.462+0000 7f7d81d9b640 1 --2- 172.21.15.120:0/3205097035 >> v2:172.21.15.134:3300/0 conn(0x7f7d8c158bb0 0x7f7d8c0c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.463+0000 7f7d8a59c640 1 --2- 172.21.15.120:0/3205097035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 0x7f7d8c158670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0097a0 tx=0x7f7d7c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3367e992088ff23b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.463+0000 7f7d89d9b640 1 -- 172.21.15.120:0/3205097035 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c037070 con 0x7f7d8c152850 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.463+0000 7f7d89d9b640 1 -- 172.21.15.120:0/3205097035 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7c032440 con 0x7f7d8c152850 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.463+0000 7f7d89d9b640 1 -- 172.21.15.120:0/3205097035 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c032810 con 0x7f7d8c152850 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.464+0000 7f7d90aac640 1 -- 172.21.15.120:0/3205097035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 msgr2=0x7f7d8c158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.464+0000 7f7d90aac640 1 --2- 172.21.15.120:0/3205097035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 0x7f7d8c158670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0097a0 tx=0x7f7d7c02a180 comp rx=0 tx=0).stop 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.464+0000 7f7d90aac640 1 -- 172.21.15.120:0/3205097035 shutdown_connections 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.464+0000 7f7d90aac640 1 --2- 172.21.15.120:0/3205097035 >> v2:172.21.15.134:3300/0 conn(0x7f7d8c158bb0 0x7f7d8c0c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.464+0000 7f7d90aac640 1 --2- 172.21.15.120:0/3205097035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 0x7f7d8c158670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.464+0000 7f7d90aac640 1 -- 172.21.15.120:0/3205097035 >> 172.21.15.120:0/3205097035 conn(0x7f7d8c06f3c0 msgr2=0x7f7d8c06f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.464+0000 7f7d90aac640 1 -- 172.21.15.120:0/3205097035 shutdown_connections 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.464+0000 7f7d90aac640 1 -- 172.21.15.120:0/3205097035 wait complete. 2024-09-10T10:28:45.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.465+0000 7f7d90aac640 1 Processor -- start 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.465+0000 7f7d90aac640 1 -- start start 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d90aac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d8c0c6bb0 0x7f7d8c152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d90aac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 0x7f7d8c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d90aac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c158bb0 0x7f7d8c14ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d90aac640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c15c630 con 0x7f7d8c158bb0 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d90aac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c15c350 con 0x7f7d8c0c6bb0 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d90aac640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c070200 con 0x7f7d8c152850 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d8ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c158bb0 0x7f7d8c14ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d8ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c158bb0 0x7f7d8c14ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32806/0 (socket says 172.21.15.120:32806) 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d8ad9d640 1 -- 172.21.15.120:0/2956189345 learned_addr learned my addr 172.21.15.120:0/2956189345 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d8a59c640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d8c0c6bb0 0x7f7d8c152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.466+0000 7f7d81d9b640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 0x7f7d8c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:45.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d8ad9d640 1 -- 172.21.15.120:0/2956189345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 msgr2=0x7f7d8c1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d8ad9d640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 0x7f7d8c1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d8ad9d640 1 -- 172.21.15.120:0/2956189345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d8c0c6bb0 msgr2=0x7f7d8c152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d8ad9d640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d8c0c6bb0 0x7f7d8c152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d8ad9d640 1 -- 172.21.15.120:0/2956189345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c1532c0 con 0x7f7d8c158bb0 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d8ad9d640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c158bb0 0x7f7d8c14ba30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400cbf0 tx=0x7f7d8400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d837fe640 1 -- 172.21.15.120:0/2956189345 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84017070 con 0x7f7d8c158bb0 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d837fe640 1 -- 172.21.15.120:0/2956189345 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d8400ad40 con 0x7f7d8c158bb0 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d90aac640 1 -- 172.21.15.120:0/2956189345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c1eedd0 con 0x7f7d8c158bb0 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d837fe640 1 -- 172.21.15.120:0/2956189345 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84012600 con 0x7f7d8c158bb0 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.467+0000 7f7d90aac640 1 -- 172.21.15.120:0/2956189345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c1ee2f0 con 0x7f7d8c158bb0 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.468+0000 7f7d90aac640 1 -- 172.21.15.120:0/2956189345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8c0718c0 con 0x7f7d8c158bb0 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.468+0000 7f7d837fe640 1 -- 172.21.15.120:0/2956189345 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d8401a070 con 0x7f7d8c158bb0 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.469+0000 7f7d837fe640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 0x7f7d6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.469+0000 7f7d837fe640 1 -- 172.21.15.120:0/2956189345 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6986+0+0 (secure 0 0 0) 0x7f7d8408afe0 con 0x7f7d8c158bb0 2024-09-10T10:28:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.469+0000 7f7d8a59c640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 0x7f7d6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:45.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.469+0000 7f7d8a59c640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 0x7f7d6c06e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c034000 tx=0x7f7d7c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:45.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.473+0000 7f7d837fe640 1 -- 172.21.15.120:0/2956189345 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7d84058e40 con 0x7f7d8c158bb0 2024-09-10T10:28:45.521 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:45.520+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:45.557 INFO:tasks.workunit:Stopping ['fs/misc/trivial_sync.sh'] on client.0... 2024-09-10T10:28:45.557 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T10:28:45.586 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-10T10:28:45.585+0000 7f8ef7a88640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37188) UID: 0 2024-09-10T10:28:45.600 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:45.600 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:45.600 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:45.600 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:45.609 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-10T10:28:45.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:45.656+0000 7f7d90aac640 1 -- 172.21.15.120:0/2956189345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f7d8c0725d0 con 0x7f7d8c158bb0 2024-09-10T10:28:45.693 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-10T10:28:45.692+0000 7fa15f73b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37192) UID: 0 2024-09-10T10:28:45.752 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:45.752 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:45.752 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:45.752 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:45.752 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:45.761 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-10T10:28:45.787 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-09-10T10:28:45.786+0000 7fd133a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36315) UID: 0 2024-09-10T10:28:45.870 DEBUG:teuthology.parallel:result is None 2024-09-10T10:28:45.871 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T10:28:45.908 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.12732423600001, 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047203150000000001, 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779665+0000", 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779674+0000", 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0359999999999995e-06 2024-09-10T10:28:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779702+0000", 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.7970000000000002e-05 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779724+0000", 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2544e-05 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779741+0000", 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6781000000000001e-05 2024-09-10T10:28:45.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779981+0000", 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000239764 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780147+0000", 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016605100000000001 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780318+0000", 2024-09-10T10:28:45.912 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017096200000000001 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781749+0000", 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001431288 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784234+0000", 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024850940000000002 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.913 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784338+0000", 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104078 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784385+0000", 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6746999999999998e-05 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.1a epoch 24/18 rep_tid 2 entries 24'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.125846126, 2024-09-10T10:28:45.914 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021396129999999998, 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781143+0000", 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781144+0000", 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.393e-06 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781156+0000", 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1514e-05 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781184+0000", 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8445999999999998e-05 2024-09-10T10:28:45.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781202+0000", 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7439e-05 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781286+0000", 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4652000000000002e-05 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.917 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783282+0000", 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0019961689999999999 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:31 2.d e24/18 2:b50e409b:::mds_snaptable:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.12492663499999, 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030643699999999999, 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.918 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782062+0000", 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782063+0000", 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3699999999999999e-07 2024-09-10T10:28:45.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782072+0000", 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9430000000000006e-06 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782083+0000", 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0998e-05 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.920 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782092+0000", 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.3270000000000001e-06 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782158+0000", 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6830999999999998e-05 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782176+0000", 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7370000000000001e-05 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.921 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785091+0000", 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029157879999999999 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785127+0000", 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5176000000000003e-05 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.922 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:27 2.13 e24/18 2:cd5a64a3:::100.00000000:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.124867744, 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002013515, 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.923 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782121+0000", 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782122+0000", 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.229e-06 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782137+0000", 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4895999999999999e-05 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782151+0000", 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3229e-05 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782198+0000", 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7301e-05 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782271+0000", 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3516000000000006e-05 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.926 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782290+0000", 2024-09-10T10:28:45.926 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8921000000000001e-05 2024-09-10T10:28:45.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.926 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.926 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784094+0000", 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00180331 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784135+0000", 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1112999999999998e-05 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.928 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:28 2.13 e24/18 2:c91d4a1d:::mds0_inotable:head v 24'2, mlcod=0'0)", 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.124756131, 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028487970000000001, 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782233+0000", 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7000000000000005e-07 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782247+0000", 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3740000000000001e-05 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.930 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782258+0000", 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0552e-05 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782265+0000", 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0700000000000001e-06 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782752+0000", 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048707399999999999 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.931 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782771+0000", 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9046999999999999e-05 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785037+0000", 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002265806 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785082+0000", 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4938000000000002e-05 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.932 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.0.4:29 2.b e24/18 2:d0630e4c:::mds0_sessionmap:head v 24'1, mlcod=0'0)", 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.123210136, 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001932907, 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.933 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783779+0000", 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783780+0000", 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9900000000000009e-07 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783789+0000", 2024-09-10T10:28:45.934 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0140000000000004e-06 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783802+0000", 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2751e-05 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783811+0000", 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3950000000000007e-06 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783894+0000", 2024-09-10T10:28:45.935 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2887999999999997e-05 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.783913+0000", 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9052999999999999e-05 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785668+0000", 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001754633 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785712+0000", 2024-09-10T10:28:45.936 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4174000000000001e-05 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:7 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.034578096000004, 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026422899999999998, 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872411+0000", 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.561e-06 2024-09-10T10:28:45.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872426+0000", 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3577999999999999e-05 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872462+0000", 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6145e-05 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872473+0000", 2024-09-10T10:28:45.939 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1109e-05 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872543+0000", 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9319000000000005e-05 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872566+0000", 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3768999999999999e-05 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.940 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875010+0000", 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024433910000000001 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875053+0000", 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3417999999999997e-05 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:8 2.b e26/18 2:d70a01c6:::60c.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:45.941 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.033889707, 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0044428820000000004, 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873099+0000", 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873100+0000", 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1999999999999999e-07 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873109+0000", 2024-09-10T10:28:45.943 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3030000000000005e-06 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873121+0000", 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1471000000000001e-05 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873129+0000", 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7339999999999997e-06 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:45.944 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00048786499999999998 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873638+0000", 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0735000000000001e-05 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877493+0000", 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0038550189999999999 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.945 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877542+0000", 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9135000000000003e-05 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:17 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'3, mlcod=24'1)", 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.033672374000005, 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042663060000000001, 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.946 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873317+0000", 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873318+0000", 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7999999999999993e-07 2024-09-10T10:28:45.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873326+0000", 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8279999999999992e-06 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873341+0000", 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4688999999999999e-05 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873348+0000", 2024-09-10T10:28:45.948 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4810000000000003e-06 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873843+0000", 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000495083 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873860+0000", 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7609999999999999e-05 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877545+0000", 2024-09-10T10:28:45.949 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036848839999999998 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877583+0000", 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7750999999999998e-05 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:18 2.13 e26/18 2:ca4dd7f5:::611.00000000:head v 26'4, mlcod=24'1)", 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.032950337000003, 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035807780000000002, 2024-09-10T10:28:45.950 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.951 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:45.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.952 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874039+0000", 2024-09-10T10:28:45.952 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9400000000000005e-07 2024-09-10T10:28:45.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.952 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874045+0000", 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9909999999999999e-06 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874055+0000", 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0147e-05 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874070+0000", 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4184e-05 2024-09-10T10:28:45.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874123+0000", 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3504999999999997e-05 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874141+0000", 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7524999999999999e-05 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877586+0000", 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034458700000000002 2024-09-10T10:28:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877619+0000", 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2962000000000001e-05 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.18 epoch 26/18 rep_tid 5 entries 26'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.9:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.031888691999995, 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048872359999999997, 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.955 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875100+0000", 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875101+0000", 2024-09-10T10:28:45.956 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875110+0000", 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2609999999999999e-06 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875129+0000", 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8685000000000002e-05 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.957 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875143+0000", 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3723000000000001e-05 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875222+0000", 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9498000000000004e-05 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879987+0000", 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047652800000000002 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.958 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:7 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'5, mlcod=26'3)", 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 91.957126395000003, 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031425310000000001, 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949863+0000", 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1100000000000005e-07 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949871+0000", 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8620000000000003e-06 2024-09-10T10:28:45.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949885+0000", 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4072e-05 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949893+0000", 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5769999999999996e-06 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949953+0000", 2024-09-10T10:28:45.961 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.978e-05 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949972+0000", 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9188e-05 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.952959+0000", 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029873859999999999 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953005+0000", 2024-09-10T10:28:45.962 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5855000000000001e-05 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:8 2.13 e26/18 2:cdf3ac81:::616.00000000:head v 26'6, mlcod=26'3)", 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 91.957073805999997, 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034236359999999999, 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.963 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949915+0000", 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949916+0000", 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6199999999999998e-07 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.964 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949922+0000", 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5390000000000004e-06 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949929+0000", 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4339999999999997e-06 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949934+0000", 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2159999999999997e-06 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.965 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950192+0000", 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00025864699999999998 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950213+0000", 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0761999999999999e-05 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953297+0000", 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030842999999999999 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.966 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953339+0000", 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1276000000000003e-05 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:34 2.11 e26/18 2:8975f766:::202.00000001:head v 26'3, mlcod=26'1)", 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 91.953100966999997, 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020619190000000002, 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.967 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953888+0000", 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953889+0000", 2024-09-10T10:28:45.968 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.879999999999999e-07 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953895+0000", 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2249999999999997e-06 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953906+0000", 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.969 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953912+0000", 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4840000000000003e-06 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953959+0000", 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7768000000000001e-05 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953976+0000", 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6365999999999999e-05 2024-09-10T10:28:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955917+0000", 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001941524 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955950+0000", 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2573999999999997e-05 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.971 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:4 2.1a e26/18 2:5fdc322d:::61e.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.984033439000001, 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0018931340000000001, 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.972 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922955+0000", 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922956+0000", 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-07 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922962+0000", 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1890000000000004e-06 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.973 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922971+0000", 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922978+0000", 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7830000000000001e-06 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923032+0000", 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3637999999999997e-05 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.974 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923049+0000", 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7249999999999999e-05 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924803+0000", 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001753788 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924849+0000", 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6050000000000001e-05 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:20 2.11 e26/18 2:890f50f7:::626.00000000:head v 26'5, mlcod=26'2)", 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.983925733000007, 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003328012, 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923063+0000", 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923064+0000", 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9900000000000001e-07 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923070+0000", 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7499999999999997e-06 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.977 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923079+0000", 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9080000000000001e-06 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923089+0000", 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0078000000000001e-05 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923150+0000", 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0300999999999997e-05 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.978 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923507+0000", 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00035713599999999998 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926368+0000", 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028615469999999999 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926391+0000", 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2793e-05 2024-09-10T10:28:45.979 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:23 2.18 e26/18 2:1c1db442:::103.00000000.inode:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.983866264, 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032793050000000002, 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.980 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923123+0000", 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923124+0000", 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0599999999999999e-07 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923130+0000", 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.81e-06 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923141+0000", 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.099e-05 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923149+0000", 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2679999999999997e-06 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923213+0000", 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4744000000000002e-05 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923233+0000", 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9190000000000001e-05 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926353+0000", 2024-09-10T10:28:45.983 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031206419999999999 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926402+0000", 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8755000000000003e-05 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:13 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'6, mlcod=26'2)", 2024-09-10T10:28:45.984 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.983813634000001, 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031818969999999999, 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.985 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923175+0000", 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923176+0000", 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4799999999999998e-07 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923182+0000", 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.592e-06 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923191+0000", 2024-09-10T10:28:45.986 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4029999999999996e-06 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923197+0000", 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1850000000000001e-06 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923230+0000", 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3290000000000001e-05 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.987 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923247+0000", 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6810999999999999e-05 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926308+0000", 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030606159999999999 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926357+0000", 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9351999999999998e-05 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.988 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "pg_update_log_missing(2.d epoch 26/18 rep_tid 20 entries 26'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.15:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.983261135999996, 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034028819999999999, 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "reached pg", 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.989 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923728+0000", 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923729+0000", 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.5900000000000005e-07 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.990 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923739+0000", 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8439999999999993e-06 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923761+0000", 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2821999999999999e-05 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923776+0000", 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4763000000000001e-05 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.923889+0000", 2024-09-10T10:28:45.991 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000112518 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927131+0000", 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0032419760000000001 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.3.15:14 2.1a e26/18 2:594e21d0:::623.00000000:head v 26'7, mlcod=26'2)", 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 85.982546174999996, 2024-09-10T10:28:45.992 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027184890000000001, 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:45.993 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924443+0000", 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5099999999999999e-07 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924449+0000", 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0310000000000004e-06 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924482+0000", 2024-09-10T10:28:45.994 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2845999999999997e-05 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924500+0000", 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.787e-05 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924594+0000", 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3566999999999994e-05 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:45.995 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924619+0000", 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4978999999999999e-05 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927118+0000", 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024995479999999999 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927161+0000", 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2997000000000002e-05 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.996 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:45.997 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:45.997 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:45.998 INFO:tasks.ceph.osd.2.smithi120.stderr:2024-09-10T10:28:45.911+0000 7f4fedefc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36328) UID: 0 2024-09-10T10:28:45.998 INFO:tasks.ceph.osd.3.smithi120.stderr:2024-09-10T10:28:45.997+0000 7fc377be4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36332) UID: 0 2024-09-10T10:28:45.999 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-10T10:28:46.028 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T10:28:46.028 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-10T10:28:46.038 INFO:tasks.thrashosds:joining thrashosds 2024-09-10T10:28:46.103 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:46.103 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:46.103 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:46.104 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:46.112 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-10T10:28:46.215 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:46.216 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:46.224 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-10T10:28:46.329 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:46.329 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:46.329 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:46.329 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:46.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.329 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:11 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.454856727999996, 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.006807553, 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872287+0000", 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872290+0000", 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2029999999999999e-06 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872310+0000", 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0508000000000001e-05 2024-09-10T10:28:46.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872331+0000", 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0577e-05 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872341+0000", 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0421999999999999e-05 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872457+0000", 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115618 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872575+0000", 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118563 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872690+0000", 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011438900000000001 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875281+0000", 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002590995 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879035+0000", 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037544879999999998 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879069+0000", 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4034e-05 2024-09-10T10:28:46.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879095+0000", 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5755999999999999e-05 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:46.335 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.453660107000005, 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0061030149999999998, 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.336 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873484+0000", 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873485+0000", 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2419999999999999e-06 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873492+0000", 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.669e-06 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873505+0000", 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.325e-05 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873514+0000", 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1949999999999992e-06 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873575+0000", 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0037000000000001e-05 2024-09-10T10:28:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873647+0000", 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2346000000000002e-05 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873778+0000", 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013125600000000001 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875444+0000", 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001665962 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879528+0000", 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040837809999999999 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879563+0000", 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5329e-05 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879587+0000", 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3947999999999998e-05 2024-09-10T10:28:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.453606057000002, 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096274789999999996, 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873538+0000", 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873539+0000", 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0299999999999998e-07 2024-09-10T10:28:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873554+0000", 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5481999999999998e-05 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873565+0000", 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0733e-05 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873573+0000", 2024-09-10T10:28:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9270000000000005e-06 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873633+0000", 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9864000000000001e-05 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873752+0000", 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118946 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873866+0000", 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000113757 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875462+0000", 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015966699999999999 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883084+0000", 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076214280000000004 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883136+0000", 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2182000000000003e-05 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883166+0000", 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.9787000000000001e-05 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.453548085999998, 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097094669999999994, 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873596+0000", 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873597+0000", 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8899999999999998e-07 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873602+0000", 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9230000000000001e-06 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873611+0000", 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9150000000000002e-06 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873617+0000", 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5239999999999997e-06 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874719+0000", 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0011012769999999999 2024-09-10T10:28:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874791+0000", 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2305000000000007e-05 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874962+0000", 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000171099 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877786+0000", 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028241059999999998 2024-09-10T10:28:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883211+0000", 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054250019999999999 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883263+0000", 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1563000000000001e-05 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883306+0000", 2024-09-10T10:28:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2864000000000002e-05 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.452627395999997, 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005164681, 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874517+0000", 2024-09-10T10:28:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874518+0000", 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3799999999999996e-07 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874521+0000", 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8550000000000004e-06 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874531+0000", 2024-09-10T10:28:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.8079999999999999e-06 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874538+0000", 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2760000000000003e-06 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874945+0000", 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000406516 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875045+0000", 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9570999999999999e-05 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.879682+0000", 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046369169999999999 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.452593500999996, 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056651770000000004, 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874551+0000", 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4600000000000005e-07 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874565+0000", 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4017e-05 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874573+0000", 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9689999999999994e-06 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874581+0000", 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874678+0000", 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7483000000000002e-05 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874786+0000", 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107914 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.880216+0000", 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005429499 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:24 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:46.360 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 92.378285177999999, 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0079888200000000006, 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.361 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948859+0000", 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948861+0000", 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9269999999999999e-06 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948869+0000", 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7749999999999993e-06 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.362 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948884+0000", 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5713e-05 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948894+0000", 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2050000000000008e-06 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948958+0000", 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4398000000000002e-05 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.363 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949093+0000", 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013508300000000001 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.949223+0000", 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012949500000000001 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953128+0000", 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039058360000000002 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.364 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956775+0000", 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0036467439999999999 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956812+0000", 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6510000000000001e-05 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.956848+0000", 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6134000000000003e-05 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.365 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.2.13:23 2.c e26/18 2:3631e767:::102.00000000.inode:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 92.376800927999994, 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054176620000000002, 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.366 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950343+0000", 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950344+0000", 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9400000000000001e-07 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950362+0000", 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8655999999999999e-05 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950384+0000", 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1929999999999998e-05 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950391+0000", 2024-09-10T10:28:46.368 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7020000000000004e-06 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950624+0000", 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000233031 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950643+0000", 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9459000000000001e-05 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.369 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955726+0000", 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050826389999999999 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.955761+0000", 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4851000000000002e-05 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:46.370 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.406151942999998, 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0070980829999999998, 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.371 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920992+0000", 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.920994+0000", 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.697e-06 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921001+0000", 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3719999999999999e-06 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921014+0000", 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2529e-05 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921021+0000", 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9519999999999997e-06 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921097+0000", 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6499000000000001e-05 2024-09-10T10:28:46.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921261+0000", 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016399599999999999 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921396+0000", 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013448400000000001 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922936+0000", 2024-09-10T10:28:46.374 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015405970000000001 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927988+0000", 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00505158 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928047+0000", 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9011000000000001e-05 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.375 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928090+0000", 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3365999999999998e-05 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.406100432000002, 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007356853, 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.376 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.377 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921044+0000", 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5699999999999998e-07 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921048+0000", 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.754e-06 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921056+0000", 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0469999999999994e-06 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921063+0000", 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5980000000000002e-06 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921747+0000", 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00068475800000000005 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921852+0000", 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000104347 2024-09-10T10:28:46.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922002+0000", 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000150233 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924357+0000", 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0023545599999999999 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928286+0000", 2024-09-10T10:28:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039299540000000003 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928349+0000", 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2044000000000004e-05 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.928401+0000", 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2101000000000001e-05 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.406060973999999, 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005625166, 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921083+0000", 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921084+0000", 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0100000000000005e-07 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921093+0000", 2024-09-10T10:28:46.383 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9910000000000008e-06 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921099+0000", 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2269999999999998e-06 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921103+0000", 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0929999999999996e-06 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.384 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921167+0000", 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4030999999999997e-05 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921301+0000", 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013388699999999999 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921471+0000", 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00017000899999999999 2024-09-10T10:28:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.924553+0000", 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0030818199999999999 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926600+0000", 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020471339999999999 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926663+0000", 2024-09-10T10:28:46.386 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3274000000000004e-05 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926708+0000", 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5099000000000003e-05 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.383509304, 2024-09-10T10:28:46.387 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010012200000000001, 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943635+0000", 2024-09-10T10:28:46.388 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943637+0000", 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5390000000000002e-06 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943646+0000", 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1340000000000004e-06 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943661+0000", 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4958e-05 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943669+0000", 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5629999999999992e-06 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943785+0000", 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011584800000000001 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943960+0000", 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000174611 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944078+0000", 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000118381 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948022+0000", 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0039442599999999998 2024-09-10T10:28:46.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953546+0000", 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055242449999999997 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953604+0000", 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7555e-05 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953647+0000", 2024-09-10T10:28:46.392 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3105999999999997e-05 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.383315745000004, 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099609380000000008, 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943828+0000", 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943829+0000", 2024-09-10T10:28:46.394 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3099999999999996e-07 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943835+0000", 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8610000000000003e-06 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943846+0000", 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0686999999999999e-05 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.395 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943854+0000", 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7200000000000006e-06 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944413+0000", 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055893399999999997 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944505+0000", 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2788999999999995e-05 2024-09-10T10:28:46.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944639+0000", 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013387699999999999 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950701+0000", 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0060612360000000002 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.397 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953700+0000", 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029997689999999998 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953743+0000", 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3024e-05 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953789+0000", 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6109999999999997e-05 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.398 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.383275722999997, 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0054304089999999998, 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.399 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943868+0000", 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943869+0000", 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1399999999999996e-07 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943874+0000", 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3830000000000002e-06 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943881+0000", 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1369999999999996e-06 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943886+0000", 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3600000000000004e-06 2024-09-10T10:28:46.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944057+0000", 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000170781 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944182+0000", 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012506299999999999 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944309+0000", 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127081 2024-09-10T10:28:46.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.947998+0000", 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003689323 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949225+0000", 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001227005 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949267+0000", 2024-09-10T10:28:46.403 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1391999999999999e-05 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949299+0000", 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2169999999999999e-05 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.383215241000002, 2024-09-10T10:28:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0068819399999999996, 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943929+0000", 2024-09-10T10:28:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943930+0000", 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1200000000000002e-07 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943934+0000", 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0969999999999999e-06 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.406 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944782+0000", 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00084778299999999996 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944793+0000", 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0993e-05 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944864+0000", 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1241000000000005e-05 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944940+0000", 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6044999999999994e-05 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 1", 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945068+0000", 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012846 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949683+0000", 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046151040000000001 2024-09-10T10:28:46.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950706+0000", 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010222149999999999 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950764+0000", 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8180999999999997e-05 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.409 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950811+0000", 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7108999999999999e-05 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:24 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.381841614999999, 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0055052950000000003, 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.410 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945303+0000", 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945304+0000", 2024-09-10T10:28:46.411 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4799999999999997e-07 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945311+0000", 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1269999999999997e-06 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945322+0000", 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1134e-05 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.412 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945330+0000", 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9039999999999992e-06 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945408+0000", 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8200000000000003e-05 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945533+0000", 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000125518 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.413 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 2", 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945659+0000", 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012583599999999999 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949147+0000", 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003487675 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950716+0000", 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015695279999999999 2024-09-10T10:28:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950767+0000", 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0577e-05 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950808+0000", 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0847999999999999e-05 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:25 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.381200246000006, 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0050290129999999997, 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945944+0000", 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945945+0000", 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1119999999999999e-06 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945952+0000", 2024-09-10T10:28:46.417 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5189999999999998e-06 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945965+0000", 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3643e-05 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945974+0000", 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5750000000000001e-06 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:46.418 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0532999999999998e-05 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946192+0000", 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015717699999999999 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946341+0000", 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014897200000000001 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.419 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.949885+0000", 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035449729999999999 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950896+0000", 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00101023 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950938+0000", 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2284999999999999e-05 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.420 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950973+0000", 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4993999999999997e-05 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.381147174000006, 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048920780000000002, 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:46.421 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945997+0000", 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945998+0000", 2024-09-10T10:28:46.422 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3900000000000004e-07 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946012+0000", 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4799e-05 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946021+0000", 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.6000000000000007e-06 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946029+0000", 2024-09-10T10:28:46.423 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9209999999999992e-06 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946093+0000", 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3608000000000005e-05 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946242+0000", 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014889699999999999 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.424 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950889+0000", 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046476140000000004 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.381103147000005, 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047976679999999997, 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.425 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946041+0000", 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.426 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946042+0000", 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5199999999999996e-07 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946046+0000", 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1439999999999996e-06 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946054+0000", 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1140000000000006e-06 2024-09-10T10:28:46.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946070+0000", 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6232e-05 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946137+0000", 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6956000000000001e-05 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.428 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946260+0000", 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012240299999999999 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950839+0000", 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045791670000000003 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.429 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:23 2.1c e28/18 2:399c8d31:::104.00000000.inode:head v 28'5, mlcod=24'1)", 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.380583807999997, 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0042418079999999997, 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946560+0000", 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946561+0000", 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3e-07 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946574+0000", 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.277e-05 2024-09-10T10:28:46.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946584+0000", 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7710000000000001e-06 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946590+0000", 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6769999999999998e-06 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.432 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946666+0000", 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5420999999999995e-05 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946687+0000", 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1665999999999999e-05 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950762+0000", 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0040749030000000004 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.434 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.434 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950802+0000", 2024-09-10T10:28:46.434 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9969999999999998e-05 2024-09-10T10:28:46.434 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.434 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.434 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.434 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.434 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.434 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:46.436 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-10T10:28:46.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.364+0000 7f7d837fe640 1 -- 172.21.15.120:0/2956189345 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v61) ==== 101+0+0 (secure 0 0 0) 0x7f7d8405d140 con 0x7f7d8c158bb0 2024-09-10T10:28:46.437 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-10T10:28:46.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 -- 172.21.15.120:0/2956189345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 msgr2=0x7f7d6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:46.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 0x7f7d6c06e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c034000 tx=0x7f7d7c02e000 comp rx=0 tx=0).stop 2024-09-10T10:28:46.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 -- 172.21.15.120:0/2956189345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c158bb0 msgr2=0x7f7d8c14ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:46.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c158bb0 0x7f7d8c14ba30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400cbf0 tx=0x7f7d8400a430 comp rx=0 tx=0).stop 2024-09-10T10:28:46.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 -- 172.21.15.120:0/2956189345 shutdown_connections 2024-09-10T10:28:46.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d6c06be30 0x7f7d6c06e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:46.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d8c158bb0 0x7f7d8c14ba30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:46.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d8c152850 0x7f7d8c1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:46.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 --2- 172.21.15.120:0/2956189345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d8c0c6bb0 0x7f7d8c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:46.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 -- 172.21.15.120:0/2956189345 >> 172.21.15.120:0/2956189345 conn(0x7f7d8c06f3c0 msgr2=0x7f7d8c14ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:46.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 -- 172.21.15.120:0/2956189345 shutdown_connections 2024-09-10T10:28:46.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:46.366+0000 7f7d90aac640 1 -- 172.21.15.120:0/2956189345 wait complete. 2024-09-10T10:28:46.543 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:46.543 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:46.543 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:46.543 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:46.552 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-10T10:28:46.654 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:46.654 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:46.654 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:46.654 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:46.655 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:46.663 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-10T10:28:46.767 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:46.767 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:46.767 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:46.767 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:46.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.767 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:46.767 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:46.767 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.98526502, 2024-09-10T10:28:46.767 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056918610000000003, 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779790+0000", 2024-09-10T10:28:46.768 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779791+0000", 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7000000000000004e-07 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779796+0000", 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7979999999999996e-06 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.769 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779809+0000", 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3091999999999999e-05 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.779826+0000", 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7306999999999999e-05 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780648+0000", 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00082201500000000005 2024-09-10T10:28:46.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780731+0000", 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2927000000000001e-05 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780887+0000", 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015630400000000001 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782402+0000", 2024-09-10T10:28:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015144360000000001 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785382+0000", 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0029802679999999999 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785437+0000", 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5089000000000001e-05 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785482+0000", 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4855000000000003e-05 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.98493452, 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057425619999999997, 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780121+0000", 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8899999999999998e-07 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780126+0000", 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7840000000000003e-06 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780149+0000", 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3078000000000002e-05 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780157+0000", 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5839999999999997e-06 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781837+0000", 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001680436 2024-09-10T10:28:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.781953+0000", 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000115319 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 3", 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782082+0000", 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012904999999999999 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.784217+0000", 2024-09-10T10:28:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0021354099999999999 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785739+0000", 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015221779999999999 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785802+0000", 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2898000000000007e-05 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.785863+0000", 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1036000000000003e-05 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 119.984887977, 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059441549999999996, 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780167+0000", 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780168+0000", 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4199999999999995e-07 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780171+0000", 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0350000000000002e-06 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780180+0000", 2024-09-10T10:28:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.7239999999999998e-06 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.780185+0000", 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782448+0000", 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022630430000000002 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.782566+0000", 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011801199999999999 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:46.786111+0000", 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035452719999999999 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.893445764999996, 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059567530000000004, 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871609+0000", 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871610+0000", 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.0699999999999996e-07 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871626+0000", 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.5729000000000001e-05 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871665+0000", 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9146000000000001e-05 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871672+0000", 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9229999999999996e-06 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873081+0000", 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014088779999999999 2024-09-10T10:28:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873159+0000", 2024-09-10T10:28:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8267000000000001e-05 2024-09-10T10:28:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873291+0000", 2024-09-10T10:28:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013130600000000001 2024-09-10T10:28:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.876148+0000", 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028573930000000002 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877471+0000", 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00132314 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877516+0000", 2024-09-10T10:28:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4941999999999998e-05 2024-09-10T10:28:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877566+0000", 2024-09-10T10:28:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0121999999999999e-05 2024-09-10T10:28:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.893364422000005, 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057519279999999999, 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871691+0000", 2024-09-10T10:28:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5799999999999999e-07 2024-09-10T10:28:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871696+0000", 2024-09-10T10:28:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5090000000000004e-06 2024-09-10T10:28:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871703+0000", 2024-09-10T10:28:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2289999999999997e-06 2024-09-10T10:28:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871709+0000", 2024-09-10T10:28:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7540000000000003e-06 2024-09-10T10:28:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872412+0000", 2024-09-10T10:28:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070355099999999996 2024-09-10T10:28:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872511+0000", 2024-09-10T10:28:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.9000999999999996e-05 2024-09-10T10:28:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872615+0000", 2024-09-10T10:28:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000103546 2024-09-10T10:28:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875747+0000", 2024-09-10T10:28:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031318019999999999 2024-09-10T10:28:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877353+0000", 2024-09-10T10:28:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0016058400000000001 2024-09-10T10:28:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877404+0000", 2024-09-10T10:28:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.1520000000000001e-05 2024-09-10T10:28:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877443+0000", 2024-09-10T10:28:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8518e-05 2024-09-10T10:28:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.893337867, 2024-09-10T10:28:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063133520000000004, 2024-09-10T10:28:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871717+0000", 2024-09-10T10:28:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871718+0000", 2024-09-10T10:28:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4200000000000001e-07 2024-09-10T10:28:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871721+0000", 2024-09-10T10:28:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0730000000000001e-06 2024-09-10T10:28:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871726+0000", 2024-09-10T10:28:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6069999999999998e-06 2024-09-10T10:28:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871744+0000", 2024-09-10T10:28:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7574000000000001e-05 2024-09-10T10:28:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872882+0000", 2024-09-10T10:28:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001137898 2024-09-10T10:28:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.872960+0000", 2024-09-10T10:28:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8231000000000004e-05 2024-09-10T10:28:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873095+0000", 2024-09-10T10:28:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013443799999999999 2024-09-10T10:28:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877450+0000", 2024-09-10T10:28:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0043559489999999996 2024-09-10T10:28:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877945+0000", 2024-09-10T10:28:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049405100000000002 2024-09-10T10:28:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.877999+0000", 2024-09-10T10:28:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4880999999999999e-05 2024-09-10T10:28:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878031+0000", 2024-09-10T10:28:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1207999999999997e-05 2024-09-10T10:28:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.1.9:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.893195218000002, 2024-09-10T10:28:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.011097221000000001, 2024-09-10T10:28:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871860+0000", 2024-09-10T10:28:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8400000000000005e-07 2024-09-10T10:28:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871863+0000", 2024-09-10T10:28:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0630000000000002e-06 2024-09-10T10:28:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871870+0000", 2024-09-10T10:28:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.917e-06 2024-09-10T10:28:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.871875+0000", 2024-09-10T10:28:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.318e-06 2024-09-10T10:28:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873311+0000", 2024-09-10T10:28:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014363970000000001 2024-09-10T10:28:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873470+0000", 2024-09-10T10:28:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015928200000000001 2024-09-10T10:28:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.873592+0000", 2024-09-10T10:28:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012160299999999999 2024-09-10T10:28:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.878096+0000", 2024-09-10T10:28:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0045037000000000002 2024-09-10T10:28:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882869+0000", 2024-09-10T10:28:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0047728689999999999 2024-09-10T10:28:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882918+0000", 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8968000000000003e-05 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882957+0000", 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9619999999999997e-05 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:13 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'1, mlcod=0'0)", 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.890548284999994, 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0084139550000000007, 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874507+0000", 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874508+0000", 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0899999999999999e-06 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874515+0000", 2024-09-10T10:28:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2860000000000002e-06 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874523+0000", 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8790000000000003e-06 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874529+0000", 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5099999999999998e-06 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874578+0000", 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9635000000000002e-05 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.874595+0000", 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6549e-05 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882877+0000", 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0082817129999999996 2024-09-10T10:28:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882921+0000", 2024-09-10T10:28:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4292999999999998e-05 2024-09-10T10:28:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:14 2.16 e26/18 2:6f2c300e:::60f.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 97.889697545000004, 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0076114889999999999, 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875358+0000", 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9199999999999999e-07 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875367+0000", 2024-09-10T10:28:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2509999999999994e-06 2024-09-10T10:28:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875383+0000", 2024-09-10T10:28:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6429000000000001e-05 2024-09-10T10:28:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875391+0000", 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5850000000000002e-06 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875505+0000", 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011455600000000001 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875531+0000", 2024-09-10T10:28:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780999999999999e-05 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882924+0000", 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073930970000000004 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882969+0000", 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4898000000000003e-05 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:21 2.1a 2:5fd68f60:::61d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 92.817545862000003, 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057005240000000002, 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947509+0000", 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947511+0000", 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.725e-06 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947519+0000", 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947531+0000", 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.188e-05 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947539+0000", 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9549999999999992e-06 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947736+0000", 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019690500000000001 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947872+0000", 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013606299999999999 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948035+0000", 2024-09-10T10:28:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00016296099999999999 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950602+0000", 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671209999999999 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953054+0000", 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0024522929999999999 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953161+0000", 2024-09-10T10:28:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00010733199999999999 2024-09-10T10:28:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953210+0000", 2024-09-10T10:28:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8439000000000002e-05 2024-09-10T10:28:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 92.817496726000002, 2024-09-10T10:28:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057603039999999999, 2024-09-10T10:28:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947558+0000", 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947559+0000", 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1200000000000003e-07 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947564+0000", 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4680000000000003e-06 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947577+0000", 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.3539e-05 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947583+0000", 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1279999999999996e-06 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948324+0000", 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00074053499999999996 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948404+0000", 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0568999999999994e-05 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948901+0000", 2024-09-10T10:28:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049628599999999997 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.950652+0000", 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017517749999999999 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953219+0000", 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0025671420000000001 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953262+0000", 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2432000000000003e-05 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953319+0000", 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.6817999999999997e-05 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.2.13:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 92.817385247999994, 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0056767420000000002, 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "started", 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947670+0000", 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947671+0000", 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6799999999999999e-07 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947674+0000", 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.5190000000000001e-06 2024-09-10T10:28:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947683+0000", 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.115e-06 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.947690+0000", 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1280000000000002e-06 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948332+0000", 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00064163800000000004 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.948490+0000", 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000158276 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:13.953347+0000", 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0048562980000000002 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.843984728999999, 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057345540000000002, 2024-09-10T10:28:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921070+0000", 2024-09-10T10:28:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921071+0000", 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9500000000000002e-07 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921078+0000", 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.126e-06 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921085+0000", 2024-09-10T10:28:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1640000000000004e-06 2024-09-10T10:28:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921091+0000", 2024-09-10T10:28:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.4269999999999999e-06 2024-09-10T10:28:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922160+0000", 2024-09-10T10:28:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0010690789999999999 2024-09-10T10:28:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922267+0000", 2024-09-10T10:28:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000107099 2024-09-10T10:28:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922375+0000", 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000108435 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925751+0000", 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0033757560000000002 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926711+0000", 2024-09-10T10:28:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095940700000000001 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926764+0000", 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3797e-05 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926805+0000", 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0669e-05 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.843954557999993, 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0063690489999999999, 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921101+0000", 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921105+0000", 2024-09-10T10:28:46.861 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8789999999999996e-06 2024-09-10T10:28:46.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.861 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.861 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921112+0000", 2024-09-10T10:28:46.861 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9619999999999996e-06 2024-09-10T10:28:46.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.861 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921116+0000", 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1139999999999999e-06 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922635+0000", 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001519079 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922853+0000", 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00021737300000000001 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922980+0000", 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012730599999999999 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925757+0000", 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027771990000000002 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927363+0000", 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001605701 2024-09-10T10:28:46.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927427+0000", 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3925999999999997e-05 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927470+0000", 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2956e-05 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.864 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.843898065999994, 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059244370000000003, 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921157+0000", 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921158+0000", 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9599999999999999e-07 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921161+0000", 2024-09-10T10:28:46.866 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4640000000000002e-06 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921168+0000", 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9709999999999998e-06 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921177+0000", 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.507e-06 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.867 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921719+0000", 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00054224799999999995 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921812+0000", 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.3301999999999995e-05 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 4", 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921956+0000", 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014418100000000001 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925143+0000", 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031869419999999999 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.926950+0000", 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001807033 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927010+0000", 2024-09-10T10:28:46.869 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9883000000000002e-05 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927082+0000", 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1409999999999996e-05 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.3.15:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 86.843796897999994, 2024-09-10T10:28:46.870 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059968590000000002, 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921258+0000", 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921259+0000", 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8800000000000002e-07 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921264+0000", 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7539999999999997e-06 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921271+0000", 2024-09-10T10:28:46.872 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.046e-06 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921276+0000", 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5430000000000001e-06 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.921977+0000", 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00070080299999999995 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922097+0000", 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000120149 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.922224+0000", 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000127353 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.925766+0000", 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035415440000000002 2024-09-10T10:28:46.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927154+0000", 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001387611 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927210+0000", 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5955000000000002e-05 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.875 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:19.927255+0000", 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5513000000000003e-05 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.823529801000007, 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069992550000000002, 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.876 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941525+0000", 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.877 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941526+0000", 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5600000000000005e-07 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941530+0000", 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7869999999999999e-06 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941537+0000", 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5399999999999998e-06 2024-09-10T10:28:46.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941544+0000", 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.6510000000000004e-06 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942241+0000", 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069658800000000005 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942326+0000", 2024-09-10T10:28:46.879 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5610000000000002e-05 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 7", 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942459+0000", 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000133065 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.943795+0000", 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0013361460000000001 2024-09-10T10:28:46.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948457+0000", 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0046617120000000001 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948494+0000", 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7319e-05 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.881 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.948525+0000", 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0181000000000001e-05 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:7 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'3, mlcod=26'1)", 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.819464581999995, 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007759307, 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945591+0000", 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.883 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945592+0000", 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.1149999999999999e-06 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945601+0000", 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.7440000000000002e-06 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945621+0000", 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9650999999999999e-05 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.884 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.2199999999999998e-06 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945691+0000", 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0544000000000002e-05 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9267999999999999e-05 2024-09-10T10:28:46.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953285+0000", 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075750820000000003 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953350+0000", 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.4683000000000004e-05 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:22 2.1d e28/18 2:be329c1b:::631.00000000:head v 28'11, mlcod=26'6)", 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.819446712000001, 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077122759999999997, 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945608+0000", 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945609+0000", 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7899999999999998e-07 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945615+0000", 2024-09-10T10:28:46.888 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5389999999999998e-06 2024-09-10T10:28:46.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945622+0000", 2024-09-10T10:28:46.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2559999999999996e-06 2024-09-10T10:28:46.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.889 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.889 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945630+0000", 2024-09-10T10:28:46.889 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5720000000000003e-06 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945690+0000", 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9908999999999998e-05 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945710+0000", 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.927e-05 2024-09-10T10:28:46.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953260+0000", 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0075506039999999998 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953321+0000", 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0547000000000002e-05 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.4.17:8 2.16 e28/18 2:6b7c230a:::62a.00000000:head v 28'4, mlcod=26'1)", 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 80.819400474000005, 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.007724369, 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945655+0000", 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1600000000000002e-07 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945660+0000", 2024-09-10T10:28:46.893 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0429999999999998e-06 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945667+0000", 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2409999999999998e-06 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945673+0000", 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5149999999999997e-06 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:46.894 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946014+0000", 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00034085400000000003 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.946034+0000", 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.9820999999999999e-05 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953353+0000", 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0073191289999999997 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:46.895 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:46.896 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953379+0000", 2024-09-10T10:28:46.896 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.635e-05 2024-09-10T10:28:46.896 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.896 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.896 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.896 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:46.896 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:46.896 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:46.897 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-10T10:28:47.000 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:47.001 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:47.001 INFO:teuthology.orchestra.run.smithi134.stdout: "num_ops": 0 2024-09-10T10:28:47.001 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:47.009 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-10T10:28:47.113 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:47.113 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [], 2024-09-10T10:28:47.113 INFO:teuthology.orchestra.run.smithi134.stdout: "complaint_time": 180, 2024-09-10T10:28:47.114 INFO:teuthology.orchestra.run.smithi134.stdout: "num_blocked_ops": 0 2024-09-10T10:28:47.114 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:47.122 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-10T10:28:47.227 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-10T10:28:47.227 INFO:teuthology.orchestra.run.smithi134.stdout: "size": 20, 2024-09-10T10:28:47.227 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 600, 2024-09-10T10:28:47.227 INFO:teuthology.orchestra.run.smithi134.stdout: "ops": [ 2024-09-10T10:28:47.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.227 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.227 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:47.227 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 135.12844430000001, 2024-09-10T10:28:47.227 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.17492211699999999, 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 1 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:47.228 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096316+0000", 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096318+0000", 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.657e-06 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.229 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096326+0000", 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.0560000000000005e-06 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096339+0000", 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.2965999999999999e-05 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096349+0000", 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.0784e-05 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096432+0000", 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.2657999999999994e-05 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op must wait for map", 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096441+0000", 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.4590000000000004e-06 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096935+0000", 2024-09-10T10:28:47.231 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00049331899999999998 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for peered", 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.096944+0000", 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.9539999999999993e-06 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267505+0000", 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.170561622 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.232 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267700+0000", 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00019420400000000001 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.267923+0000", 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000223451 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.270575+0000", 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0026522030000000001 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271131+0000", 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00055599000000000002 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271191+0000", 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9947000000000001e-05 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.271238+0000", 2024-09-10T10:28:47.234 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6847000000000001e-05 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.739610056, 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078035889999999997, 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.235 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 115 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.236 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485150+0000", 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1300000000000001e-07 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485209+0000", 2024-09-10T10:28:47.237 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8628999999999997e-05 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485214+0000", 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0849999999999996e-06 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485218+0000", 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3840000000000001e-06 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.238 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490185+0000", 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0049675270000000002 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490231+0000", 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5488000000000002e-05 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490307+0000", 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.6038000000000006e-05 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.239 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491263+0000", 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00095625099999999998 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492908+0000", 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001645355 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492934+0000", 2024-09-10T10:28:47.240 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5431000000000001e-05 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492954+0000", 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0088000000000001e-05 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:47.241 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.73953556699999, 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0077741099999999999, 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 116 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485225+0000", 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1100000000000002e-07 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485227+0000", 2024-09-10T10:28:47.243 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1890000000000001e-06 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485231+0000", 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4010000000000001e-06 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485233+0000", 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.728e-06 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.244 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490628+0000", 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0053952109999999996 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490674+0000", 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5946999999999999e-05 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490750+0000", 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5227000000000003e-05 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.245 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491443+0000", 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00069346800000000004 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492959+0000", 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0015154470000000001 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492983+0000", 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.406e-05 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.492999+0000", 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6121e-05 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.73952343900001, 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0078059490000000004, 2024-09-10T10:28:47.247 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 117 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485237+0000", 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.23e-07 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485240+0000", 2024-09-10T10:28:47.249 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5780000000000001e-06 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485243+0000", 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8430000000000003e-06 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.485247+0000", 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2279999999999998e-06 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.250 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.490995+0000", 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0057480730000000002 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491043+0000", 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7936000000000003e-05 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491113+0000", 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0872999999999997e-05 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.491636+0000", 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00052242300000000005 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493002+0000", 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001365808 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.252 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493025+0000", 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2932999999999999e-05 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.493043+0000", 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8031000000000001e-05 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:47.253 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.407168297, 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095639920000000003, 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 131 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.254 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817592+0000", 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1199999999999998e-07 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.255 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817663+0000", 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1099000000000005e-05 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817670+0000", 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.8539999999999999e-06 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817674+0000", 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.574e-06 2024-09-10T10:28:47.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818465+0000", 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000791445 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818509+0000", 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3569000000000002e-05 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818580+0000", 2024-09-10T10:28:47.257 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1503999999999998e-05 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821292+0000", 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027120529999999999 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827108+0000", 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0058160759999999999 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.258 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827134+0000", 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.5859000000000001e-05 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827156+0000", 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1546999999999999e-05 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.259 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.40707908900001, 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095198769999999995, 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 132 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.260 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817681+0000", 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.2899999999999999e-07 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817683+0000", 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0379999999999998e-06 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817690+0000", 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.3110000000000002e-06 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817699+0000", 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8999999999999995e-06 2024-09-10T10:28:47.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820828+0000", 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031296829999999999 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820873+0000", 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.4869e-05 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.820943+0000", 2024-09-10T10:28:47.263 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9919999999999995e-05 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823683+0000", 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027395140000000002 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827161+0000", 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003477857 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827183+0000", 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.2612999999999998e-05 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827201+0000", 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7842999999999998e-05 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.265 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.40701178200001, 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0094944009999999995, 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 133 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.266 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817748+0000", 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817749+0000", 2024-09-10T10:28:47.267 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2399999999999998e-07 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817752+0000", 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.1970000000000001e-06 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817760+0000", 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8080000000000005e-06 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.268 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817764+0000", 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9419999999999997e-06 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821319+0000", 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0035547740000000001 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821364+0000", 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5398000000000001e-05 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821434+0000", 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0382999999999993e-05 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823717+0000", 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0022825089999999998 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827205+0000", 2024-09-10T10:28:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0034879009999999998 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827225+0000", 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0611999999999998e-05 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827243+0000", 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7353000000000001e-05 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.271 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.406958941, 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095863660000000007, 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 134 2024-09-10T10:28:47.272 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817801+0000", 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.273 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817802+0000", 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.8999999999999996e-07 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817871+0000", 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9572000000000004e-05 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817878+0000", 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2079999999999997e-06 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817882+0000", 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5889999999999996e-06 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821625+0000", 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0037431069999999999 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821669+0000", 2024-09-10T10:28:47.275 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3455000000000003e-05 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.821740+0000", 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1390999999999995e-05 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823826+0000", 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0020863399999999999 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.276 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827343+0000", 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.003516267 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827369+0000", 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.6044999999999999e-05 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827388+0000", 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8802000000000001e-05 2024-09-10T10:28:47.277 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.40686624599999, 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095738179999999996, 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.278 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 135 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:47.279 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817894+0000", 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5999999999999999e-07 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817897+0000", 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.0149999999999999e-06 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.280 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817904+0000", 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5089999999999999e-06 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817909+0000", 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3120000000000001e-06 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823862+0000", 2024-09-10T10:28:47.281 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0059524199999999999 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823911+0000", 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.9323000000000002e-05 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.823983+0000", 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1736000000000006e-05 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.282 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827101+0000", 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031181009999999999 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827392+0000", 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00029090700000000003 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827426+0000", 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4127e-05 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.283 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827468+0000", 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.1907999999999999e-05 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.40683923, 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.009594488, 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.284 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 136 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817921+0000", 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3700000000000001e-07 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817925+0000", 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.4659999999999999e-06 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.286 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817931+0000", 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.5710000000000003e-06 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817936+0000", 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7550000000000002e-06 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824178+0000", 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062415700000000001 2024-09-10T10:28:47.287 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824225+0000", 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6966999999999999e-05 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824296+0000", 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.1284000000000004e-05 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827290+0000", 2024-09-10T10:28:47.288 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.002993697 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827473+0000", 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018376699999999999 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827497+0000", 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3445e-05 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.289 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827515+0000", 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8529000000000002e-05 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.40680754100001, 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0096066220000000004, 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.290 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 137 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.291 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.817953+0000", 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3000000000000001e-07 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818004+0000", 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.0902000000000002e-05 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.292 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818013+0000", 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.5420000000000005e-06 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818019+0000", 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.7939999999999997e-06 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.293 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824489+0000", 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0064692580000000003 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824532+0000", 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3631000000000003e-05 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.824601+0000", 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9028000000000004e-05 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.294 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827334+0000", 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0027325409999999998 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827520+0000", 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00018642499999999999 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827541+0000", 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1151000000000001e-05 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.295 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827559+0000", 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7819999999999999e-05 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.40672821999999, 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010100303, 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.296 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 138 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.297 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818032+0000", 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818033+0000", 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.5400000000000001e-07 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818037+0000", 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.0419999999999996e-06 2024-09-10T10:28:47.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818045+0000", 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9759999999999995e-06 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818052+0000", 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5909999999999998e-06 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.299 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826594+0000", 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0085421660000000003 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826671+0000", 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7121999999999998e-05 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.826795+0000", 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00012315199999999999 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828081+0000", 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0012869139999999999 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828088+0000", 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.9870000000000002e-06 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828112+0000", 2024-09-10T10:28:47.301 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3405e-05 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828132+0000", 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.0394000000000002e-05 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.302 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.40669784900001, 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01125819, 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 139 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:47.303 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818062+0000", 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818063+0000", 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7800000000000002e-07 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.304 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818067+0000", 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8500000000000004e-06 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818075+0000", 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.8499999999999994e-06 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818081+0000", 2024-09-10T10:28:47.305 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.1829999999999999e-06 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827594+0000", 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0095129340000000007 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827669+0000", 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.5239000000000002e-05 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.827789+0000", 2024-09-10T10:28:47.306 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000119757 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829195+0000", 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0014064279999999999 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829232+0000", 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.7058999999999999e-05 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.307 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829281+0000", 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8884000000000002e-05 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829321+0000", 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.9527999999999998e-05 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.308 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.40666378, 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012263886, 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 140 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.309 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818096+0000", 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818097+0000", 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8999999999999997e-07 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.310 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818145+0000", 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.8040000000000001e-05 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818160+0000", 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.4671999999999999e-05 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818169+0000", 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 9.1900000000000001e-06 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.311 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828158+0000", 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0099891440000000001 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828228+0000", 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0407000000000005e-05 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828343+0000", 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000114821 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.312 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829914+0000", 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001570866 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830312+0000", 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00039770499999999999 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830339+0000", 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.69e-05 2024-09-10T10:28:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830360+0000", 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.1651e-05 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.40657653900001, 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.012742035000000001, 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.314 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 141 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.315 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818184+0000", 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7700000000000004e-07 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818188+0000", 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.8659999999999999e-06 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.316 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818196+0000", 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.9480000000000008e-06 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.818205+0000", 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8570000000000001e-06 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828883+0000", 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.010678600999999999 2024-09-10T10:28:47.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.828927+0000", 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.3887999999999997e-05 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.829001+0000", 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.3442000000000006e-05 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830548+0000", 2024-09-10T10:28:47.318 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001547194 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830884+0000", 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00033638100000000002 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830907+0000", 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.3051999999999999e-05 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.319 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.830926+0000", 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8229000000000001e-05 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(client.4206.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 134.39342880199999, 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0097270880000000001, 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.320 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "client_info": { 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "client": "client.4206", 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "client_addr": "172.21.15.120:0/121713597", 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "tid": 145 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831331+0000", 2024-09-10T10:28:47.321 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831332+0000", 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8900000000000001e-07 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831379+0000", 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7134e-05 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.322 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831387+0000", 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7409999999999992e-06 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.831391+0000", 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.6190000000000002e-06 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833155+0000", 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0017641969999999999 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.323 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833237+0000", 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.1957000000000002e-05 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 0", 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.833356+0000", 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00011843100000000001 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.834068+0000", 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00071179000000000001 2024-09-10T10:28:47.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841004+0000", 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0069359749999999996 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841034+0000", 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.042e-05 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:26:32.841058+0000", 2024-09-10T10:28:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.4535000000000001e-05 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:20 2.9 e26/18 2:92e99a1d:::612.00000000:head v 26'2, mlcod=0'0)", 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 98.349232537999995, 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074964979999999999, 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.326 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875528+0000", 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.3499999999999996e-07 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875534+0000", 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9109999999999998e-06 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875543+0000", 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.8319999999999995e-06 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875549+0000", 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.2380000000000002e-06 2024-09-10T10:28:47.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875699+0000", 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00014945900000000001 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875717+0000", 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.8584e-05 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.882982+0000", 2024-09-10T10:28:47.329 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0072648039999999997 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883024+0000", 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2135000000000002e-05 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_repop(mds.1.9:27 2.9 e26/18 2:91bd965c:::501.00000000:head v 26'3, mlcod=0'0)", 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 98.349171904000002, 2024-09-10T10:28:47.330 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0074993129999999996, 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875588+0000", 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875589+0000", 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.3999999999999997e-07 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875595+0000", 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.9680000000000003e-06 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875602+0000", 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.0729999999999999e-06 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875608+0000", 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0430000000000004e-06 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875887+0000", 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000279578 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.875904+0000", 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.6439e-05 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883027+0000", 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0071234480000000001 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:08.883088+0000", 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0424000000000003e-05 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:47.334 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.283092827000004, 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.01193195, 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.335 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941667+0000", 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941668+0000", 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.2399999999999999e-07 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941671+0000", 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 2.8329999999999998e-06 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941678+0000", 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.2919999999999998e-06 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941686+0000", 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 7.7489999999999998e-06 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.337 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944874+0000", 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0031881639999999998 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945305+0000", 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00043084599999999997 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 6", 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.945438+0000", 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00013344600000000001 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950653+0000", 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.005214733 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953496+0000", 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0028433989999999999 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953554+0000", 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.7933000000000003e-05 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.953599+0000", 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.5130999999999999e-05 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: "description": "osd_op(mds.4.17:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e26)", 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: "initiated_at": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: "age": 81.283066880999996, 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0093592050000000006, 2024-09-10T10:28:47.340 INFO:teuthology.orchestra.run.smithi134.stdout: "continuous": false, 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: "type_data": { 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: "events": [ 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "initiated", 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "header_read", 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941693+0000", 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "throttled", 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941694+0000", 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 4.7899999999999999e-07 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "all_read", 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941711+0000", 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 1.7530999999999999e-05 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "dispatched", 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941720+0000", 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 8.4279999999999996e-06 2024-09-10T10:28:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "queued_for_pg", 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.941725+0000", 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 5.2390000000000002e-06 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "reached_pg", 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942690+0000", 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00096541400000000005 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "started", 2024-09-10T10:28:47.343 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942807+0000", 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.000116777 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "waiting for subops from 5", 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.942967+0000", 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.00015997599999999999 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "op_commit", 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.944708+0000", 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.001740492 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "sub_op_commit_rec", 2024-09-10T10:28:47.344 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.950956+0000", 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 0.0062481580000000002 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "commit_sent", 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951016+0000", 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 6.0299e-05 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: "event": "done", 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: "time": "2024-09-10T10:27:25.951052+0000", 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: "duration": 3.6412000000000003e-05 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.345 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.346 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.346 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-10T10:28:47.346 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-10T10:28:47.346 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-10T10:28:48.439 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-10T10:28:48.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.549+0000 7f647535f640 1 Processor -- start 2024-09-10T10:28:48.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.549+0000 7f647535f640 1 -- start start 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f647535f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64700d0bf0 0x7f64700c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f647535f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f64700d0fd0 0x7f64700ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f647535f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f647015edc0 0x7f64700b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f647535f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f6470155430 con 0x7f647015edc0 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f647535f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f647015aea0 con 0x7f64700d0fd0 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f647535f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f647015b180 con 0x7f64700d0bf0 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f646f7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f647015edc0 0x7f64700b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f646f7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f647015edc0 0x7f64700b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57692/0 (socket says 172.21.15.120:57692) 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f646f7fe640 1 -- 172.21.15.120:0/1806057015 learned_addr learned my addr 172.21.15.120:0/1806057015 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f6467fff640 1 --2- 172.21.15.120:0/1806057015 >> v2:172.21.15.134:3300/0 conn(0x7f64700d0fd0 0x7f64700ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f646effd640 1 --2- 172.21.15.120:0/1806057015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64700d0bf0 0x7f64700c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:48.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f646f7fe640 1 -- 172.21.15.120:0/1806057015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64700d0bf0 msgr2=0x7f64700c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:48.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f646f7fe640 1 --2- 172.21.15.120:0/1806057015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64700d0bf0 0x7f64700c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:48.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f646f7fe640 1 -- 172.21.15.120:0/1806057015 >> v2:172.21.15.134:3300/0 conn(0x7f64700d0fd0 msgr2=0x7f64700ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:48.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f646f7fe640 1 --2- 172.21.15.120:0/1806057015 >> v2:172.21.15.134:3300/0 conn(0x7f64700d0fd0 0x7f64700ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:48.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.550+0000 7f646f7fe640 1 -- 172.21.15.120:0/1806057015 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647015b460 con 0x7f647015edc0 2024-09-10T10:28:48.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.551+0000 7f646f7fe640 1 --2- 172.21.15.120:0/1806057015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f647015edc0 0x7f64700b3080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f64700ce010 tx=0x7f646802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681d25b502ba385c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:48.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.551+0000 7f646e7fc640 1 -- 172.21.15.120:0/1806057015 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468036070 con 0x7f647015edc0 2024-09-10T10:28:48.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.551+0000 7f646e7fc640 1 -- 172.21.15.120:0/1806057015 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6468031600 con 0x7f647015edc0 2024-09-10T10:28:48.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.551+0000 7f646e7fc640 1 -- 172.21.15.120:0/1806057015 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64680319d0 con 0x7f647015edc0 2024-09-10T10:28:48.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.552+0000 7f647535f640 1 -- 172.21.15.120:0/1806057015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f647015edc0 msgr2=0x7f64700b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:48.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.552+0000 7f647535f640 1 --2- 172.21.15.120:0/1806057015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f647015edc0 0x7f64700b3080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f64700ce010 tx=0x7f646802a730 comp rx=0 tx=0).stop 2024-09-10T10:28:48.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.552+0000 7f647535f640 1 -- 172.21.15.120:0/1806057015 shutdown_connections 2024-09-10T10:28:48.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.552+0000 7f647535f640 1 --2- 172.21.15.120:0/1806057015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f647015edc0 0x7f64700b3080 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:48.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.552+0000 7f647535f640 1 --2- 172.21.15.120:0/1806057015 >> v2:172.21.15.134:3300/0 conn(0x7f64700d0fd0 0x7f64700ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:48.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.552+0000 7f647535f640 1 --2- 172.21.15.120:0/1806057015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64700d0bf0 0x7f64700c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.552+0000 7f647535f640 1 -- 172.21.15.120:0/1806057015 >> 172.21.15.120:0/1806057015 conn(0x7f647006c900 msgr2=0x7f647006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.552+0000 7f647535f640 1 -- 172.21.15.120:0/1806057015 shutdown_connections 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.552+0000 7f647535f640 1 -- 172.21.15.120:0/1806057015 wait complete. 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.553+0000 7f647535f640 1 Processor -- start 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.553+0000 7f647535f640 1 -- start start 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f647535f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64700d0bf0 0x7f647015e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f647535f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64700d0fd0 0x7f64701f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f647535f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f647015edc0 0x7f6470157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f647535f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f64700b35c0 con 0x7f64700d0fd0 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f647535f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f647015ba20 con 0x7f64700d0bf0 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f647535f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f64700b3740 con 0x7f647015edc0 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f6467fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64700d0fd0 0x7f64701f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f6467fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64700d0fd0 0x7f64701f34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57704/0 (socket says 172.21.15.120:57704) 2024-09-10T10:28:48.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f6467fff640 1 -- 172.21.15.120:0/3059915508 learned_addr learned my addr 172.21.15.120:0/3059915508 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f646effd640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64700d0bf0 0x7f647015e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f6467fff640 1 -- 172.21.15.120:0/3059915508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f647015edc0 msgr2=0x7f6470157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f6467fff640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f647015edc0 0x7f6470157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f6467fff640 1 -- 172.21.15.120:0/3059915508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64700d0bf0 msgr2=0x7f647015e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f6467fff640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64700d0bf0 0x7f647015e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f6467fff640 1 -- 172.21.15.120:0/3059915508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64700cf940 con 0x7f64700d0fd0 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f6467fff640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64700d0fd0 0x7f64701f34a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f645800d800 tx=0x7f645800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f64677fe640 1 -- 172.21.15.120:0/3059915508 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645800b6c0 con 0x7f64700d0fd0 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f64677fe640 1 -- 172.21.15.120:0/3059915508 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6458019070 con 0x7f64700d0fd0 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f647535f640 1 -- 172.21.15.120:0/3059915508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64700cfc00 con 0x7f64700d0fd0 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f64677fe640 1 -- 172.21.15.120:0/3059915508 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645800ba80 con 0x7f64700d0fd0 2024-09-10T10:28:48.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.554+0000 7f647535f640 1 -- 172.21.15.120:0/3059915508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647015d530 con 0x7f64700d0fd0 2024-09-10T10:28:48.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.556+0000 7f647535f640 1 -- 172.21.15.120:0/3059915508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647006ce10 con 0x7f64700d0fd0 2024-09-10T10:28:48.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.556+0000 7f64677fe640 1 -- 172.21.15.120:0/3059915508 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f645800bc20 con 0x7f64700d0fd0 2024-09-10T10:28:48.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.556+0000 7f64677fe640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f645006be30 0x7f645006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:48.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.556+0000 7f64677fe640 1 -- 172.21.15.120:0/3059915508 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6986+0+0 (secure 0 0 0) 0x7f645808b8e0 con 0x7f64700d0fd0 2024-09-10T10:28:48.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.556+0000 7f646effd640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f645006be30 0x7f645006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:48.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.557+0000 7f646effd640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f645006be30 0x7f645006e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f646000a990 tx=0x7f6460008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:48.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.561+0000 7f64677fe640 1 -- 172.21.15.120:0/3059915508 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6458059d00 con 0x7f64700d0fd0 2024-09-10T10:28:48.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:48.734+0000 7f647535f640 1 -- 172.21.15.120:0/3059915508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f64700d0210 con 0x7f64700d0fd0 2024-09-10T10:28:49.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.366+0000 7f64677fe640 1 -- 172.21.15.120:0/3059915508 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v62) ==== 91+0+0 (secure 0 0 0) 0x7f6458059600 con 0x7f64700d0fd0 2024-09-10T10:28:49.367 INFO:teuthology.orchestra.run.smithi120.stderr:noscrub is unset 2024-09-10T10:28:49.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 -- 172.21.15.120:0/3059915508 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f645006be30 msgr2=0x7f645006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f645006be30 0x7f645006e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f646000a990 tx=0x7f6460008040 comp rx=0 tx=0).stop 2024-09-10T10:28:49.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 -- 172.21.15.120:0/3059915508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64700d0fd0 msgr2=0x7f64701f34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64700d0fd0 0x7f64701f34a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f645800d800 tx=0x7f645800dcc0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 -- 172.21.15.120:0/3059915508 shutdown_connections 2024-09-10T10:28:49.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f645006be30 0x7f645006e2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f647015edc0 0x7f6470157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64700d0fd0 0x7f64701f34a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 --2- 172.21.15.120:0/3059915508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64700d0bf0 0x7f647015e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 -- 172.21.15.120:0/3059915508 >> 172.21.15.120:0/3059915508 conn(0x7f647006c900 msgr2=0x7f64700c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:49.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 -- 172.21.15.120:0/3059915508 shutdown_connections 2024-09-10T10:28:49.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.370+0000 7f647535f640 1 -- 172.21.15.120:0/3059915508 wait complete. 2024-09-10T10:28:49.384 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-10T10:28:49.462 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-10T10:28:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.486+0000 7fcb9c2e4640 1 Processor -- start 2024-09-10T10:28:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.486+0000 7fcb9c2e4640 1 -- start start 2024-09-10T10:28:49.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9c2e4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcb94154d50 0x7fcb940c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9c2e4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcb9406bd30 0x7fcb9415e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9c2e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9415a9b0 0x7fcb940c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9c2e4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fcb94155130 con 0x7fcb9406bd30 2024-09-10T10:28:49.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9c2e4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcb9415ee20 con 0x7fcb94154d50 2024-09-10T10:28:49.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9c2e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcb940c9db0 con 0x7fcb9415a9b0 2024-09-10T10:28:49.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9987f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcb9406bd30 0x7fcb9415e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9987f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcb9406bd30 0x7fcb9415e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57708/0 (socket says 172.21.15.120:57708) 2024-09-10T10:28:49.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9987f640 1 -- 172.21.15.120:0/821341871 learned_addr learned my addr 172.21.15.120:0/821341871 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:49.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9a881640 1 --2- 172.21.15.120:0/821341871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9415a9b0 0x7fcb940c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9a080640 1 --2- 172.21.15.120:0/821341871 >> v2:172.21.15.134:3300/0 conn(0x7fcb94154d50 0x7fcb940c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9987f640 1 -- 172.21.15.120:0/821341871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9415a9b0 msgr2=0x7fcb940c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9987f640 1 --2- 172.21.15.120:0/821341871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9415a9b0 0x7fcb940c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9987f640 1 -- 172.21.15.120:0/821341871 >> v2:172.21.15.134:3300/0 conn(0x7fcb94154d50 msgr2=0x7fcb940c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9987f640 1 --2- 172.21.15.120:0/821341871 >> v2:172.21.15.134:3300/0 conn(0x7fcb94154d50 0x7fcb940c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.487+0000 7fcb9987f640 1 -- 172.21.15.120:0/821341871 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb940ca090 con 0x7fcb9406bd30 2024-09-10T10:28:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.488+0000 7fcb9987f640 1 --2- 172.21.15.120:0/821341871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 0x7fcb9415e8e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcb94071240 tx=0x7fcb8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747ad8fc888e319 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.488+0000 7fcb9907e640 1 -- 172.21.15.120:0/821341871 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84037070 con 0x7fcb9406bd30 2024-09-10T10:28:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.488+0000 7fcb9907e640 1 -- 172.21.15.120:0/821341871 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb84032440 con 0x7fcb9406bd30 2024-09-10T10:28:49.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.488+0000 7fcb9907e640 1 -- 172.21.15.120:0/821341871 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84032810 con 0x7fcb9406bd30 2024-09-10T10:28:49.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.488+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/821341871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 msgr2=0x7fcb9415e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.488+0000 7fcb9c2e4640 1 --2- 172.21.15.120:0/821341871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 0x7fcb9415e8e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcb94071240 tx=0x7fcb8402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.489+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/821341871 shutdown_connections 2024-09-10T10:28:49.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.489+0000 7fcb9c2e4640 1 --2- 172.21.15.120:0/821341871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9415a9b0 0x7fcb940c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.489+0000 7fcb9c2e4640 1 --2- 172.21.15.120:0/821341871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 0x7fcb9415e8e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.489+0000 7fcb9c2e4640 1 --2- 172.21.15.120:0/821341871 >> v2:172.21.15.134:3300/0 conn(0x7fcb94154d50 0x7fcb940c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.489+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/821341871 >> 172.21.15.120:0/821341871 conn(0x7fcb94068140 msgr2=0x7fcb94070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.489+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/821341871 shutdown_connections 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.489+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/821341871 wait complete. 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.490+0000 7fcb9c2e4640 1 Processor -- start 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.490+0000 7fcb9c2e4640 1 -- start start 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9c2e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 0x7fcb9414e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9c2e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb94154d50 0x7fcb941594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9c2e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb9415a9b0 0x7fcb941599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9c2e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9415e3c0 con 0x7fcb9406bd30 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9c2e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9415e0e0 con 0x7fcb9415a9b0 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9c2e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcb940c7320 con 0x7fcb94154d50 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9a080640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 0x7fcb9414e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9a080640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 0x7fcb9414e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57714/0 (socket says 172.21.15.120:57714) 2024-09-10T10:28:49.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9a080640 1 -- 172.21.15.120:0/2697110386 learned_addr learned my addr 172.21.15.120:0/2697110386 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:49.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9987f640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb94154d50 0x7fcb941594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9a881640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb9415a9b0 0x7fcb941599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9a080640 1 -- 172.21.15.120:0/2697110386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb94154d50 msgr2=0x7fcb941594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9a080640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb94154d50 0x7fcb941594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9a080640 1 -- 172.21.15.120:0/2697110386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb9415a9b0 msgr2=0x7fcb941599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9a080640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb9415a9b0 0x7fcb941599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9a080640 1 -- 172.21.15.120:0/2697110386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb94158bb0 con 0x7fcb9406bd30 2024-09-10T10:28:49.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9a080640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 0x7fcb9414e3f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000d830 tx=0x7fcb9000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb8affd640 1 -- 172.21.15.120:0/2697110386 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90018020 con 0x7fcb9406bd30 2024-09-10T10:28:49.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb8affd640 1 -- 172.21.15.120:0/2697110386 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb900096a0 con 0x7fcb9406bd30 2024-09-10T10:28:49.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/2697110386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb941f4dc0 con 0x7fcb9406bd30 2024-09-10T10:28:49.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb8affd640 1 -- 172.21.15.120:0/2697110386 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90009a70 con 0x7fcb9406bd30 2024-09-10T10:28:49.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.491+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/2697110386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9414e930 con 0x7fcb9406bd30 2024-09-10T10:28:49.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.492+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/2697110386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9415e8e0 con 0x7fcb9406bd30 2024-09-10T10:28:49.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.493+0000 7fcb8affd640 1 -- 172.21.15.120:0/2697110386 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb9001b020 con 0x7fcb9406bd30 2024-09-10T10:28:49.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.494+0000 7fcb8affd640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcb7806be30 0x7fcb7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.494+0000 7fcb8affd640 1 -- 172.21.15.120:0/2697110386 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6986+0+0 (secure 0 0 0) 0x7fcb9008b3d0 con 0x7fcb9406bd30 2024-09-10T10:28:49.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.494+0000 7fcb9987f640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcb7806be30 0x7fcb7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.494+0000 7fcb9987f640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcb7806be30 0x7fcb7806e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb84034000 tx=0x7fcb8402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.498+0000 7fcb8affd640 1 -- 172.21.15.120:0/2697110386 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcb90022030 con 0x7fcb9406bd30 2024-09-10T10:28:49.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.561+0000 7f7d4a531640 1 Processor -- start 2024-09-10T10:28:49.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.561+0000 7f7d4a531640 1 -- start start 2024-09-10T10:28:49.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d4a531640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441526c0 0x7f7d440c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d4a531640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7d440cbab0 0x7f7d4415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d4a531640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d440c9f70 con 0x7f7d44152aa0 2024-09-10T10:28:49.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d4a531640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7d4415bc50 con 0x7f7d440cbab0 2024-09-10T10:28:49.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d4a531640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d440ca210 con 0x7f7d441526c0 2024-09-10T10:28:49.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d48ace640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7d440cbab0 0x7f7d4415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d48ace640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7d440cbab0 0x7f7d4415c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57720/0 (socket says 172.21.15.120:57720) 2024-09-10T10:28:49.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d48ace640 1 -- 172.21.15.120:0/1745357153 learned_addr learned my addr 172.21.15.120:0/1745357153 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:49.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d43fff640 1 --2- 172.21.15.120:0/1745357153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441526c0 0x7f7d440c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d48ace640 1 -- 172.21.15.120:0/1745357153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441526c0 msgr2=0x7f7d440c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:49.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d48ace640 1 --2- 172.21.15.120:0/1745357153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441526c0 0x7f7d440c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d48ace640 1 -- 172.21.15.120:0/1745357153 >> v1:172.21.15.134:6789/0 conn(0x7f7d44152aa0 legacy=0x7f7d440c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.562+0000 7f7d48ace640 1 -- 172.21.15.120:0/1745357153 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d440ca4d0 con 0x7f7d440cbab0 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.563+0000 7f7d48ace640 1 --2- 172.21.15.120:0/1745357153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cbab0 0x7f7d4415c0e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7d4415bdd0 tx=0x7f7d3c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f708276c99787 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.563+0000 7f7d42ffd640 1 -- 172.21.15.120:0/1745357153 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c036070 con 0x7f7d440cbab0 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.563+0000 7f7d42ffd640 1 -- 172.21.15.120:0/1745357153 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d3c031600 con 0x7f7d440cbab0 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.563+0000 7f7d42ffd640 1 -- 172.21.15.120:0/1745357153 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0319d0 con 0x7f7d440cbab0 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.563+0000 7f7d4a531640 1 -- 172.21.15.120:0/1745357153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cbab0 msgr2=0x7f7d4415c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.563+0000 7f7d4a531640 1 --2- 172.21.15.120:0/1745357153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cbab0 0x7f7d4415c0e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7d4415bdd0 tx=0x7f7d3c02a7a0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.564+0000 7f7d4a531640 1 -- 172.21.15.120:0/1745357153 shutdown_connections 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.564+0000 7f7d4a531640 1 --2- 172.21.15.120:0/1745357153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cbab0 0x7f7d4415c0e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.564+0000 7f7d4a531640 1 --2- 172.21.15.120:0/1745357153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441526c0 0x7f7d440c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.564+0000 7f7d4a531640 1 -- 172.21.15.120:0/1745357153 >> 172.21.15.120:0/1745357153 conn(0x7f7d4406bc10 msgr2=0x7f7d4406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.564+0000 7f7d4a531640 1 -- 172.21.15.120:0/1745357153 shutdown_connections 2024-09-10T10:28:49.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.564+0000 7f7d4a531640 1 -- 172.21.15.120:0/1745357153 wait complete. 2024-09-10T10:28:49.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.565+0000 7f7d4a531640 1 Processor -- start 2024-09-10T10:28:49.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.565+0000 7f7d4a531640 1 -- start start 2024-09-10T10:28:49.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d4a531640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440cbab0 0x7f7d440ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d4a531640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441526c0 0x7f7d4414bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d4a531640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d44152aa0 0x7f7d4414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d4a531640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d440cad70 con 0x7f7d441526c0 2024-09-10T10:28:49.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d4a531640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d440caa90 con 0x7f7d440cbab0 2024-09-10T10:28:49.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d4a531640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d440c44e0 con 0x7f7d44152aa0 2024-09-10T10:28:49.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d437fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441526c0 0x7f7d4414bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d437fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441526c0 0x7f7d4414bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57734/0 (socket says 172.21.15.120:57734) 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d48ace640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d44152aa0 0x7f7d4414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d43fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440cbab0 0x7f7d440ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d437fe640 1 -- 172.21.15.120:0/2967645634 learned_addr learned my addr 172.21.15.120:0/2967645634 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d437fe640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d44152aa0 msgr2=0x7f7d4414c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d437fe640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d44152aa0 0x7f7d4414c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d437fe640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440cbab0 msgr2=0x7f7d440ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d437fe640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440cbab0 0x7f7d440ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d437fe640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d441ef7f0 con 0x7f7d441526c0 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d437fe640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441526c0 0x7f7d4414bce0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c440 tx=0x7f7d3800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d380052d0 con 0x7f7d441526c0 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d38005470 con 0x7f7d441526c0 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d44151e40 con 0x7f7d441526c0 2024-09-10T10:28:49.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38007260 con 0x7f7d441526c0 2024-09-10T10:28:49.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.566+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d441ef510 con 0x7f7d441526c0 2024-09-10T10:28:49.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.567+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7d440c9b90 con 0x7f7d441526c0 2024-09-10T10:28:49.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.569+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d380074c0 con 0x7f7d441526c0 2024-09-10T10:28:49.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.569+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d3006be30 0x7f7d3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.569+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6986+0+0 (secure 0 0 0) 0x7f7d3808b230 con 0x7f7d441526c0 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.569+0000 7f7d43fff640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d3006be30 0x7f7d3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.569+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7f7d3808b7a0 con 0x7f7d441526c0 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.569+0000 7f7d4a531640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cee90 0x7f7d440b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.569+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441ef2c0 con 0x7f7d440cee90 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.569+0000 7f7d43fff640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d3006be30 0x7f7d3006e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7d34006fd0 tx=0x7f7d34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.570+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cee90 0x7f7d440b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.570+0000 7f7d48ace640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7f7d4406ed30 con 0x7f7d440cee90 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.570+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cee90 0x7f7d440b3e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7d440726a0 tx=0x7f7d3c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.570+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c009510 con 0x7f7d440cee90 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.570+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.0 v2:172.21.15.120:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7d3c045430 con 0x7f7d440cee90 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.570+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cee90 msgr2=0x7f7d440b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.570+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cee90 0x7f7d440b3e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7d440726a0 tx=0x7f7d3c004ab0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.570+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cee90 msgr2=0x7f7d440b3e30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:28:49.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.570+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d440cee90 0x7f7d440b3e30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.576+0000 7f7d4a531640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441f0110 0x7f7d441f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.576+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d440c9440 con 0x7f7d441f0110 2024-09-10T10:28:49.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.577+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441f0110 0x7f7d441f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.577+0000 7f7d48ace640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f7d440b4370 con 0x7f7d441f0110 2024-09-10T10:28:49.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.577+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441f0110 0x7f7d441f04f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7d440cb8c0 tx=0x7f7d3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.577+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c009510 con 0x7f7d441f0110 2024-09-10T10:28:49.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.578+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.0 v2:172.21.15.120:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f7d3c009af0 con 0x7f7d441f0110 2024-09-10T10:28:49.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.578+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441f0110 msgr2=0x7f7d441f04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.578+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441f0110 0x7f7d441f04f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7d440cb8c0 tx=0x7f7d3c004a40 comp rx=0 tx=0).stop 2024-09-10T10:28:49.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.578+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441f0110 msgr2=0x7f7d441f04f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:28:49.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.578+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441f0110 0x7f7d441f04f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.580 INFO:teuthology.orchestra.run.smithi120.stdout:mon.a: {} 2024-09-10T10:28:49.581 INFO:teuthology.orchestra.run.smithi120.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T10:28:49.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.579+0000 7f7d4a531640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ccee0 0x7f7d440cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.579+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441500f0 con 0x7f7d440ccee0 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.579+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ccee0 0x7f7d440cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.579+0000 7f7d48ace640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7f7d4414e290 con 0x7f7d440ccee0 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.579+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ccee0 0x7f7d440cd2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c02a7e0 tx=0x7f7d3c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.580+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c04a770 con 0x7f7d440ccee0 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.580+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7d3c04a770 con 0x7f7d440ccee0 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.580+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ccee0 msgr2=0x7f7d440cd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.580+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ccee0 0x7f7d440cd2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c02a7e0 tx=0x7f7d3c004ab0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.580+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ccee0 msgr2=0x7f7d440cd2c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.580+0000 7f7d48ace640 1 -- 172.21.15.120:0/2967645634 reap_dead start 2024-09-10T10:28:49.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.580+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ccee0 0x7f7d440cd2c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.586+0000 7f7d4a531640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ce4b0 0x7f7d4414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.586+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4414dfb0 con 0x7f7d440ce4b0 2024-09-10T10:28:49.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.586+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ce4b0 0x7f7d4414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.586+0000 7f7d48ace640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f7d441f0a30 con 0x7f7d440ce4b0 2024-09-10T10:28:49.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.587+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ce4b0 0x7f7d4414c220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c004a80 tx=0x7f7d3c046000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.587+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c064070 con 0x7f7d440ce4b0 2024-09-10T10:28:49.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.588+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f7d3c064770 con 0x7f7d440ce4b0 2024-09-10T10:28:49.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.588+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ce4b0 msgr2=0x7f7d4414c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.588+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ce4b0 0x7f7d4414c220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c004a80 tx=0x7f7d3c046000 comp rx=0 tx=0).stop 2024-09-10T10:28:49.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.588+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ce4b0 msgr2=0x7f7d4414c220 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:28:49.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.588+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d440ce4b0 0x7f7d4414c220 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.590 INFO:teuthology.orchestra.run.smithi120.stdout:mon.b: {} 2024-09-10T10:28:49.591 INFO:teuthology.orchestra.run.smithi120.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T10:28:49.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.588+0000 7f7d4a531640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f1de0 0x7f7d441f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.588+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d44075880 con 0x7f7d441f1de0 2024-09-10T10:28:49.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.589+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f1de0 0x7f7d441f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.589+0000 7f7d48ace640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f7d441efd30 con 0x7f7d441f1de0 2024-09-10T10:28:49.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.589+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f1de0 0x7f7d441f21c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0093e0 tx=0x7f7d3c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.589+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c055430 con 0x7f7d441f1de0 2024-09-10T10:28:49.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.589+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7d3c04ada0 con 0x7f7d441f1de0 2024-09-10T10:28:49.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.589+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f1de0 msgr2=0x7f7d441f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.589+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f1de0 0x7f7d441f21c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0093e0 tx=0x7f7d3c004ab0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.589+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f1de0 msgr2=0x7f7d441f21c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:28:49.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.589+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f1de0 0x7f7d441f21c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.595+0000 7f7d4a531640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f2700 0x7f7d440b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.595+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7d44075880 con 0x7f7d441f2700 2024-09-10T10:28:49.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.595+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f2700 0x7f7d440b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.596+0000 7f7d48ace640 1 -- 172.21.15.120:0/2967645634 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f7d441f2ae0 con 0x7f7d441f2700 2024-09-10T10:28:49.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.596+0000 7f7d48ace640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f2700 0x7f7d440b1bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c02a7e0 tx=0x7f7d3c06b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.596+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c07b250 con 0x7f7d441f2700 2024-09-10T10:28:49.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f7d3c064d90 con 0x7f7d441f2700 2024-09-10T10:28:49.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f2700 msgr2=0x7f7d440b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f2700 0x7f7d440b1bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c02a7e0 tx=0x7f7d3c06b000 comp rx=0 tx=0).stop 2024-09-10T10:28:49.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d40ff9640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f2700 msgr2=0x7f7d440b1bd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T10:28:49.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d40ff9640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d441f2700 0x7f7d440b1bd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.599 INFO:teuthology.orchestra.run.smithi120.stdout:mon.c: {} 2024-09-10T10:28:49.600 INFO:teuthology.orchestra.run.smithi120.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T10:28:49.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d3006be30 msgr2=0x7f7d3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d4a531640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7d3006be30 0x7f7d3006e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7d34006fd0 tx=0x7f7d34008040 comp rx=0 tx=0).stop 2024-09-10T10:28:49.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441526c0 msgr2=0x7f7d4414bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d4a531640 1 --2- 172.21.15.120:0/2967645634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d441526c0 0x7f7d4414bce0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c440 tx=0x7f7d3800c900 comp rx=0 tx=0).stop 2024-09-10T10:28:49.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d48ace640 1 -- 172.21.15.120:0/2967645634 reap_dead start 2024-09-10T10:28:49.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 shutdown_connections 2024-09-10T10:28:49.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 >> 172.21.15.120:0/2967645634 conn(0x7f7d4406bc10 msgr2=0x7f7d441574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:49.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 shutdown_connections 2024-09-10T10:28:49.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.598+0000 7f7d4a531640 1 -- 172.21.15.120:0/2967645634 wait complete. 2024-09-10T10:28:49.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-10T10:28:49.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.681+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/2697110386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fcb941f4f90 con 0x7fcb9406bd30 2024-09-10T10:28:49.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.748+0000 7f7c1b4c9640 1 Processor -- start 2024-09-10T10:28:49.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.748+0000 7f7c1b4c9640 1 -- start start 2024-09-10T10:28:49.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.749+0000 7f7c1b4c9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c14154d10 0x7f7c140c7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.749+0000 7f7c1b4c9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7c141550f0 0x7f7c140cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.749+0000 7f7c1b4c9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c140cb850 con 0x7f7c1415b680 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.749+0000 7f7c1b4c9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7c140cc200 con 0x7f7c141550f0 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.749+0000 7f7c1b4c9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7c1415bab0 con 0x7f7c14154d10 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.749+0000 7f7c18a64640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7c141550f0 0x7f7c140cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.749+0000 7f7c18a64640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7c141550f0 0x7f7c140cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57768/0 (socket says 172.21.15.120:57768) 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.749+0000 7f7c18a64640 1 -- 172.21.15.120:0/3329630974 learned_addr learned my addr 172.21.15.120:0/3329630974 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.750+0000 7f7c19265640 1 --2- 172.21.15.120:0/3329630974 >> v2:172.21.15.134:3300/0 conn(0x7f7c14154d10 0x7f7c140c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.750+0000 7f7c18a64640 1 -- 172.21.15.120:0/3329630974 >> v2:172.21.15.134:3300/0 conn(0x7f7c14154d10 msgr2=0x7f7c140c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.750+0000 7f7c18a64640 1 --2- 172.21.15.120:0/3329630974 >> v2:172.21.15.134:3300/0 conn(0x7f7c14154d10 0x7f7c140c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.750+0000 7f7c18a64640 1 -- 172.21.15.120:0/3329630974 >> v1:172.21.15.134:6789/0 conn(0x7f7c1415b680 legacy=0x7f7c1415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:49.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.750+0000 7f7c18a64640 1 -- 172.21.15.120:0/3329630974 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1415ea00 con 0x7f7c141550f0 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.750+0000 7f7c19265640 1 --2- 172.21.15.120:0/3329630974 >> v2:172.21.15.134:3300/0 conn(0x7f7c14154d10 0x7f7c140c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.750+0000 7f7c18a64640 1 --2- 172.21.15.120:0/3329630974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c141550f0 0x7f7c140cc690 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7c140d12d0 tx=0x7f7c0802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e918570cd5acd44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c07fff640 1 -- 172.21.15.120:0/3329630974 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08037070 con 0x7f7c141550f0 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c07fff640 1 -- 172.21.15.120:0/3329630974 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c08032440 con 0x7f7c141550f0 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c07fff640 1 -- 172.21.15.120:0/3329630974 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08032810 con 0x7f7c141550f0 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/3329630974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c141550f0 msgr2=0x7f7c140cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c1b4c9640 1 --2- 172.21.15.120:0/3329630974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c141550f0 0x7f7c140cc690 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7c140d12d0 tx=0x7f7c0802a2c0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/3329630974 shutdown_connections 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c1b4c9640 1 --2- 172.21.15.120:0/3329630974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c141550f0 0x7f7c140cc690 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c1b4c9640 1 --2- 172.21.15.120:0/3329630974 >> v2:172.21.15.134:3300/0 conn(0x7f7c14154d10 0x7f7c140c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/3329630974 >> 172.21.15.120:0/3329630974 conn(0x7f7c1406cb70 msgr2=0x7f7c1406e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/3329630974 shutdown_connections 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.751+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/3329630974 wait complete. 2024-09-10T10:28:49.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.752+0000 7f7c1b4c9640 1 Processor -- start 2024-09-10T10:28:49.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.752+0000 7f7c1b4c9640 1 -- start start 2024-09-10T10:28:49.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c1b4c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c14154d10 0x7f7c140c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c1b4c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c1415b680 0x7f7c1415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c1b4c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c140c6eb0 0x7f7c141f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c1b4c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140cbbb0 con 0x7f7c14154d10 2024-09-10T10:28:49.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c1b4c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1415efc0 con 0x7f7c140c6eb0 2024-09-10T10:28:49.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c1b4c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7c140cbe90 con 0x7f7c1415b680 2024-09-10T10:28:49.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c19265640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c14154d10 0x7f7c140c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c19265640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c14154d10 0x7f7c140c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57776/0 (socket says 172.21.15.120:57776) 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c19265640 1 -- 172.21.15.120:0/1263093534 learned_addr learned my addr 172.21.15.120:0/1263093534 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c19a66640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c140c6eb0 0x7f7c141f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c18a64640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c1415b680 0x7f7c1415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c19265640 1 -- 172.21.15.120:0/1263093534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c1415b680 msgr2=0x7f7c1415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c19265640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c1415b680 0x7f7c1415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c19265640 1 -- 172.21.15.120:0/1263093534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c140c6eb0 msgr2=0x7f7c141f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c19265640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c140c6eb0 0x7f7c141f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.753+0000 7f7c19265640 1 -- 172.21.15.120:0/1263093534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c140c41a0 con 0x7f7c14154d10 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.754+0000 7f7c19265640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c14154d10 0x7f7c140c8400 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000c7f0 tx=0x7f7c1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.754+0000 7f7c05ffb640 1 -- 172.21.15.120:0/1263093534 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10015070 con 0x7f7c14154d10 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.754+0000 7f7c05ffb640 1 -- 172.21.15.120:0/1263093534 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c10007a30 con 0x7f7c14154d10 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.754+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/1263093534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c140c4460 con 0x7f7c14154d10 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.754+0000 7f7c05ffb640 1 -- 172.21.15.120:0/1263093534 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10007e00 con 0x7f7c14154d10 2024-09-10T10:28:49.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.754+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/1263093534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c141f09d0 con 0x7f7c14154d10 2024-09-10T10:28:49.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.755+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/1263093534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c140cadd0 con 0x7f7c14154d10 2024-09-10T10:28:49.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.755+0000 7f7c05ffb640 1 -- 172.21.15.120:0/1263093534 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c10018070 con 0x7f7c14154d10 2024-09-10T10:28:49.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.756+0000 7f7c05ffb640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bf806be30 0x7f7bf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:49.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.756+0000 7f7c18a64640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bf806be30 0x7f7bf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:49.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.756+0000 7f7c05ffb640 1 -- 172.21.15.120:0/1263093534 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6986+0+0 (secure 0 0 0) 0x7f7c1008b370 con 0x7f7c14154d10 2024-09-10T10:28:49.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.756+0000 7f7c18a64640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bf806be30 0x7f7bf806e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7c08034000 tx=0x7f7c0802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:49.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.760+0000 7f7c05ffb640 1 -- 172.21.15.120:0/1263093534 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7c140cadd0 con 0x7f7c14154d10 2024-09-10T10:28:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:49.941+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/1263093534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7f7c140c7770 con 0x7f7c14154d10 2024-09-10T10:28:50.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.371+0000 7fcb8affd640 1 -- 172.21.15.120:0/2697110386 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v63) ==== 101+0+0 (secure 0 0 0) 0x7fcb900592c0 con 0x7fcb9406bd30 2024-09-10T10:28:50.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.371+0000 7f7c05ffb640 1 -- 172.21.15.120:0/1263093534 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v63) ==== 113+0+0 (secure 0 0 0) 0x7f7c10059790 con 0x7f7c14154d10 2024-09-10T10:28:50.373 INFO:teuthology.orchestra.run.smithi120.stderr:nodeep-scrub is unset 2024-09-10T10:28:50.373 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.0 to 1 (10000) 2024-09-10T10:28:50.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/2697110386 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcb7806be30 msgr2=0x7fcb7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:50.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcb7806be30 0x7fcb7806e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb84034000 tx=0x7fcb8402e000 comp rx=0 tx=0).stop 2024-09-10T10:28:50.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/2697110386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 msgr2=0x7fcb9414e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:50.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 0x7fcb9414e3f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000d830 tx=0x7fcb9000dcf0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/1263093534 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bf806be30 msgr2=0x7f7bf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:50.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7f7c1b4c9640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bf806be30 0x7f7bf806e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7c08034000 tx=0x7f7c0802e000 comp rx=0 tx=0).stop 2024-09-10T10:28:50.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/2697110386 shutdown_connections 2024-09-10T10:28:50.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fcb7806be30 0x7fcb7806e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb9415a9b0 0x7fcb941599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb94154d50 0x7fcb941594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 --2- 172.21.15.120:0/2697110386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb9406bd30 0x7fcb9414e3f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/2697110386 >> 172.21.15.120:0/2697110386 conn(0x7fcb94068140 msgr2=0x7fcb940c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:50.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/1263093534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c14154d10 msgr2=0x7f7c140c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:50.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7f7c1b4c9640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c14154d10 0x7f7c140c8400 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000c7f0 tx=0x7f7c1000cbf0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/1263093534 shutdown_connections 2024-09-10T10:28:50.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7f7c1b4c9640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bf806be30 0x7f7bf806e2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7f7c1b4c9640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c140c6eb0 0x7f7c141f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7f7c1b4c9640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c1415b680 0x7f7c1415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7f7c1b4c9640 1 --2- 172.21.15.120:0/1263093534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c14154d10 0x7f7c140c8400 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/2697110386 shutdown_connections 2024-09-10T10:28:50.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/1263093534 >> 172.21.15.120:0/1263093534 conn(0x7f7c1406cb70 msgr2=0x7f7c140c7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:50.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.374+0000 7fcb9c2e4640 1 -- 172.21.15.120:0/2697110386 wait complete. 2024-09-10T10:28:50.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.375+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/1263093534 shutdown_connections 2024-09-10T10:28:50.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.375+0000 7f7c1b4c9640 1 -- 172.21.15.120:0/1263093534 wait complete. 2024-09-10T10:28:50.389 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-10T10:28:50.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.530+0000 7fd185d86640 1 Processor -- start 2024-09-10T10:28:50.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.530+0000 7fd185d86640 1 -- start start 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd185d86640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd1801528e0 0x7fd1800c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd185d86640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1800c34b0 con 0x7fd180070e00 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd185d86640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1800c3790 con 0x7fd180152cc0 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd185d86640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd1800c3a70 con 0x7fd1801528e0 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd17ffff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fd180070e00 0x7fd180163090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:37466/0 (socket says 172.21.15.120:37466) 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd17ffff640 1 -- 172.21.15.120:0/3800004588 learned_addr learned my addr 172.21.15.120:0/3800004588 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd17f7fe640 1 --2- 172.21.15.120:0/3800004588 >> v2:172.21.15.134:3300/0 conn(0x7fd1801528e0 0x7fd1800c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd17e7fc640 1 -- 172.21.15.120:0/3800004588 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3568982337 0 0) 0x7fd1800c34b0 con 0x7fd180070e00 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd17e7fc640 1 -- 172.21.15.120:0/3800004588 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd168003520 con 0x7fd180070e00 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd17e7fc640 1 -- 172.21.15.120:0/3800004588 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3688974950 0 0) 0x7fd168003520 con 0x7fd180070e00 2024-09-10T10:28:50.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd17f7fe640 1 -- 172.21.15.120:0/3800004588 >> v1:172.21.15.134:6789/0 conn(0x7fd180152cc0 legacy=0x7fd1800ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:50.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd17f7fe640 1 -- 172.21.15.120:0/3800004588 >> v1:172.21.15.120:6789/0 conn(0x7fd180070e00 legacy=0x7fd180163090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:50.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.531+0000 7fd17f7fe640 1 -- 172.21.15.120:0/3800004588 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18015bd00 con 0x7fd1801528e0 2024-09-10T10:28:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.532+0000 7fd17f7fe640 1 --2- 172.21.15.120:0/3800004588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1801528e0 0x7fd1800c9fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1700097a0 tx=0x7fd17002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ff173e10a19af2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.532+0000 7fd17e7fc640 1 -- 172.21.15.120:0/3800004588 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170037070 con 0x7fd1801528e0 2024-09-10T10:28:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.532+0000 7fd17e7fc640 1 -- 172.21.15.120:0/3800004588 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17002ace0 con 0x7fd1801528e0 2024-09-10T10:28:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.532+0000 7fd17e7fc640 1 -- 172.21.15.120:0/3800004588 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1780019f0 con 0x7fd1801528e0 2024-09-10T10:28:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.533+0000 7fd185d86640 1 -- 172.21.15.120:0/3800004588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1801528e0 msgr2=0x7fd1800c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.533+0000 7fd185d86640 1 --2- 172.21.15.120:0/3800004588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1801528e0 0x7fd1800c9fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1700097a0 tx=0x7fd17002a250 comp rx=0 tx=0).stop 2024-09-10T10:28:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.533+0000 7fd185d86640 1 -- 172.21.15.120:0/3800004588 shutdown_connections 2024-09-10T10:28:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.533+0000 7fd185d86640 1 --2- 172.21.15.120:0/3800004588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1801528e0 0x7fd1800c9fc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.533+0000 7fd185d86640 1 -- 172.21.15.120:0/3800004588 >> 172.21.15.120:0/3800004588 conn(0x7fd18006c240 msgr2=0x7fd180067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.533+0000 7fd185d86640 1 -- 172.21.15.120:0/3800004588 shutdown_connections 2024-09-10T10:28:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.533+0000 7fd185d86640 1 -- 172.21.15.120:0/3800004588 wait complete. 2024-09-10T10:28:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.534+0000 7fd185d86640 1 Processor -- start 2024-09-10T10:28:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.534+0000 7fd185d86640 1 -- start start 2024-09-10T10:28:50.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd185d86640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd180070e00 0x7fd18014c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:50.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd185d86640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1801528e0 0x7fd18014d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd185d86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd180152cc0 0x7fd18014d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd185d86640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1800cea90 con 0x7fd180152cc0 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd185d86640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd18015c2c0 con 0x7fd180070e00 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd185d86640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd1800cec10 con 0x7fd1801528e0 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd180152cc0 0x7fd18014d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd180152cc0 0x7fd18014d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57778/0 (socket says 172.21.15.120:57778) 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17ffff640 1 -- 172.21.15.120:0/77144506 learned_addr learned my addr 172.21.15.120:0/77144506 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17f7fe640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd180070e00 0x7fd18014c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17effd640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1801528e0 0x7fd18014d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17ffff640 1 -- 172.21.15.120:0/77144506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1801528e0 msgr2=0x7fd18014d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17ffff640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1801528e0 0x7fd18014d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17ffff640 1 -- 172.21.15.120:0/77144506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd180070e00 msgr2=0x7fd18014c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17ffff640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd180070e00 0x7fd18014c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17ffff640 1 -- 172.21.15.120:0/77144506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18006ed00 con 0x7fd180152cc0 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17f7fe640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd180070e00 0x7fd18014c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd17ffff640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd180152cc0 0x7fd18014d840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd178001b60 tx=0x7fd17800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd167fff640 1 -- 172.21.15.120:0/77144506 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178017070 con 0x7fd180152cc0 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd167fff640 1 -- 172.21.15.120:0/77144506 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd178012400 con 0x7fd180152cc0 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.535+0000 7fd185d86640 1 -- 172.21.15.120:0/77144506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd180156450 con 0x7fd180152cc0 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.536+0000 7fd167fff640 1 -- 172.21.15.120:0/77144506 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1780127b0 con 0x7fd180152cc0 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.536+0000 7fd185d86640 1 -- 172.21.15.120:0/77144506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd180156a10 con 0x7fd180152cc0 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.536+0000 7fd185d86640 1 -- 172.21.15.120:0/77144506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1800c5190 con 0x7fd180152cc0 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.537+0000 7fd167fff640 1 -- 172.21.15.120:0/77144506 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd17801a070 con 0x7fd180152cc0 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.537+0000 7fd167fff640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd16c06be60 0x7fd16c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.537+0000 7fd167fff640 1 -- 172.21.15.120:0/77144506 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6986+0+0 (secure 0 0 0) 0x7fd17808b230 con 0x7fd180152cc0 2024-09-10T10:28:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.537+0000 7fd17f7fe640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd16c06be60 0x7fd16c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:50.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.538+0000 7fd17f7fe640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd16c06be60 0x7fd16c06e320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd170034000 tx=0x7fd17002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:50.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.541+0000 7fd167fff640 1 -- 172.21.15.120:0/77144506 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd178059000 con 0x7fd180152cc0 2024-09-10T10:28:50.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:50.719+0000 7fd185d86640 1 -- 172.21.15.120:0/77144506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} v 0) -- 0x7fd180070190 con 0x7fd180152cc0 2024-09-10T10:28:51.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.373+0000 7fd167fff640 1 -- 172.21.15.120:0/77144506 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]=0 set osd.0 primary-affinity to 1 (8655362) v64) ==== 133+0+0 (secure 0 0 0) 0x7fd17805d300 con 0x7fd180152cc0 2024-09-10T10:28:51.374 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-10T10:28:51.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.376+0000 7fd185d86640 1 -- 172.21.15.120:0/77144506 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd16c06be60 msgr2=0x7fd16c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:51.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.376+0000 7fd185d86640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd16c06be60 0x7fd16c06e320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd170034000 tx=0x7fd17002e000 comp rx=0 tx=0).stop 2024-09-10T10:28:51.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.376+0000 7fd185d86640 1 -- 172.21.15.120:0/77144506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd180152cc0 msgr2=0x7fd18014d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:51.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.376+0000 7fd185d86640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd180152cc0 0x7fd18014d840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd178001b60 tx=0x7fd17800a430 comp rx=0 tx=0).stop 2024-09-10T10:28:51.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.376+0000 7fd185d86640 1 -- 172.21.15.120:0/77144506 shutdown_connections 2024-09-10T10:28:51.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.376+0000 7fd185d86640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd16c06be60 0x7fd16c06e320 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:51.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.376+0000 7fd185d86640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd180152cc0 0x7fd18014d840 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:51.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.376+0000 7fd185d86640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1801528e0 0x7fd18014d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:51.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.376+0000 7fd185d86640 1 --2- 172.21.15.120:0/77144506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd180070e00 0x7fd18014c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:51.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.376+0000 7fd185d86640 1 -- 172.21.15.120:0/77144506 >> 172.21.15.120:0/77144506 conn(0x7fd18006c240 msgr2=0x7fd18014cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:51.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.377+0000 7fd185d86640 1 -- 172.21.15.120:0/77144506 shutdown_connections 2024-09-10T10:28:51.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.377+0000 7fd185d86640 1 -- 172.21.15.120:0/77144506 wait complete. 2024-09-10T10:28:51.392 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-10T10:28:51.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.493+0000 7f483b39f640 1 Processor -- start 2024-09-10T10:28:51.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.493+0000 7f483b39f640 1 -- start start 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.494+0000 7f483b39f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4834072d20 0x7f48340c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.494+0000 7f483b39f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48340c8050 con 0x7f48340c8d50 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.494+0000 7f483b39f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48340c5780 con 0x7f4834073100 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.494+0000 7f483b39f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f48340c5a60 con 0x7f4834072d20 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.494+0000 7f483893a640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f48340c8d50 0x7f48340c37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:37478/0 (socket says 172.21.15.120:37478) 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.494+0000 7f483893a640 1 -- 172.21.15.120:0/189995151 learned_addr learned my addr 172.21.15.120:0/189995151 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.494+0000 7f483913b640 1 --2- 172.21.15.120:0/189995151 >> v2:172.21.15.134:3300/0 conn(0x7f4834072d20 0x7f48340c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.494+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 465107019 0 0) 0x7f48340c8050 con 0x7f48340c8d50 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.494+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4820003520 con 0x7f48340c8d50 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 583897389 0 0) 0x7f4820003520 con 0x7f48340c8d50 2024-09-10T10:28:51.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 >> v2:172.21.15.134:3300/0 conn(0x7f4834072d20 msgr2=0x7f48340c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 --2- 172.21.15.120:0/189995151 >> v2:172.21.15.134:3300/0 conn(0x7f4834072d20 0x7f48340c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 >> v1:172.21.15.134:6789/0 conn(0x7f4834073100 legacy=0x7f4834151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48340c5d40 con 0x7f48340c8d50 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f4824001670 con 0x7f48340c8d50 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 >> v1:172.21.15.120:6789/0 conn(0x7f48340c8d50 legacy=0x7f48340c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48200039b0 0x7f4820003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4820004800 0x7f4820004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48200051c0 0x7f482000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48340c5a60 con 0x7f4820004800 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f482000c070 con 0x7f48200051c0 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f482000c270 con 0x7f48200039b0 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f483893a640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4820004800 0x7f4820004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f483913b640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48200039b0 0x7f4820003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f483993c640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48200051c0 0x7f482000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:51.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f483893a640 1 -- 172.21.15.120:0/189995151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48200039b0 msgr2=0x7f4820003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f483893a640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48200039b0 0x7f4820003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f483893a640 1 -- 172.21.15.120:0/189995151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48200051c0 msgr2=0x7f482000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f483893a640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48200051c0 0x7f482000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f483893a640 1 -- 172.21.15.120:0/189995151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4820003520 con 0x7f4820004800 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f483993c640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48200051c0 0x7f482000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.495+0000 7f483993c640 1 -- 172.21.15.120:0/189995151 reap_dead start 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.496+0000 7f483893a640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4820004800 0x7f4820004c80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f482400c210 tx=0x7f4820005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d579de0111e469e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.496+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824007d30 con 0x7f4820004800 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.496+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4824005ad0 con 0x7f4820004800 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.496+0000 7f482bfff640 1 -- 172.21.15.120:0/189995151 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824007070 con 0x7f4820004800 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.496+0000 7f483b39f640 1 -- 172.21.15.120:0/189995151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4820004800 msgr2=0x7f4820004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.496+0000 7f483b39f640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4820004800 0x7f4820004c80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f482400c210 tx=0x7f4820005830 comp rx=0 tx=0).stop 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.496+0000 7f483b39f640 1 -- 172.21.15.120:0/189995151 shutdown_connections 2024-09-10T10:28:51.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.496+0000 7f483b39f640 1 --2- 172.21.15.120:0/189995151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4820004800 0x7f4820004c80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:51.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.496+0000 7f483b39f640 1 -- 172.21.15.120:0/189995151 >> 172.21.15.120:0/189995151 conn(0x7f483406e670 msgr2=0x7f483406ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:51.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.497+0000 7f483b39f640 1 -- 172.21.15.120:0/189995151 shutdown_connections 2024-09-10T10:28:51.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.497+0000 7f483b39f640 1 -- 172.21.15.120:0/189995151 wait complete. 2024-09-10T10:28:51.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483b39f640 1 Processor -- start 2024-09-10T10:28:51.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483b39f640 1 -- start start 2024-09-10T10:28:51.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483b39f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4820004800 0x7f4834156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:51.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483b39f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4834164740 0x7f483414d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:51.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483b39f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483414d7c0 0x7f48341f1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:51.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483b39f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48340c72c0 con 0x7f483414d7c0 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483b39f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48340c5480 con 0x7f4820004800 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483b39f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f48340c65e0 con 0x7f4834164740 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483993c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483414d7c0 0x7f48341f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483993c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483414d7c0 0x7f48341f1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57786/0 (socket says 172.21.15.120:57786) 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483993c640 1 -- 172.21.15.120:0/3813154316 learned_addr learned my addr 172.21.15.120:0/3813154316 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.498+0000 7f483913b640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4820004800 0x7f4834156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f483893a640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4834164740 0x7f483414d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f483993c640 1 -- 172.21.15.120:0/3813154316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4834164740 msgr2=0x7f483414d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f483993c640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4834164740 0x7f483414d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f483993c640 1 -- 172.21.15.120:0/3813154316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4820004800 msgr2=0x7f4834156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f483993c640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4820004800 0x7f4834156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f483993c640 1 -- 172.21.15.120:0/3813154316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48341426a0 con 0x7f483414d7c0 2024-09-10T10:28:51.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f483993c640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483414d7c0 0x7f48341f1920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f48340c4610 tx=0x7f482c00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f4829ffb640 1 -- 172.21.15.120:0/3813154316 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c019070 con 0x7f483414d7c0 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f4829ffb640 1 -- 172.21.15.120:0/3813154316 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482c003b80 con 0x7f483414d7c0 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f483b39f640 1 -- 172.21.15.120:0/3813154316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4834142940 con 0x7f483414d7c0 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f4829ffb640 1 -- 172.21.15.120:0/3813154316 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c0050d0 con 0x7f483414d7c0 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.499+0000 7f483b39f640 1 -- 172.21.15.120:0/3813154316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4834142f00 con 0x7f483414d7c0 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.500+0000 7f483b39f640 1 -- 172.21.15.120:0/3813154316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48341433e0 con 0x7f483414d7c0 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.501+0000 7f4829ffb640 1 -- 172.21.15.120:0/3813154316 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f482c01c070 con 0x7f483414d7c0 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.501+0000 7f4829ffb640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f481806be30 0x7f481806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.501+0000 7f483913b640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f481806be30 0x7f481806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.501+0000 7f4829ffb640 1 -- 172.21.15.120:0/3813154316 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6986+0+0 (secure 0 0 0) 0x7f482c08a860 con 0x7f483414d7c0 2024-09-10T10:28:51.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.502+0000 7f483913b640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f481806be30 0x7f481806e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4830009c40 tx=0x7f4830009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:51.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.506+0000 7f4829ffb640 1 -- 172.21.15.120:0/3813154316 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f48341433e0 con 0x7f483414d7c0 2024-09-10T10:28:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:51.683+0000 7f483b39f640 1 -- 172.21.15.120:0/3813154316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7f48340c6cc0 con 0x7f483414d7c0 2024-09-10T10:28:52.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.379+0000 7f4829ffb640 1 -- 172.21.15.120:0/3813154316 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v65) ==== 113+0+0 (secure 0 0 0) 0x7f482c0586c0 con 0x7f483414d7c0 2024-09-10T10:28:52.381 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.1 to 1 (10000) 2024-09-10T10:28:52.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.382+0000 7f483b39f640 1 -- 172.21.15.120:0/3813154316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f481806be30 msgr2=0x7f481806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:52.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.382+0000 7f483b39f640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f481806be30 0x7f481806e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4830009c40 tx=0x7f4830009190 comp rx=0 tx=0).stop 2024-09-10T10:28:52.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.382+0000 7f483b39f640 1 -- 172.21.15.120:0/3813154316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483414d7c0 msgr2=0x7f48341f1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:52.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.382+0000 7f483b39f640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483414d7c0 0x7f48341f1920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f48340c4610 tx=0x7f482c00ead0 comp rx=0 tx=0).stop 2024-09-10T10:28:52.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.383+0000 7f483b39f640 1 -- 172.21.15.120:0/3813154316 shutdown_connections 2024-09-10T10:28:52.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.383+0000 7f483b39f640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f481806be30 0x7f481806e2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:52.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.383+0000 7f483b39f640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483414d7c0 0x7f48341f1920 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:52.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.383+0000 7f483b39f640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4834164740 0x7f483414d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:52.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.383+0000 7f483b39f640 1 --2- 172.21.15.120:0/3813154316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4820004800 0x7f4834156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:52.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.383+0000 7f483b39f640 1 -- 172.21.15.120:0/3813154316 >> 172.21.15.120:0/3813154316 conn(0x7f483406e670 msgr2=0x7f48340c8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:52.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.383+0000 7f483b39f640 1 -- 172.21.15.120:0/3813154316 shutdown_connections 2024-09-10T10:28:52.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.383+0000 7f483b39f640 1 -- 172.21.15.120:0/3813154316 wait complete. 2024-09-10T10:28:52.397 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-10T10:28:52.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.498+0000 7f902a68d640 1 Processor -- start 2024-09-10T10:28:52.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.498+0000 7f902a68d640 1 -- start start 2024-09-10T10:28:52.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.498+0000 7f902a68d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f902406fdb0 0x7f90240c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:52.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.498+0000 7f902a68d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90240c7320 0x7f90240ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:52.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.498+0000 7f902a68d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90240c99b0 con 0x7f9024154be0 2024-09-10T10:28:52.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.498+0000 7f902a68d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f90240c92b0 con 0x7f902406fdb0 2024-09-10T10:28:52.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.498+0000 7f902a68d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f902415aec0 con 0x7f90240c7320 2024-09-10T10:28:52.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.499+0000 7f90237fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f902406fdb0 0x7f90240c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:52.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.499+0000 7f90237fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f902406fdb0 0x7f90240c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57800/0 (socket says 172.21.15.120:57800) 2024-09-10T10:28:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.499+0000 7f90237fe640 1 -- 172.21.15.120:0/3739729594 learned_addr learned my addr 172.21.15.120:0/3739729594 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.499+0000 7f9023fff640 1 --1- 172.21.15.120:0/3739729594 >> v1:172.21.15.120:6789/0 conn(0x7f9024154be0 0x7f902415b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:37484/0 (socket says 172.21.15.120:37484) 2024-09-10T10:28:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.499+0000 7f9028c2a640 1 --2- 172.21.15.120:0/3739729594 >> v2:172.21.15.134:3300/0 conn(0x7f90240c7320 0x7f90240ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.499+0000 7f90237fe640 1 -- 172.21.15.120:0/3739729594 >> v2:172.21.15.134:3300/0 conn(0x7f90240c7320 msgr2=0x7f90240ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.499+0000 7f90237fe640 1 --2- 172.21.15.120:0/3739729594 >> v2:172.21.15.134:3300/0 conn(0x7f90240c7320 0x7f90240ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.499+0000 7f90237fe640 1 -- 172.21.15.120:0/3739729594 >> v1:172.21.15.120:6789/0 conn(0x7f9024154be0 legacy=0x7f902415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.499+0000 7f90237fe640 1 -- 172.21.15.120:0/3739729594 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90240c8cb0 con 0x7f902406fdb0 2024-09-10T10:28:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.499+0000 7f9022ffd640 1 -- 172.21.15.120:0/3739729594 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722327370 0 0) 0x7f90240c99b0 con 0x7f9024154be0 2024-09-10T10:28:52.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f90237fe640 1 --2- 172.21.15.120:0/3739729594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f902406fdb0 0x7f90240c9c30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f902415b040 tx=0x7f901802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72987d5fa83dd9e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f9022ffd640 1 -- 172.21.15.120:0/3739729594 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018037070 con 0x7f902406fdb0 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f9022ffd640 1 -- 172.21.15.120:0/3739729594 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f901802ae50 con 0x7f902406fdb0 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f9022ffd640 1 -- 172.21.15.120:0/3739729594 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018032860 con 0x7f902406fdb0 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f902a68d640 1 -- 172.21.15.120:0/3739729594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f902406fdb0 msgr2=0x7f90240c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f902a68d640 1 --2- 172.21.15.120:0/3739729594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f902406fdb0 0x7f90240c9c30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f902415b040 tx=0x7f901802a300 comp rx=0 tx=0).stop 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f902a68d640 1 -- 172.21.15.120:0/3739729594 shutdown_connections 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f902a68d640 1 --2- 172.21.15.120:0/3739729594 >> v2:172.21.15.134:3300/0 conn(0x7f90240c7320 0x7f90240ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f902a68d640 1 --2- 172.21.15.120:0/3739729594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f902406fdb0 0x7f90240c9c30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f902a68d640 1 -- 172.21.15.120:0/3739729594 >> 172.21.15.120:0/3739729594 conn(0x7f90240680e0 msgr2=0x7f9024070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f902a68d640 1 -- 172.21.15.120:0/3739729594 shutdown_connections 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.500+0000 7f902a68d640 1 -- 172.21.15.120:0/3739729594 wait complete. 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.501+0000 7f902a68d640 1 Processor -- start 2024-09-10T10:28:52.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.502+0000 7f902a68d640 1 -- start start 2024-09-10T10:28:52.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.502+0000 7f902a68d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f902406fdb0 0x7f90240c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:52.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.502+0000 7f902a68d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f90240c7320 0x7f90241589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.502+0000 7f902a68d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9024154be0 0x7f9024158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.502+0000 7f902a68d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9024154fc0 con 0x7f90240c7320 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.502+0000 7f902a68d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f902415ab50 con 0x7f902406fdb0 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.502+0000 7f902a68d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9024155140 con 0x7f9024154be0 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.502+0000 7f90237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f90240c7320 0x7f90241589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f90237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f90240c7320 0x7f90241589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57810/0 (socket says 172.21.15.120:57810) 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f90237fe640 1 -- 172.21.15.120:0/3189039541 learned_addr learned my addr 172.21.15.120:0/3189039541 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f9023fff640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f902406fdb0 0x7f90240c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f9028c2a640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9024154be0 0x7f9024158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f90237fe640 1 -- 172.21.15.120:0/3189039541 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9024154be0 msgr2=0x7f9024158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f90237fe640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9024154be0 0x7f9024158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:52.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f90237fe640 1 -- 172.21.15.120:0/3189039541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f902406fdb0 msgr2=0x7f90240c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:52.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f90237fe640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f902406fdb0 0x7f90240c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:52.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f90237fe640 1 -- 172.21.15.120:0/3189039541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902414b580 con 0x7f90240c7320 2024-09-10T10:28:52.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f9028c2a640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9024154be0 0x7f9024158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:52.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f90237fe640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f90240c7320 0x7f90241589d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f90180090b0 tx=0x7f90180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:52.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f9020ff9640 1 -- 172.21.15.120:0/3189039541 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018037040 con 0x7f90240c7320 2024-09-10T10:28:52.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f9020ff9640 1 -- 172.21.15.120:0/3189039541 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9018004030 con 0x7f90240c7320 2024-09-10T10:28:52.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f9020ff9640 1 -- 172.21.15.120:0/3189039541 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018004400 con 0x7f90240c7320 2024-09-10T10:28:52.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f902a68d640 1 -- 172.21.15.120:0/3189039541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902414b860 con 0x7f90240c7320 2024-09-10T10:28:52.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.503+0000 7f902a68d640 1 -- 172.21.15.120:0/3189039541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f902414be20 con 0x7f90240c7320 2024-09-10T10:28:52.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.505+0000 7f902a68d640 1 -- 172.21.15.120:0/3189039541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f902406c5b0 con 0x7f90240c7320 2024-09-10T10:28:52.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.505+0000 7f9020ff9640 1 -- 172.21.15.120:0/3189039541 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f901803a070 con 0x7f90240c7320 2024-09-10T10:28:52.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.505+0000 7f9020ff9640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f901006be60 0x7f901006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:52.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.505+0000 7f9020ff9640 1 -- 172.21.15.120:0/3189039541 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6986+0+0 (secure 0 0 0) 0x7f90180ab970 con 0x7f90240c7320 2024-09-10T10:28:52.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.505+0000 7f9023fff640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f901006be60 0x7f901006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:52.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.506+0000 7f9023fff640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f901006be60 0x7f901006e320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9014007fd0 tx=0x7f9014007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:52.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.510+0000 7f9020ff9640 1 -- 172.21.15.120:0/3189039541 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9018041020 con 0x7f90240c7320 2024-09-10T10:28:52.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:52.687+0000 7f902a68d640 1 -- 172.21.15.120:0/3189039541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} v 0) -- 0x7f90240c8760 con 0x7f90240c7320 2024-09-10T10:28:53.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.387+0000 7f9020ff9640 1 -- 172.21.15.120:0/3189039541 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]=0 set osd.1 primary-affinity to 1 (8655362) v66) ==== 133+0+0 (secure 0 0 0) 0x7f9018079750 con 0x7f90240c7320 2024-09-10T10:28:53.389 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-10T10:28:53.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 -- 172.21.15.120:0/3189039541 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f901006be60 msgr2=0x7f901006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:53.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f901006be60 0x7f901006e320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9014007fd0 tx=0x7f9014007de0 comp rx=0 tx=0).stop 2024-09-10T10:28:53.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 -- 172.21.15.120:0/3189039541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f90240c7320 msgr2=0x7f90241589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:53.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f90240c7320 0x7f90241589d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f90180090b0 tx=0x7f90180046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:53.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 -- 172.21.15.120:0/3189039541 shutdown_connections 2024-09-10T10:28:53.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f901006be60 0x7f901006e320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:53.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9024154be0 0x7f9024158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:53.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f90240c7320 0x7f90241589d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:53.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 --2- 172.21.15.120:0/3189039541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f902406fdb0 0x7f90240c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:53.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 -- 172.21.15.120:0/3189039541 >> 172.21.15.120:0/3189039541 conn(0x7f90240680e0 msgr2=0x7f90240c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:53.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 -- 172.21.15.120:0/3189039541 shutdown_connections 2024-09-10T10:28:53.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.391+0000 7f902a68d640 1 -- 172.21.15.120:0/3189039541 wait complete. 2024-09-10T10:28:53.406 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-10T10:28:53.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.511+0000 7f1005531640 1 Processor -- start 2024-09-10T10:28:53.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.511+0000 7f1005531640 1 -- start start 2024-09-10T10:28:53.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f1005531640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f10000cc0a0 0x7f10000cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:53.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f1005531640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f100015ba20 con 0x7f100015ed70 2024-09-10T10:28:53.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f1005531640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f100015aea0 con 0x7f10000d0630 2024-09-10T10:28:53.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f1005531640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f100015e1d0 con 0x7f10000cc0a0 2024-09-10T10:28:53.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f0ffeffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f10000cc0a0 0x7f10000cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:53.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f0fff7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f100015ed70 0x7f1000165430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:37496/0 (socket says 172.21.15.120:37496) 2024-09-10T10:28:53.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f0fff7fe640 1 -- 172.21.15.120:0/3130241587 learned_addr learned my addr 172.21.15.120:0/3130241587 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:53.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f0ffeffd640 1 -- 172.21.15.120:0/3130241587 >> v1:172.21.15.134:6789/0 conn(0x7f10000d0630 legacy=0x7f100015e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:53.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f0ffeffd640 1 -- 172.21.15.120:0/3130241587 >> v1:172.21.15.120:6789/0 conn(0x7f100015ed70 legacy=0x7f1000165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:53.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f0ffeffd640 1 -- 172.21.15.120:0/3130241587 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100015b180 con 0x7f10000cc0a0 2024-09-10T10:28:53.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.512+0000 7f0ffe7fc640 1 -- 172.21.15.120:0/3130241587 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033531133 0 0) 0x7f100015ba20 con 0x7f100015ed70 2024-09-10T10:28:53.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.513+0000 7f0ffeffd640 1 --2- 172.21.15.120:0/3130241587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10000cc0a0 0x7f10000cc480 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00097a0 tx=0x7f0ff002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fe5ffcc8a69bae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:53.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.513+0000 7f0ffe7fc640 1 -- 172.21.15.120:0/3130241587 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0037070 con 0x7f10000cc0a0 2024-09-10T10:28:53.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.513+0000 7f0ffe7fc640 1 -- 172.21.15.120:0/3130241587 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff0032440 con 0x7f10000cc0a0 2024-09-10T10:28:53.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.513+0000 7f0ffe7fc640 1 -- 172.21.15.120:0/3130241587 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0032810 con 0x7f10000cc0a0 2024-09-10T10:28:53.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.514+0000 7f1005531640 1 -- 172.21.15.120:0/3130241587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10000cc0a0 msgr2=0x7f10000cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:53.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.514+0000 7f1005531640 1 --2- 172.21.15.120:0/3130241587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10000cc0a0 0x7f10000cc480 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00097a0 tx=0x7f0ff002a270 comp rx=0 tx=0).stop 2024-09-10T10:28:53.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.514+0000 7f1005531640 1 -- 172.21.15.120:0/3130241587 shutdown_connections 2024-09-10T10:28:53.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.514+0000 7f1005531640 1 --2- 172.21.15.120:0/3130241587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10000cc0a0 0x7f10000cc480 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:53.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.514+0000 7f1005531640 1 -- 172.21.15.120:0/3130241587 >> 172.21.15.120:0/3130241587 conn(0x7f100006d560 msgr2=0x7f1000073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:53.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.514+0000 7f1005531640 1 -- 172.21.15.120:0/3130241587 shutdown_connections 2024-09-10T10:28:53.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.514+0000 7f1005531640 1 -- 172.21.15.120:0/3130241587 wait complete. 2024-09-10T10:28:53.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.515+0000 7f1005531640 1 Processor -- start 2024-09-10T10:28:53.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.515+0000 7f1005531640 1 -- start start 2024-09-10T10:28:53.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f1005531640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10000cc0a0 0x7f1000152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:53.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f1005531640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10000d0630 0x7f10001f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:53.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f1005531640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f100015ed70 0x7f10001e6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f1005531640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10000cbde0 con 0x7f100015ed70 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f1005531640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f100015b740 con 0x7f10000cc0a0 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f1005531640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f100015aaf0 con 0x7f10000d0630 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0fff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f100015ed70 0x7f10001e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0fff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f100015ed70 0x7f10001e6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57824/0 (socket says 172.21.15.120:57824) 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0fff7fe640 1 -- 172.21.15.120:0/3394421039 learned_addr learned my addr 172.21.15.120:0/3394421039 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0ffeffd640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10000cc0a0 0x7f1000152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0ff67fc640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10000d0630 0x7f10001f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0fff7fe640 1 -- 172.21.15.120:0/3394421039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10000d0630 msgr2=0x7f10001f15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0fff7fe640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10000d0630 0x7f10001f15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0fff7fe640 1 -- 172.21.15.120:0/3394421039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10000cc0a0 msgr2=0x7f1000152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0fff7fe640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10000cc0a0 0x7f1000152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0fff7fe640 1 -- 172.21.15.120:0/3394421039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10001e4b50 con 0x7f100015ed70 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0ff67fc640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10000d0630 0x7f10001f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:53.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.516+0000 7f0fff7fe640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f100015ed70 0x7f10001e6490 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800c810 tx=0x7f0ff800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:53.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.517+0000 7f0ff7fff640 1 -- 172.21.15.120:0/3394421039 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8018070 con 0x7f100015ed70 2024-09-10T10:28:53.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.517+0000 7f0ff7fff640 1 -- 172.21.15.120:0/3394421039 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff800a5e0 con 0x7f100015ed70 2024-09-10T10:28:53.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.517+0000 7f1005531640 1 -- 172.21.15.120:0/3394421039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10001e69d0 con 0x7f100015ed70 2024-09-10T10:28:53.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.517+0000 7f0ff7fff640 1 -- 172.21.15.120:0/3394421039 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800a9b0 con 0x7f100015ed70 2024-09-10T10:28:53.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.517+0000 7f1005531640 1 -- 172.21.15.120:0/3394421039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100015da10 con 0x7f100015ed70 2024-09-10T10:28:53.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.518+0000 7f1005531640 1 -- 172.21.15.120:0/3394421039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10000c7090 con 0x7f100015ed70 2024-09-10T10:28:53.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.518+0000 7f0ff7fff640 1 -- 172.21.15.120:0/3394421039 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ff801b070 con 0x7f100015ed70 2024-09-10T10:28:53.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.519+0000 7f0ff7fff640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fe006be30 0x7f0fe006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:53.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.519+0000 7f0ff7fff640 1 -- 172.21.15.120:0/3394421039 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6986+0+0 (secure 0 0 0) 0x7f0ff808a850 con 0x7f100015ed70 2024-09-10T10:28:53.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.519+0000 7f0ffeffd640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fe006be30 0x7f0fe006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:53.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.519+0000 7f0ffeffd640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fe006be30 0x7f0fe006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0034000 tx=0x7f0ff002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:53.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.523+0000 7f0ff7fff640 1 -- 172.21.15.120:0/3394421039 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f10000c7090 con 0x7f100015ed70 2024-09-10T10:28:53.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:53.696+0000 7f1005531640 1 -- 172.21.15.120:0/3394421039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7f1000155140 con 0x7f100015ed70 2024-09-10T10:28:54.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.396+0000 7f0ff7fff640 1 -- 172.21.15.120:0/3394421039 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v67) ==== 113+0+0 (secure 0 0 0) 0x7f0ff80586b0 con 0x7f100015ed70 2024-09-10T10:28:54.398 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.2 to 1 (10000) 2024-09-10T10:28:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 -- 172.21.15.120:0/3394421039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fe006be30 msgr2=0x7f0fe006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fe006be30 0x7f0fe006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0034000 tx=0x7f0ff002e000 comp rx=0 tx=0).stop 2024-09-10T10:28:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 -- 172.21.15.120:0/3394421039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f100015ed70 msgr2=0x7f10001e6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f100015ed70 0x7f10001e6490 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800c810 tx=0x7f0ff800ccd0 comp rx=0 tx=0).stop 2024-09-10T10:28:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 -- 172.21.15.120:0/3394421039 shutdown_connections 2024-09-10T10:28:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fe006be30 0x7f0fe006e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f100015ed70 0x7f10001e6490 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10000d0630 0x7f10001f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 --2- 172.21.15.120:0/3394421039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10000cc0a0 0x7f1000152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 -- 172.21.15.120:0/3394421039 >> 172.21.15.120:0/3394421039 conn(0x7f100006d560 msgr2=0x7f10000cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 -- 172.21.15.120:0/3394421039 shutdown_connections 2024-09-10T10:28:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.400+0000 7f1005531640 1 -- 172.21.15.120:0/3394421039 wait complete. 2024-09-10T10:28:54.415 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-10T10:28:54.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.517+0000 7f89b130a640 1 Processor -- start 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.517+0000 7f89b130a640 1 -- start start 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89b130a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f89ac0d0dc0 0x7f89ac1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89b130a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ac0b31e0 con 0x7f89ac15c7a0 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89b130a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ac0b4110 con 0x7f89ac0d09e0 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89b130a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f89ac0b43f0 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89aa7fc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f89ac0d0dc0 0x7f89ac1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89aa7fc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f89ac0d0dc0 0x7f89ac1548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57838/0 (socket says 172.21.15.120:57838) 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89aa7fc640 1 -- 172.21.15.120:0/206485469 learned_addr learned my addr 172.21.15.120:0/206485469 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89ab7fe640 1 --1- 172.21.15.120:0/206485469 >> v1:172.21.15.120:6789/0 conn(0x7f89ac15c7a0 0x7f89ac15adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:37508/0 (socket says 172.21.15.120:37508) 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89aa7fc640 1 -- 172.21.15.120:0/206485469 >> v1:172.21.15.134:6789/0 conn(0x7f89ac0d09e0 legacy=0x7f89ac15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89aa7fc640 1 -- 172.21.15.120:0/206485469 >> v1:172.21.15.120:6789/0 conn(0x7f89ac15c7a0 legacy=0x7f89ac15adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89aa7fc640 1 -- 172.21.15.120:0/206485469 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ac0b46d0 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.518+0000 7f89a9ffb640 1 -- 172.21.15.120:0/206485469 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3504385693 0 0) 0x7f89ac0b31e0 con 0x7f89ac15c7a0 2024-09-10T10:28:54.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.519+0000 7f89aa7fc640 1 --2- 172.21.15.120:0/206485469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 0x7f89ac1548f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009870 tx=0x7f89a002a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3b68903baeea4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.519+0000 7f89a9ffb640 1 -- 172.21.15.120:0/206485469 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0037070 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.519+0000 7f89a9ffb640 1 -- 172.21.15.120:0/206485469 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a0032640 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.519+0000 7f89a9ffb640 1 -- 172.21.15.120:0/206485469 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0032a10 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.520+0000 7f89b130a640 1 -- 172.21.15.120:0/206485469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 msgr2=0x7f89ac1548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.520+0000 7f89b130a640 1 --2- 172.21.15.120:0/206485469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 0x7f89ac1548f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009870 tx=0x7f89a002a330 comp rx=0 tx=0).stop 2024-09-10T10:28:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.520+0000 7f89b130a640 1 -- 172.21.15.120:0/206485469 shutdown_connections 2024-09-10T10:28:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.520+0000 7f89b130a640 1 --2- 172.21.15.120:0/206485469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 0x7f89ac1548f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.520+0000 7f89b130a640 1 -- 172.21.15.120:0/206485469 >> 172.21.15.120:0/206485469 conn(0x7f89ac06c920 msgr2=0x7f89ac06cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.520+0000 7f89b130a640 1 -- 172.21.15.120:0/206485469 shutdown_connections 2024-09-10T10:28:54.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.520+0000 7f89b130a640 1 -- 172.21.15.120:0/206485469 wait complete. 2024-09-10T10:28:54.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.521+0000 7f89b130a640 1 Processor -- start 2024-09-10T10:28:54.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.521+0000 7f89b130a640 1 -- start start 2024-09-10T10:28:54.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89b130a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ac0d09e0 0x7f89ac159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:54.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89b130a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 0x7f89ac1f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:54.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89b130a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac15c7a0 0x7f89ac1f1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:54.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89b130a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac0b4f70 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89b130a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac0b4c90 con 0x7f89ac15c7a0 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89b130a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f89ac154e30 con 0x7f89ac0d09e0 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 0x7f89ac1f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 0x7f89ac1f0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57848/0 (socket says 172.21.15.120:57848) 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89aa7fc640 1 -- 172.21.15.120:0/1618908546 learned_addr learned my addr 172.21.15.120:0/1618908546 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89ab7fe640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac15c7a0 0x7f89ac1f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89aaffd640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ac0d09e0 0x7f89ac159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89aa7fc640 1 -- 172.21.15.120:0/1618908546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ac0d09e0 msgr2=0x7f89ac159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89aa7fc640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ac0d09e0 0x7f89ac159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89aa7fc640 1 -- 172.21.15.120:0/1618908546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac15c7a0 msgr2=0x7f89ac1f1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89aa7fc640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac15c7a0 0x7f89ac1f1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.522+0000 7f89aa7fc640 1 -- 172.21.15.120:0/1618908546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ac1f05e0 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.523+0000 7f89aa7fc640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 0x7f89ac1f0ae0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f89a0000c00 tx=0x7f89a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.523+0000 7f89937fe640 1 -- 172.21.15.120:0/1618908546 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0041070 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.523+0000 7f89937fe640 1 -- 172.21.15.120:0/1618908546 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a003c870 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.523+0000 7f89b130a640 1 -- 172.21.15.120:0/1618908546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ac1f4670 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.523+0000 7f89937fe640 1 -- 172.21.15.120:0/1618908546 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0037040 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.523+0000 7f89b130a640 1 -- 172.21.15.120:0/1618908546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ac0c63c0 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.524+0000 7f89b130a640 1 -- 172.21.15.120:0/1618908546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ac0c5800 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.524+0000 7f89937fe640 1 -- 172.21.15.120:0/1618908546 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89a0044070 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.525+0000 7f89937fe640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f899806be30 0x7f899806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:54.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.525+0000 7f89937fe640 1 -- 172.21.15.120:0/1618908546 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6986+0+0 (secure 0 0 0) 0x7f89a00abdf0 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.525+0000 7f89aaffd640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f899806be30 0x7f899806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:54.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.525+0000 7f89aaffd640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f899806be30 0x7f899806e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f899c004f60 tx=0x7f899c007e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.529+0000 7f89937fe640 1 -- 172.21.15.120:0/1618908546 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f89a0079ce0 con 0x7f89ac0d0dc0 2024-09-10T10:28:54.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:54.705+0000 7f89b130a640 1 -- 172.21.15.120:0/1618908546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} v 0) -- 0x7f89ac1551f0 con 0x7f89ac0d0dc0 2024-09-10T10:28:55.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.407+0000 7f89937fe640 1 -- 172.21.15.120:0/1618908546 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]=0 set osd.2 primary-affinity to 1 (8655362) v68) ==== 133+0+0 (secure 0 0 0) 0x7f89a007dfe0 con 0x7f89ac0d0dc0 2024-09-10T10:28:55.408 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-10T10:28:55.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 -- 172.21.15.120:0/1618908546 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f899806be30 msgr2=0x7f899806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:55.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f899806be30 0x7f899806e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f899c004f60 tx=0x7f899c007e30 comp rx=0 tx=0).stop 2024-09-10T10:28:55.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 -- 172.21.15.120:0/1618908546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 msgr2=0x7f89ac1f0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:55.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 0x7f89ac1f0ae0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f89a0000c00 tx=0x7f89a00046c0 comp rx=0 tx=0).stop 2024-09-10T10:28:55.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 -- 172.21.15.120:0/1618908546 shutdown_connections 2024-09-10T10:28:55.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f899806be30 0x7f899806e2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:55.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac15c7a0 0x7f89ac1f1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:55.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89ac0d0dc0 0x7f89ac1f0ae0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:55.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 --2- 172.21.15.120:0/1618908546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ac0d09e0 0x7f89ac159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:55.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 -- 172.21.15.120:0/1618908546 >> 172.21.15.120:0/1618908546 conn(0x7f89ac06c920 msgr2=0x7f89ac0b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:55.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 -- 172.21.15.120:0/1618908546 shutdown_connections 2024-09-10T10:28:55.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.410+0000 7f89b130a640 1 -- 172.21.15.120:0/1618908546 wait complete. 2024-09-10T10:28:55.425 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-10T10:28:55.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.531+0000 7efd1776d640 1 Processor -- start 2024-09-10T10:28:55.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.531+0000 7efd1776d640 1 -- start start 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.532+0000 7efd1776d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efd100ce9d0 0x7efd1015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.532+0000 7efd1776d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 0x7efd10157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.532+0000 7efd1776d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd100c9a20 con 0x7efd100ce5f0 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.532+0000 7efd1776d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7efd1015ef70 con 0x7efd100ce9d0 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.532+0000 7efd1776d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efd1006cd20 con 0x7efd1006eaa0 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.532+0000 7efd15509640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7efd100ce5f0 0x7efd1015b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:37516/0 (socket says 172.21.15.120:37516) 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.532+0000 7efd15509640 1 -- 172.21.15.120:0/2406400298 learned_addr learned my addr 172.21.15.120:0/2406400298 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.532+0000 7efd14d08640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efd100ce9d0 0x7efd1015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.532+0000 7efd15d0a640 1 --2- 172.21.15.120:0/2406400298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 0x7efd10157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.532+0000 7efd14d08640 1 -- 172.21.15.120:0/2406400298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 msgr2=0x7efd10157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.533+0000 7efd14d08640 1 --2- 172.21.15.120:0/2406400298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 0x7efd10157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.533+0000 7efd14d08640 1 -- 172.21.15.120:0/2406400298 >> v1:172.21.15.120:6789/0 conn(0x7efd100ce5f0 legacy=0x7efd1015b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:55.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.533+0000 7efd07fff640 1 -- 172.21.15.120:0/2406400298 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1212540062 0 0) 0x7efd100c9a20 con 0x7efd100ce5f0 2024-09-10T10:28:55.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.533+0000 7efd14d08640 1 -- 172.21.15.120:0/2406400298 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1006cf70 con 0x7efd100ce9d0 2024-09-10T10:28:55.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.533+0000 7efd14d08640 1 --2- 172.21.15.120:0/2406400298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 0x7efd1015ea30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efd000097a0 tx=0x7efd0002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfaad22110f244a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:55.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.533+0000 7efd07fff640 1 -- 172.21.15.120:0/2406400298 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00037070 con 0x7efd100ce9d0 2024-09-10T10:28:55.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.533+0000 7efd07fff640 1 -- 172.21.15.120:0/2406400298 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd00032440 con 0x7efd100ce9d0 2024-09-10T10:28:55.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.533+0000 7efd07fff640 1 -- 172.21.15.120:0/2406400298 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00032810 con 0x7efd100ce9d0 2024-09-10T10:28:55.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.534+0000 7efd1776d640 1 -- 172.21.15.120:0/2406400298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 msgr2=0x7efd1015ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:55.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.534+0000 7efd1776d640 1 --2- 172.21.15.120:0/2406400298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 0x7efd1015ea30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efd000097a0 tx=0x7efd0002a180 comp rx=0 tx=0).stop 2024-09-10T10:28:55.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.534+0000 7efd1776d640 1 -- 172.21.15.120:0/2406400298 shutdown_connections 2024-09-10T10:28:55.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.534+0000 7efd1776d640 1 --2- 172.21.15.120:0/2406400298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 0x7efd10157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:55.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.534+0000 7efd1776d640 1 --2- 172.21.15.120:0/2406400298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 0x7efd1015ea30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:55.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.534+0000 7efd1776d640 1 -- 172.21.15.120:0/2406400298 >> 172.21.15.120:0/2406400298 conn(0x7efd1006c240 msgr2=0x7efd10067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:55.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.534+0000 7efd1776d640 1 -- 172.21.15.120:0/2406400298 shutdown_connections 2024-09-10T10:28:55.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.534+0000 7efd1776d640 1 -- 172.21.15.120:0/2406400298 wait complete. 2024-09-10T10:28:55.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.535+0000 7efd1776d640 1 Processor -- start 2024-09-10T10:28:55.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.535+0000 7efd1776d640 1 -- start start 2024-09-10T10:28:55.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd1776d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 0x7efd101f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:55.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd1776d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd100ce5f0 0x7efd101e5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:55.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd1776d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 0x7efd1015d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd1776d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd1015b8f0 con 0x7efd100ce9d0 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd1776d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd1015acc0 con 0x7efd100ce5f0 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd1776d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efd1015ba70 con 0x7efd1006eaa0 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd15d0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 0x7efd1015d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd15d0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 0x7efd1015d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57864/0 (socket says 172.21.15.120:57864) 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd15d0a640 1 -- 172.21.15.120:0/890216676 learned_addr learned my addr 172.21.15.120:0/890216676 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd14d08640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd100ce5f0 0x7efd101e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd15509640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 0x7efd101f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd15d0a640 1 -- 172.21.15.120:0/890216676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 msgr2=0x7efd101f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd15d0a640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 0x7efd101f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd15d0a640 1 -- 172.21.15.120:0/890216676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd100ce5f0 msgr2=0x7efd101e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:55.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd15d0a640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd100ce5f0 0x7efd101e5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:55.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd15d0a640 1 -- 172.21.15.120:0/890216676 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd101f48e0 con 0x7efd100ce9d0 2024-09-10T10:28:55.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.536+0000 7efd15509640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 0x7efd101f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:28:55.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.537+0000 7efd15d0a640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 0x7efd1015d310 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efd0800cd20 tx=0x7efd0800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:55.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.537+0000 7efd05ffb640 1 -- 172.21.15.120:0/890216676 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08003b90 con 0x7efd100ce9d0 2024-09-10T10:28:55.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.537+0000 7efd05ffb640 1 -- 172.21.15.120:0/890216676 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd08003d30 con 0x7efd100ce9d0 2024-09-10T10:28:55.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.537+0000 7efd05ffb640 1 -- 172.21.15.120:0/890216676 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08004240 con 0x7efd100ce9d0 2024-09-10T10:28:55.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.537+0000 7efd1776d640 1 -- 172.21.15.120:0/890216676 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd100b2560 con 0x7efd100ce9d0 2024-09-10T10:28:55.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.537+0000 7efd1776d640 1 -- 172.21.15.120:0/890216676 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd101f3e80 con 0x7efd100ce9d0 2024-09-10T10:28:55.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.538+0000 7efd1776d640 1 -- 172.21.15.120:0/890216676 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd100c94e0 con 0x7efd100ce9d0 2024-09-10T10:28:55.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.538+0000 7efd05ffb640 1 -- 172.21.15.120:0/890216676 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd080043e0 con 0x7efd100ce9d0 2024-09-10T10:28:55.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.539+0000 7efd05ffb640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efcf406be30 0x7efcf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:55.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.539+0000 7efd15509640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efcf406be30 0x7efcf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:55.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.539+0000 7efd05ffb640 1 -- 172.21.15.120:0/890216676 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6986+0+0 (secure 0 0 0) 0x7efd0808bd60 con 0x7efd100ce9d0 2024-09-10T10:28:55.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.539+0000 7efd15509640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efcf406be30 0x7efcf406e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd0c000f60 tx=0x7efd0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:55.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.543+0000 7efd05ffb640 1 -- 172.21.15.120:0/890216676 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efd0801d340 con 0x7efd100ce9d0 2024-09-10T10:28:55.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:55.719+0000 7efd1776d640 1 -- 172.21.15.120:0/890216676 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7efd100ca0e0 con 0x7efd100ce9d0 2024-09-10T10:28:56.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.414+0000 7efd05ffb640 1 -- 172.21.15.120:0/890216676 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v69) ==== 113+0+0 (secure 0 0 0) 0x7efd0808b030 con 0x7efd100ce9d0 2024-09-10T10:28:56.415 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.3 to 1 (10000) 2024-09-10T10:28:56.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 -- 172.21.15.120:0/890216676 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efcf406be30 msgr2=0x7efcf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:56.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efcf406be30 0x7efcf406e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd0c000f60 tx=0x7efd0c009040 comp rx=0 tx=0).stop 2024-09-10T10:28:56.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 -- 172.21.15.120:0/890216676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 msgr2=0x7efd1015d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:56.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 0x7efd1015d310 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efd0800cd20 tx=0x7efd0800a660 comp rx=0 tx=0).stop 2024-09-10T10:28:56.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 -- 172.21.15.120:0/890216676 shutdown_connections 2024-09-10T10:28:56.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efcf406be30 0x7efcf406e2f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:56.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd100ce9d0 0x7efd1015d310 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:56.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd100ce5f0 0x7efd101e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:56.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 --2- 172.21.15.120:0/890216676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd1006eaa0 0x7efd101f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:56.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 -- 172.21.15.120:0/890216676 >> 172.21.15.120:0/890216676 conn(0x7efd1006c240 msgr2=0x7efd10071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:56.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 -- 172.21.15.120:0/890216676 shutdown_connections 2024-09-10T10:28:56.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.417+0000 7efd1776d640 1 -- 172.21.15.120:0/890216676 wait complete. 2024-09-10T10:28:56.431 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-10T10:28:56.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.533+0000 7f0d5ce77640 1 Processor -- start 2024-09-10T10:28:56.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.533+0000 7f0d5ce77640 1 -- start start 2024-09-10T10:28:56.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d5ce77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581528e0 0x7f0d580c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:56.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d5ce77640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0d58070e00 0x7f0d5815c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:56.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d5ce77640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d580c34b0 con 0x7f0d58152cc0 2024-09-10T10:28:56.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d5ce77640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f0d580c3790 con 0x7f0d58070e00 2024-09-10T10:28:56.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d5ce77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0d580c3a70 con 0x7f0d581528e0 2024-09-10T10:28:56.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d56d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0d58070e00 0x7f0d5815c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:56.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d56d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0d58070e00 0x7f0d5815c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57880/0 (socket says 172.21.15.120:57880) 2024-09-10T10:28:56.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d56d9d640 1 -- 172.21.15.120:0/2267511403 learned_addr learned my addr 172.21.15.120:0/2267511403 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:56.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d5659c640 1 --2- 172.21.15.120:0/2267511403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581528e0 0x7f0d580c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:56.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d56d9d640 1 -- 172.21.15.120:0/2267511403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581528e0 msgr2=0x7f0d580c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:56.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d56d9d640 1 --2- 172.21.15.120:0/2267511403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581528e0 0x7f0d580c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:56.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d56d9d640 1 -- 172.21.15.120:0/2267511403 >> v1:172.21.15.134:6789/0 conn(0x7f0d58152cc0 legacy=0x7f0d580ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:56.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d56d9d640 1 -- 172.21.15.120:0/2267511403 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5815bd00 con 0x7f0d58070e00 2024-09-10T10:28:56.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.534+0000 7f0d5659c640 1 --2- 172.21.15.120:0/2267511403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581528e0 0x7f0d580c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:28:56.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.535+0000 7f0d56d9d640 1 --2- 172.21.15.120:0/2267511403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d58070e00 0x7f0d5815c7e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009e50 tx=0x7f0d5002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa6d3ade48d9931 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:56.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.535+0000 7f0d5559a640 1 -- 172.21.15.120:0/2267511403 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50036070 con 0x7f0d58070e00 2024-09-10T10:28:56.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.535+0000 7f0d5559a640 1 -- 172.21.15.120:0/2267511403 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d50031600 con 0x7f0d58070e00 2024-09-10T10:28:56.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.535+0000 7f0d5559a640 1 -- 172.21.15.120:0/2267511403 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d500319d0 con 0x7f0d58070e00 2024-09-10T10:28:56.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.536+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2267511403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d58070e00 msgr2=0x7f0d5815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:56.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.536+0000 7f0d5ce77640 1 --2- 172.21.15.120:0/2267511403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d58070e00 0x7f0d5815c7e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009e50 tx=0x7f0d5002a750 comp rx=0 tx=0).stop 2024-09-10T10:28:56.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.536+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2267511403 shutdown_connections 2024-09-10T10:28:56.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.536+0000 7f0d5ce77640 1 --2- 172.21.15.120:0/2267511403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d58070e00 0x7f0d5815c7e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:56.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.536+0000 7f0d5ce77640 1 --2- 172.21.15.120:0/2267511403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581528e0 0x7f0d580c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:56.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.536+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2267511403 >> 172.21.15.120:0/2267511403 conn(0x7f0d5806c240 msgr2=0x7f0d58067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:56.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.536+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2267511403 shutdown_connections 2024-09-10T10:28:56.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.536+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2267511403 wait complete. 2024-09-10T10:28:56.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.537+0000 7f0d5ce77640 1 Processor -- start 2024-09-10T10:28:56.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.537+0000 7f0d5ce77640 1 -- start start 2024-09-10T10:28:56.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d5ce77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d58070e00 0x7f0d580c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:56.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d5ce77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d581528e0 0x7f0d581f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:56.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d5ce77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d58152cc0 0x7f0d5815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:56.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d5ce77640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d580cea90 con 0x7f0d581528e0 2024-09-10T10:28:56.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d5ce77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5815c2c0 con 0x7f0d58070e00 2024-09-10T10:28:56.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d5ce77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0d580cec10 con 0x7f0d58152cc0 2024-09-10T10:28:56.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d55d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d581528e0 0x7f0d581f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:56.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d55d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d581528e0 0x7f0d581f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57888/0 (socket says 172.21.15.120:57888) 2024-09-10T10:28:56.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d55d9b640 1 -- 172.21.15.120:0/2480433124 learned_addr learned my addr 172.21.15.120:0/2480433124 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:56.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d5659c640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d58070e00 0x7f0d580c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:56.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d56d9d640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d58152cc0 0x7f0d5815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:56.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d55d9b640 1 -- 172.21.15.120:0/2480433124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d58152cc0 msgr2=0x7f0d5815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:56.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d55d9b640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d58152cc0 0x7f0d5815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:56.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d55d9b640 1 -- 172.21.15.120:0/2480433124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d58070e00 msgr2=0x7f0d580c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:56.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d55d9b640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d58070e00 0x7f0d580c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:56.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d55d9b640 1 -- 172.21.15.120:0/2480433124 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5815b7c0 con 0x7f0d581528e0 2024-09-10T10:28:56.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d56d9d640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d58152cc0 0x7f0d5815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:56.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.538+0000 7f0d55d9b640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d581528e0 0x7f0d581f2790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b3c0 tx=0x7f0d4000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:56.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.539+0000 7f0d4effd640 1 -- 172.21.15.120:0/2480433124 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40011830 con 0x7f0d581528e0 2024-09-10T10:28:56.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.539+0000 7f0d4effd640 1 -- 172.21.15.120:0/2480433124 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d40016070 con 0x7f0d581528e0 2024-09-10T10:28:56.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.539+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2480433124 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d580c55f0 con 0x7f0d581528e0 2024-09-10T10:28:56.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.539+0000 7f0d4effd640 1 -- 172.21.15.120:0/2480433124 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40011c50 con 0x7f0d581528e0 2024-09-10T10:28:56.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.539+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2480433124 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d580c5bb0 con 0x7f0d581528e0 2024-09-10T10:28:56.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.540+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2480433124 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d580c9b90 con 0x7f0d581528e0 2024-09-10T10:28:56.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.540+0000 7f0d4effd640 1 -- 172.21.15.120:0/2480433124 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d4001b460 con 0x7f0d581528e0 2024-09-10T10:28:56.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.541+0000 7f0d4effd640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0d3806be60 0x7f0d3806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:56.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.541+0000 7f0d4effd640 1 -- 172.21.15.120:0/2480433124 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6986+0+0 (secure 0 0 0) 0x7f0d4008b020 con 0x7f0d581528e0 2024-09-10T10:28:56.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.541+0000 7f0d5659c640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0d3806be60 0x7f0d3806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:56.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.541+0000 7f0d5659c640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0d3806be60 0x7f0d3806e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800ab60 tx=0x7f0d48008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:56.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.545+0000 7f0d4effd640 1 -- 172.21.15.120:0/2480433124 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d40059440 con 0x7f0d581528e0 2024-09-10T10:28:56.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:56.721+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2480433124 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} v 0) -- 0x7f0d58155fb0 con 0x7f0d581528e0 2024-09-10T10:28:57.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.424+0000 7f0d4effd640 1 -- 172.21.15.120:0/2480433124 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]=0 set osd.3 primary-affinity to 1 (8655362) v70) ==== 133+0+0 (secure 0 0 0) 0x7f0d40058d40 con 0x7f0d581528e0 2024-09-10T10:28:57.425 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2480433124 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0d3806be60 msgr2=0x7f0d3806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0d3806be60 0x7f0d3806e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800ab60 tx=0x7f0d48008040 comp rx=0 tx=0).stop 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2480433124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d581528e0 msgr2=0x7f0d581f2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d581528e0 0x7f0d581f2790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b3c0 tx=0x7f0d4000b880 comp rx=0 tx=0).stop 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2480433124 shutdown_connections 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0d3806be60 0x7f0d3806e320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d58152cc0 0x7f0d5815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d581528e0 0x7f0d581f2790 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 --2- 172.21.15.120:0/2480433124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d58070e00 0x7f0d580c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2480433124 >> 172.21.15.120:0/2480433124 conn(0x7f0d5806c240 msgr2=0x7f0d580ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2480433124 shutdown_connections 2024-09-10T10:28:57.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.427+0000 7f0d5ce77640 1 -- 172.21.15.120:0/2480433124 wait complete. 2024-09-10T10:28:57.443 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-10T10:28:57.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.548+0000 7f0524518640 1 Processor -- start 2024-09-10T10:28:57.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.548+0000 7f0524518640 1 -- start start 2024-09-10T10:28:57.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0524518640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0ce5f0 0x7f051c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:57.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0524518640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f051c06eaa0 0x7f051c157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0524518640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0c9a20 con 0x7f051c0ce9d0 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0524518640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f051c15ef70 con 0x7f051c06eaa0 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0524518640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f051c06cd20 con 0x7f051c0ce5f0 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0522ab5640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f051c06eaa0 0x7f051c157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0522ab5640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f051c06eaa0 0x7f051c157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60960/0 (socket says 172.21.15.120:60960) 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0522ab5640 1 -- 172.21.15.120:0/3458190718 learned_addr learned my addr 172.21.15.120:0/3458190718 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f05222b4640 1 --2- 172.21.15.120:0/3458190718 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0ce5f0 0x7f051c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0522ab5640 1 -- 172.21.15.120:0/3458190718 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0ce5f0 msgr2=0x7f051c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0522ab5640 1 --2- 172.21.15.120:0/3458190718 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0ce5f0 0x7f051c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0522ab5640 1 -- 172.21.15.120:0/3458190718 >> v1:172.21.15.134:6789/0 conn(0x7f051c0ce9d0 legacy=0x7f051c15b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:57.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.549+0000 7f0522ab5640 1 -- 172.21.15.120:0/3458190718 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c06cf70 con 0x7f051c06eaa0 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.550+0000 7f0522ab5640 1 --2- 172.21.15.120:0/3458190718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 0x7f051c157dc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0514009e50 tx=0x7f051402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdf34cfeea539 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.550+0000 7f05212b2640 1 -- 172.21.15.120:0/3458190718 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0514036070 con 0x7f051c06eaa0 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.550+0000 7f05212b2640 1 -- 172.21.15.120:0/3458190718 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0514031600 con 0x7f051c06eaa0 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.550+0000 7f05212b2640 1 -- 172.21.15.120:0/3458190718 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05140319d0 con 0x7f051c06eaa0 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.550+0000 7f0524518640 1 -- 172.21.15.120:0/3458190718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 msgr2=0x7f051c157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.550+0000 7f0524518640 1 --2- 172.21.15.120:0/3458190718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 0x7f051c157dc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0514009e50 tx=0x7f051402a710 comp rx=0 tx=0).stop 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.551+0000 7f0524518640 1 -- 172.21.15.120:0/3458190718 shutdown_connections 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.551+0000 7f0524518640 1 --2- 172.21.15.120:0/3458190718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 0x7f051c157dc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.551+0000 7f0524518640 1 --2- 172.21.15.120:0/3458190718 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0ce5f0 0x7f051c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.551+0000 7f0524518640 1 -- 172.21.15.120:0/3458190718 >> 172.21.15.120:0/3458190718 conn(0x7f051c06c240 msgr2=0x7f051c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.551+0000 7f0524518640 1 -- 172.21.15.120:0/3458190718 shutdown_connections 2024-09-10T10:28:57.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.551+0000 7f0524518640 1 -- 172.21.15.120:0/3458190718 wait complete. 2024-09-10T10:28:57.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.552+0000 7f0524518640 1 Processor -- start 2024-09-10T10:28:57.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.552+0000 7f0524518640 1 -- start start 2024-09-10T10:28:57.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.552+0000 7f0524518640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 0x7f051c0b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:57.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.552+0000 7f0524518640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0ce5f0 0x7f051c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:57.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.552+0000 7f0524518640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f051c0ce9d0 0x7f051c159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:57.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f0524518640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f051c15b8f0 con 0x7f051c06eaa0 2024-09-10T10:28:57.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f05222b4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 0x7f051c0b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:57.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f0524518640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f051c15acc0 con 0x7f051c0ce9d0 2024-09-10T10:28:57.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f0524518640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f051c15ba70 con 0x7f051c0ce5f0 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f05222b4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 0x7f051c0b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60972/0 (socket says 172.21.15.120:60972) 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f05222b4640 1 -- 172.21.15.120:0/3034021436 learned_addr learned my addr 172.21.15.120:0/3034021436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f0521ab3640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0ce5f0 0x7f051c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f0522ab5640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f051c0ce9d0 0x7f051c159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f05222b4640 1 -- 172.21.15.120:0/3034021436 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0ce5f0 msgr2=0x7f051c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f05222b4640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0ce5f0 0x7f051c0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f05222b4640 1 -- 172.21.15.120:0/3034021436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f051c0ce9d0 msgr2=0x7f051c159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f05222b4640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f051c0ce9d0 0x7f051c159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f05222b4640 1 -- 172.21.15.120:0/3034021436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c154c40 con 0x7f051c06eaa0 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f05222b4640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 0x7f051c0b1360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f051800e9b0 tx=0x7f051800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f0512ffd640 1 -- 172.21.15.120:0/3034021436 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518017070 con 0x7f051c06eaa0 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f0512ffd640 1 -- 172.21.15.120:0/3034021436 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051800c9a0 con 0x7f051c06eaa0 2024-09-10T10:28:57.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f0524518640 1 -- 172.21.15.120:0/3034021436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051c0b0ba0 con 0x7f051c06eaa0 2024-09-10T10:28:57.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f0512ffd640 1 -- 172.21.15.120:0/3034021436 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051800cd90 con 0x7f051c06eaa0 2024-09-10T10:28:57.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.553+0000 7f0524518640 1 -- 172.21.15.120:0/3034021436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051c0c49f0 con 0x7f051c06eaa0 2024-09-10T10:28:57.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.554+0000 7f0524518640 1 -- 172.21.15.120:0/3034021436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051c070fe0 con 0x7f051c06eaa0 2024-09-10T10:28:57.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.555+0000 7f0512ffd640 1 -- 172.21.15.120:0/3034021436 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f051801a070 con 0x7f051c06eaa0 2024-09-10T10:28:57.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.555+0000 7f0512ffd640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f050006be30 0x7f050006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:57.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.555+0000 7f0521ab3640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f050006be30 0x7f050006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:57.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.555+0000 7f0512ffd640 1 -- 172.21.15.120:0/3034021436 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6986+0+0 (secure 0 0 0) 0x7f051808b330 con 0x7f051c06eaa0 2024-09-10T10:28:57.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.555+0000 7f0521ab3640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f050006be30 0x7f050006e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f051c0cb3a0 tx=0x7f050c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:57.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.559+0000 7f0512ffd640 1 -- 172.21.15.120:0/3034021436 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f051c070fe0 con 0x7f051c06eaa0 2024-09-10T10:28:57.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:57.734+0000 7f0524518640 1 -- 172.21.15.120:0/3034021436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7f051c0c94e0 con 0x7f051c06eaa0 2024-09-10T10:28:58.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.432+0000 7f0512ffd640 1 -- 172.21.15.120:0/3034021436 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v71) ==== 113+0+0 (secure 0 0 0) 0x7f0518059100 con 0x7f051c06eaa0 2024-09-10T10:28:58.434 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.4 to 1 (10000) 2024-09-10T10:28:58.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 -- 172.21.15.120:0/3034021436 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f050006be30 msgr2=0x7f050006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:58.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f050006be30 0x7f050006e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f051c0cb3a0 tx=0x7f050c009040 comp rx=0 tx=0).stop 2024-09-10T10:28:58.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 -- 172.21.15.120:0/3034021436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 msgr2=0x7f051c0b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 0x7f051c0b1360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f051800e9b0 tx=0x7f051800ee70 comp rx=0 tx=0).stop 2024-09-10T10:28:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 -- 172.21.15.120:0/3034021436 shutdown_connections 2024-09-10T10:28:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f050006be30 0x7f050006e2f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f051c0ce9d0 0x7f051c159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f051c0ce5f0 0x7f051c0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 --2- 172.21.15.120:0/3034021436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f051c06eaa0 0x7f051c0b1360 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 -- 172.21.15.120:0/3034021436 >> 172.21.15.120:0/3034021436 conn(0x7f051c06c240 msgr2=0x7f051c155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 -- 172.21.15.120:0/3034021436 shutdown_connections 2024-09-10T10:28:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.436+0000 7f0524518640 1 -- 172.21.15.120:0/3034021436 wait complete. 2024-09-10T10:28:58.452 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-10T10:28:58.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.554+0000 7efc72c53640 1 Processor -- start 2024-09-10T10:28:58.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.554+0000 7efc72c53640 1 -- start start 2024-09-10T10:28:58.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.555+0000 7efc72c53640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efc6c1528e0 0x7efc6c0c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.555+0000 7efc72c53640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc6c0c34b0 con 0x7efc6c070e00 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.555+0000 7efc72c53640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc6c0c3790 con 0x7efc6c152cc0 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.555+0000 7efc72c53640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7efc6c0c3a70 con 0x7efc6c1528e0 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.555+0000 7efc711f0640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7efc6c070e00 0x7efc6c163090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:52216/0 (socket says 172.21.15.120:52216) 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.555+0000 7efc711f0640 1 -- 172.21.15.120:0/1533896406 learned_addr learned my addr 172.21.15.120:0/1533896406 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.555+0000 7efc709ef640 1 --2- 172.21.15.120:0/1533896406 >> v2:172.21.15.134:3300/0 conn(0x7efc6c1528e0 0x7efc6c0c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.555+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1578105515 0 0) 0x7efc6c0c34b0 con 0x7efc6c070e00 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.555+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc54003520 con 0x7efc6c070e00 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1106414572 0 0) 0x7efc54003520 con 0x7efc6c070e00 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 >> v2:172.21.15.134:3300/0 conn(0x7efc6c1528e0 msgr2=0x7efc6c0c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:58.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 --2- 172.21.15.120:0/1533896406 >> v2:172.21.15.134:3300/0 conn(0x7efc6c1528e0 0x7efc6c0c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 >> v1:172.21.15.134:6789/0 conn(0x7efc6c152cc0 legacy=0x7efc6c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c15bd00 con 0x7efc6c070e00 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7efc640019f0 con 0x7efc6c070e00 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 >> v1:172.21.15.120:6789/0 conn(0x7efc6c070e00 legacy=0x7efc6c163090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc540039b0 0x7efc54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc54004800 0x7efc54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c0c3a70 con 0x7efc540051c0 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc5400c070 con 0x7efc540039b0 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efc5400c270 con 0x7efc54004800 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc711f0640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc709ef640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc540039b0 0x7efc54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:58.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc63fff640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc54004800 0x7efc54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc711f0640 1 -- 172.21.15.120:0/1533896406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc54004800 msgr2=0x7efc54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc711f0640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc54004800 0x7efc54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc711f0640 1 -- 172.21.15.120:0/1533896406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc540039b0 msgr2=0x7efc54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc711f0640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc540039b0 0x7efc54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc711f0640 1 -- 172.21.15.120:0/1533896406 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc54003520 con 0x7efc540051c0 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.556+0000 7efc63fff640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc54004800 0x7efc54004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.557+0000 7efc711f0640 1 -- 172.21.15.120:0/1533896406 reap_dead start 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.557+0000 7efc711f0640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc5400b9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efc6400a840 tx=0x7efc54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5db4b33e8e2b479 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.557+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6400a460 con 0x7efc540051c0 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.557+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6402f070 con 0x7efc540051c0 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.557+0000 7efc637fe640 1 -- 172.21.15.120:0/1533896406 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6402bcb0 con 0x7efc540051c0 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.558+0000 7efc72c53640 1 -- 172.21.15.120:0/1533896406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 msgr2=0x7efc5400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:58.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.558+0000 7efc72c53640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc5400b9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efc6400a840 tx=0x7efc54005830 comp rx=0 tx=0).stop 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.558+0000 7efc72c53640 1 -- 172.21.15.120:0/1533896406 shutdown_connections 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.558+0000 7efc72c53640 1 --2- 172.21.15.120:0/1533896406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc5400b9c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.558+0000 7efc72c53640 1 -- 172.21.15.120:0/1533896406 >> 172.21.15.120:0/1533896406 conn(0x7efc6c06c240 msgr2=0x7efc6c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.558+0000 7efc72c53640 1 -- 172.21.15.120:0/1533896406 shutdown_connections 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.558+0000 7efc72c53640 1 -- 172.21.15.120:0/1533896406 wait complete. 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.559+0000 7efc72c53640 1 Processor -- start 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.559+0000 7efc72c53640 1 -- start start 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc72c53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc6c14ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc72c53640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc6c0c5f40 0x7efc6c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc72c53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c1f1fd0 0x7efc6c1f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc72c53640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c0c5190 con 0x7efc540051c0 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc72c53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c15c2c0 con 0x7efc6c1f1fd0 2024-09-10T10:28:58.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc72c53640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efc6c0c5310 con 0x7efc6c0c5f40 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc709ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc6c14ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc709ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc6c14ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60996/0 (socket says 172.21.15.120:60996) 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc709ef640 1 -- 172.21.15.120:0/1675458678 learned_addr learned my addr 172.21.15.120:0/1675458678 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc711f0640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c1f1fd0 0x7efc6c1f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc63fff640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc6c0c5f40 0x7efc6c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc709ef640 1 -- 172.21.15.120:0/1675458678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc6c0c5f40 msgr2=0x7efc6c0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc709ef640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc6c0c5f40 0x7efc6c0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc709ef640 1 -- 172.21.15.120:0/1675458678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c1f1fd0 msgr2=0x7efc6c1f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc709ef640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c1f1fd0 0x7efc6c1f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc709ef640 1 -- 172.21.15.120:0/1675458678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c1f2990 con 0x7efc540051c0 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc709ef640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc6c14ae40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efc680118e0 tx=0x7efc68011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc617fa640 1 -- 172.21.15.120:0/1675458678 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6800faf0 con 0x7efc540051c0 2024-09-10T10:28:58.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc617fa640 1 -- 172.21.15.120:0/1675458678 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6800fc90 con 0x7efc540051c0 2024-09-10T10:28:58.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc72c53640 1 -- 172.21.15.120:0/1675458678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6c0c6ad0 con 0x7efc540051c0 2024-09-10T10:28:58.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.560+0000 7efc617fa640 1 -- 172.21.15.120:0/1675458678 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68014440 con 0x7efc540051c0 2024-09-10T10:28:58.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.561+0000 7efc72c53640 1 -- 172.21.15.120:0/1675458678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6c1534e0 con 0x7efc540051c0 2024-09-10T10:28:58.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.561+0000 7efc72c53640 1 -- 172.21.15.120:0/1675458678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6c0c46d0 con 0x7efc540051c0 2024-09-10T10:28:58.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.562+0000 7efc617fa640 1 -- 172.21.15.120:0/1675458678 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc68012040 con 0x7efc540051c0 2024-09-10T10:28:58.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.562+0000 7efc617fa640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc5806be60 0x7efc5806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:58.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.562+0000 7efc617fa640 1 -- 172.21.15.120:0/1675458678 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6986+0+0 (secure 0 0 0) 0x7efc6808b940 con 0x7efc540051c0 2024-09-10T10:28:58.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.562+0000 7efc63fff640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc5806be60 0x7efc5806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:58.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.563+0000 7efc63fff640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc5806be60 0x7efc5806e320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc5c000c00 tx=0x7efc5c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:58.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.567+0000 7efc617fa640 1 -- 172.21.15.120:0/1675458678 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efc6c0c46d0 con 0x7efc540051c0 2024-09-10T10:28:58.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:58.743+0000 7efc72c53640 1 -- 172.21.15.120:0/1675458678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} v 0) -- 0x7efc6c0c9fc0 con 0x7efc540051c0 2024-09-10T10:28:59.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.433+0000 7efc617fa640 1 -- 172.21.15.120:0/1675458678 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]=0 set osd.4 primary-affinity to 1 (8655362) v72) ==== 133+0+0 (secure 0 0 0) 0x7efc6c1f2990 con 0x7efc540051c0 2024-09-10T10:28:59.436 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-10T10:28:59.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 -- 172.21.15.120:0/1675458678 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc5806be60 msgr2=0x7efc5806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:59.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc5806be60 0x7efc5806e320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc5c000c00 tx=0x7efc5c009040 comp rx=0 tx=0).stop 2024-09-10T10:28:59.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 -- 172.21.15.120:0/1675458678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 msgr2=0x7efc6c14ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:59.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc6c14ae40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efc680118e0 tx=0x7efc68011da0 comp rx=0 tx=0).stop 2024-09-10T10:28:59.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 -- 172.21.15.120:0/1675458678 shutdown_connections 2024-09-10T10:28:59.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efc5806be60 0x7efc5806e320 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:59.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c1f1fd0 0x7efc6c1f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:59.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc6c0c5f40 0x7efc6c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:59.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 --2- 172.21.15.120:0/1675458678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc540051c0 0x7efc6c14ae40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:59.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 -- 172.21.15.120:0/1675458678 >> 172.21.15.120:0/1675458678 conn(0x7efc6c06c240 msgr2=0x7efc6c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:59.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 -- 172.21.15.120:0/1675458678 shutdown_connections 2024-09-10T10:28:59.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.438+0000 7efc72c53640 1 -- 172.21.15.120:0/1675458678 wait complete. 2024-09-10T10:28:59.458 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-10T10:28:59.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.560+0000 7fdc98077640 1 Processor -- start 2024-09-10T10:28:59.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.560+0000 7fdc98077640 1 -- start start 2024-09-10T10:28:59.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.561+0000 7fdc98077640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc90151170 0x7fdc900c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:59.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.561+0000 7fdc98077640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdc900c7380 0x7fdc900c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:59.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.561+0000 7fdc98077640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc90151550 con 0x7fdc90156e40 2024-09-10T10:28:59.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.561+0000 7fdc98077640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fdc900c4cf0 con 0x7fdc900c7380 2024-09-10T10:28:59.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.561+0000 7fdc98077640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdc901572d0 con 0x7fdc90151170 2024-09-10T10:28:59.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.561+0000 7fdc96614640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdc900c7380 0x7fdc900c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:59.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.561+0000 7fdc96614640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdc900c7380 0x7fdc900c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32768/0 (socket says 172.21.15.120:32768) 2024-09-10T10:28:59.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.561+0000 7fdc96614640 1 -- 172.21.15.120:0/862328733 learned_addr learned my addr 172.21.15.120:0/862328733 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:59.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.561+0000 7fdc95612640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fdc90156e40 0x7fdc900cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:52230/0 (socket says 172.21.15.120:52230) 2024-09-10T10:28:59.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.561+0000 7fdc95e13640 1 --2- 172.21.15.120:0/862328733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc90151170 0x7fdc900c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:59.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.562+0000 7fdc96614640 1 -- 172.21.15.120:0/862328733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc90151170 msgr2=0x7fdc900c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:28:59.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.562+0000 7fdc96614640 1 --2- 172.21.15.120:0/862328733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc90151170 0x7fdc900c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:59.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.562+0000 7fdc96614640 1 -- 172.21.15.120:0/862328733 >> v1:172.21.15.120:6789/0 conn(0x7fdc90156e40 legacy=0x7fdc900cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:59.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.562+0000 7fdc96614640 1 -- 172.21.15.120:0/862328733 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc90157570 con 0x7fdc900c7380 2024-09-10T10:28:59.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.562+0000 7fdc96614640 1 --2- 172.21.15.120:0/862328733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc900c7380 0x7fdc900c5180 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800d150 tx=0x7fdc8802c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c64538e8bf9d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:59.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.562+0000 7fdc94e11640 1 -- 172.21.15.120:0/862328733 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc880337a0 con 0x7fdc900c7380 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.563+0000 7fdc94e11640 1 -- 172.21.15.120:0/862328733 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc88039070 con 0x7fdc900c7380 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.563+0000 7fdc94e11640 1 -- 172.21.15.120:0/862328733 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88033b60 con 0x7fdc900c7380 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.563+0000 7fdc98077640 1 -- 172.21.15.120:0/862328733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc900c7380 msgr2=0x7fdc900c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.563+0000 7fdc98077640 1 --2- 172.21.15.120:0/862328733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc900c7380 0x7fdc900c5180 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800d150 tx=0x7fdc8802c930 comp rx=0 tx=0).stop 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.563+0000 7fdc98077640 1 -- 172.21.15.120:0/862328733 shutdown_connections 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.563+0000 7fdc98077640 1 --2- 172.21.15.120:0/862328733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc900c7380 0x7fdc900c5180 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.563+0000 7fdc98077640 1 --2- 172.21.15.120:0/862328733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc90151170 0x7fdc900c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.563+0000 7fdc98077640 1 -- 172.21.15.120:0/862328733 >> 172.21.15.120:0/862328733 conn(0x7fdc9006e670 msgr2=0x7fdc9006ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.563+0000 7fdc98077640 1 -- 172.21.15.120:0/862328733 shutdown_connections 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.563+0000 7fdc98077640 1 -- 172.21.15.120:0/862328733 wait complete. 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.564+0000 7fdc98077640 1 Processor -- start 2024-09-10T10:28:59.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.564+0000 7fdc98077640 1 -- start start 2024-09-10T10:28:59.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc98077640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc900c7380 0x7fdc901440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:59.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc98077640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc90151170 0x7fdc90152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:59.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc98077640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc901445e0 0x7fdc90145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:59.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc98077640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90157af0 con 0x7fdc90151170 2024-09-10T10:28:59.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc98077640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90157810 con 0x7fdc901445e0 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc98077640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdc900cc940 con 0x7fdc900c7380 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc95612640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc90151170 0x7fdc90152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc95612640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc90151170 0x7fdc90152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32770/0 (socket says 172.21.15.120:32770) 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc95612640 1 -- 172.21.15.120:0/3870667376 learned_addr learned my addr 172.21.15.120:0/3870667376 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc95e13640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc900c7380 0x7fdc901440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.565+0000 7fdc96614640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc901445e0 0x7fdc90145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc95612640 1 -- 172.21.15.120:0/3870667376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc900c7380 msgr2=0x7fdc901440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc95612640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc900c7380 0x7fdc901440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc95612640 1 -- 172.21.15.120:0/3870667376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc901445e0 msgr2=0x7fdc90145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc95612640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc901445e0 0x7fdc90145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc95612640 1 -- 172.21.15.120:0/3870667376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc900c7a50 con 0x7fdc90151170 2024-09-10T10:28:59.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc95612640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc90151170 0x7fdc90152680 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000c440 tx=0x7fdc80001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc867fc640 1 -- 172.21.15.120:0/3870667376 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc800074e0 con 0x7fdc90151170 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc867fc640 1 -- 172.21.15.120:0/3870667376 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc80007680 con 0x7fdc90151170 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc98077640 1 -- 172.21.15.120:0/3870667376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc900c7d30 con 0x7fdc90151170 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc867fc640 1 -- 172.21.15.120:0/3870667376 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80004a20 con 0x7fdc90151170 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.566+0000 7fdc98077640 1 -- 172.21.15.120:0/3870667376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc900c94a0 con 0x7fdc90151170 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.567+0000 7fdc98077640 1 -- 172.21.15.120:0/3870667376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc900c6cc0 con 0x7fdc90151170 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.567+0000 7fdc867fc640 1 -- 172.21.15.120:0/3870667376 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc8001a020 con 0x7fdc90151170 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.568+0000 7fdc867fc640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc7406be30 0x7fdc7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.568+0000 7fdc95e13640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc7406be30 0x7fdc7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.568+0000 7fdc867fc640 1 -- 172.21.15.120:0/3870667376 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6986+0+0 (secure 0 0 0) 0x7fdc8008a980 con 0x7fdc90151170 2024-09-10T10:28:59.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.568+0000 7fdc95e13640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc7406be30 0x7fdc7406e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c006fd0 tx=0x7fdc8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:28:59.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.572+0000 7fdc867fc640 1 -- 172.21.15.120:0/3870667376 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc90151550 con 0x7fdc90151170 2024-09-10T10:28:59.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:28:59.748+0000 7fdc98077640 1 -- 172.21.15.120:0/3870667376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7fdc9006fc90 con 0x7fdc90151170 2024-09-10T10:29:00.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.442+0000 7fdc867fc640 1 -- 172.21.15.120:0/3870667376 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v73) ==== 113+0+0 (secure 0 0 0) 0x7fdc80058870 con 0x7fdc90151170 2024-09-10T10:29:00.444 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.5 to 1 (10000) 2024-09-10T10:29:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 -- 172.21.15.120:0/3870667376 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc7406be30 msgr2=0x7fdc7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc7406be30 0x7fdc7406e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c006fd0 tx=0x7fdc8c008040 comp rx=0 tx=0).stop 2024-09-10T10:29:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 -- 172.21.15.120:0/3870667376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc90151170 msgr2=0x7fdc90152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc90151170 0x7fdc90152680 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000c440 tx=0x7fdc80001e70 comp rx=0 tx=0).stop 2024-09-10T10:29:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 -- 172.21.15.120:0/3870667376 shutdown_connections 2024-09-10T10:29:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fdc7406be30 0x7fdc7406e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc901445e0 0x7fdc90145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc90151170 0x7fdc90152680 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 --2- 172.21.15.120:0/3870667376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc900c7380 0x7fdc901440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 -- 172.21.15.120:0/3870667376 >> 172.21.15.120:0/3870667376 conn(0x7fdc9006e670 msgr2=0x7fdc9014f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 -- 172.21.15.120:0/3870667376 shutdown_connections 2024-09-10T10:29:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.445+0000 7fdc98077640 1 -- 172.21.15.120:0/3870667376 wait complete. 2024-09-10T10:29:00.460 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-10T10:29:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.561+0000 7fbc1f4b7640 1 Processor -- start 2024-09-10T10:29:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.562+0000 7fbc1f4b7640 1 -- start start 2024-09-10T10:29:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.562+0000 7fbc1f4b7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbc180cc0a0 0x7fbc180cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.562+0000 7fbc1f4b7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d0630 0x7fbc180d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.562+0000 7fbc1f4b7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1815ba20 con 0x7fbc1815e660 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.562+0000 7fbc1f4b7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1815e1d0 con 0x7fbc180cc0a0 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.562+0000 7fbc1f4b7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1815aea0 con 0x7fbc180d0630 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc1da54640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fbc1815e660 0x7fbc180cd0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:52236/0 (socket says 172.21.15.120:52236) 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc1da54640 1 -- 172.21.15.120:0/2972903426 learned_addr learned my addr 172.21.15.120:0/2972903426 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc1ca52640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d0630 0x7fbc180d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc1d253640 1 --2- 172.21.15.120:0/2972903426 >> v2:172.21.15.134:3300/0 conn(0x7fbc180cc0a0 0x7fbc180cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3757739742 0 0) 0x7fbc1815ba20 con 0x7fbc1815e660 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc04003520 con 0x7fbc1815e660 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1278088067 0 0) 0x7fbc04003520 con 0x7fbc1815e660 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d0630 msgr2=0x7fbc180d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d0630 0x7fbc180d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 >> v2:172.21.15.134:3300/0 conn(0x7fbc180cc0a0 msgr2=0x7fbc180cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:00.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 --2- 172.21.15.120:0/2972903426 >> v2:172.21.15.134:3300/0 conn(0x7fbc180cc0a0 0x7fbc180cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1815b180 con 0x7fbc1815e660 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7fbc100019f0 con 0x7fbc1815e660 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 >> v1:172.21.15.120:6789/0 conn(0x7fbc1815e660 legacy=0x7fbc180cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc040039b0 0x7fbc04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc04004800 0x7fbc04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d0630 0x7fbc040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1815e1d0 con 0x7fbc040039b0 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1815aea0 con 0x7fbc04004800 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0400b9c0 con 0x7fbc180d0630 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc1d253640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc040039b0 0x7fbc04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc1ca52640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc04004800 0x7fbc04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.563+0000 7fbc1da54640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d0630 0x7fbc040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:00.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.564+0000 7fbc1d253640 1 -- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d0630 msgr2=0x7fbc040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.564+0000 7fbc1d253640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc180d0630 0x7fbc040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.564+0000 7fbc1d253640 1 -- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc04004800 msgr2=0x7fbc04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.564+0000 7fbc1d253640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc04004800 0x7fbc04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.564+0000 7fbc1d253640 1 -- 172.21.15.120:0/2972903426 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc04003520 con 0x7fbc040039b0 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.564+0000 7fbc1d253640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc040039b0 0x7fbc04003dd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009ac0 tx=0x7fbc04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdd14a4d778c908 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.564+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c036070 con 0x7fbc040039b0 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.564+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0c031400 con 0x7fbc040039b0 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.564+0000 7fbc0bfff640 1 -- 172.21.15.120:0/2972903426 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c009690 con 0x7fbc040039b0 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.565+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/2972903426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc040039b0 msgr2=0x7fbc04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.565+0000 7fbc1f4b7640 1 --2- 172.21.15.120:0/2972903426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc040039b0 0x7fbc04003dd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009ac0 tx=0x7fbc04005830 comp rx=0 tx=0).stop 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.565+0000 7fbc1da54640 1 -- 172.21.15.120:0/2972903426 reap_dead start 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.565+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/2972903426 shutdown_connections 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.565+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/2972903426 >> 172.21.15.120:0/2972903426 conn(0x7fbc1806d560 msgr2=0x7fbc18073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:00.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.565+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/2972903426 shutdown_connections 2024-09-10T10:29:00.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.565+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/2972903426 wait complete. 2024-09-10T10:29:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.566+0000 7fbc1f4b7640 1 Processor -- start 2024-09-10T10:29:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.566+0000 7fbc1f4b7640 1 -- start start 2024-09-10T10:29:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1f4b7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc040039b0 0x7fbc18151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1f4b7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc1814be60 0x7fbc181512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1f4b7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc1815cf30 0x7fbc1814a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1f4b7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180cbde0 con 0x7fbc1815cf30 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1f4b7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1815b740 con 0x7fbc1814be60 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1f4b7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1815aaf0 con 0x7fbc040039b0 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1da54640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc1815cf30 0x7fbc1814a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1da54640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc1815cf30 0x7fbc1814a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32786/0 (socket says 172.21.15.120:32786) 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1da54640 1 -- 172.21.15.120:0/3176071646 learned_addr learned my addr 172.21.15.120:0/3176071646 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1d253640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc040039b0 0x7fbc18151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1ca52640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc1814be60 0x7fbc181512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1da54640 1 -- 172.21.15.120:0/3176071646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc040039b0 msgr2=0x7fbc18151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1da54640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc040039b0 0x7fbc18151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1da54640 1 -- 172.21.15.120:0/3176071646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc1814be60 msgr2=0x7fbc181512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1da54640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc1814be60 0x7fbc181512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1da54640 1 -- 172.21.15.120:0/3176071646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc18152440 con 0x7fbc1815cf30 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.567+0000 7fbc1da54640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc1815cf30 0x7fbc1814a320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbc180b3ad0 tx=0x7fbc1000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.568+0000 7fbc09ffb640 1 -- 172.21.15.120:0/3176071646 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1000ec10 con 0x7fbc1815cf30 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.568+0000 7fbc09ffb640 1 -- 172.21.15.120:0/3176071646 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1000edb0 con 0x7fbc1815cf30 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.568+0000 7fbc09ffb640 1 -- 172.21.15.120:0/3176071646 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10014670 con 0x7fbc1815cf30 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.568+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/3176071646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc180c5ef0 con 0x7fbc1815cf30 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.568+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/3176071646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1814e8f0 con 0x7fbc1815cf30 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.569+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/3176071646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc180ce5a0 con 0x7fbc1815cf30 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.569+0000 7fbc09ffb640 1 -- 172.21.15.120:0/3176071646 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc10014810 con 0x7fbc1815cf30 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.570+0000 7fbc09ffb640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbbfc06be60 0x7fbbfc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.570+0000 7fbc1d253640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbbfc06be60 0x7fbbfc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.570+0000 7fbc09ffb640 1 -- 172.21.15.120:0/3176071646 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6986+0+0 (secure 0 0 0) 0x7fbc1008b0e0 con 0x7fbc1815cf30 2024-09-10T10:29:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.570+0000 7fbc1d253640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbbfc06be60 0x7fbbfc06e320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c033000 tx=0x7fbc0c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.574+0000 7fbc09ffb640 1 -- 172.21.15.120:0/3176071646 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbc180ce5a0 con 0x7fbc1815cf30 2024-09-10T10:29:00.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:00.751+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/3176071646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} v 0) -- 0x7fbc181517f0 con 0x7fbc1815cf30 2024-09-10T10:29:01.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.448+0000 7fbc09ffb640 1 -- 172.21.15.120:0/3176071646 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]=0 set osd.5 primary-affinity to 1 (8655362) v74) ==== 133+0+0 (secure 0 0 0) 0x7fbc18152440 con 0x7fbc1815cf30 2024-09-10T10:29:01.451 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-10T10:29:01.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/3176071646 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbbfc06be60 msgr2=0x7fbbfc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:01.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbbfc06be60 0x7fbbfc06e320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c033000 tx=0x7fbc0c008030 comp rx=0 tx=0).stop 2024-09-10T10:29:01.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/3176071646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc1815cf30 msgr2=0x7fbc1814a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:01.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc1815cf30 0x7fbc1814a320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbc180b3ad0 tx=0x7fbc1000e530 comp rx=0 tx=0).stop 2024-09-10T10:29:01.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/3176071646 shutdown_connections 2024-09-10T10:29:01.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fbbfc06be60 0x7fbbfc06e320 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc1815cf30 0x7fbc1814a320 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc1814be60 0x7fbc181512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 --2- 172.21.15.120:0/3176071646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc040039b0 0x7fbc18151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/3176071646 >> 172.21.15.120:0/3176071646 conn(0x7fbc1806d560 msgr2=0x7fbc180ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:01.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/3176071646 shutdown_connections 2024-09-10T10:29:01.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.452+0000 7fbc1f4b7640 1 -- 172.21.15.120:0/3176071646 wait complete. 2024-09-10T10:29:01.468 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-10T10:29:01.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.570+0000 7f53a3f03640 1 Processor -- start 2024-09-10T10:29:01.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.570+0000 7f53a3f03640 1 -- start start 2024-09-10T10:29:01.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a3f03640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f539c1528e0 0x7f539c0c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:01.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a3f03640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f539c152cc0 0x7f539c15c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:01.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a3f03640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f539c070e00 0x7f539c0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:01.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a3f03640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f539c0c9330 con 0x7f539c152cc0 2024-09-10T10:29:01.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a3f03640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f539c0c34b0 con 0x7f539c1528e0 2024-09-10T10:29:01.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a3f03640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f539c0c3790 con 0x7f539c070e00 2024-09-10T10:29:01.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a149e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f539c152cc0 0x7f539c15c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:01.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a149e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f539c152cc0 0x7f539c15c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32788/0 (socket says 172.21.15.120:32788) 2024-09-10T10:29:01.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a149e640 1 -- 172.21.15.120:0/394719359 learned_addr learned my addr 172.21.15.120:0/394719359 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:01.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a1c9f640 1 --2- 172.21.15.120:0/394719359 >> v2:172.21.15.134:3300/0 conn(0x7f539c1528e0 0x7f539c0c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:01.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a24a0640 1 --2- 172.21.15.120:0/394719359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f539c070e00 0x7f539c0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:01.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a149e640 1 -- 172.21.15.120:0/394719359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f539c070e00 msgr2=0x7f539c0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:01.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a149e640 1 --2- 172.21.15.120:0/394719359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f539c070e00 0x7f539c0c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a149e640 1 -- 172.21.15.120:0/394719359 >> v2:172.21.15.134:3300/0 conn(0x7f539c1528e0 msgr2=0x7f539c0c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:01.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a149e640 1 --2- 172.21.15.120:0/394719359 >> v2:172.21.15.134:3300/0 conn(0x7f539c1528e0 0x7f539c0c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.571+0000 7f53a149e640 1 -- 172.21.15.120:0/394719359 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c0c3a70 con 0x7f539c152cc0 2024-09-10T10:29:01.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.572+0000 7f53a149e640 1 --2- 172.21.15.120:0/394719359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 0x7f539c15c7e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f539c0cbf90 tx=0x7f538c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53066aebc89307fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:01.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.572+0000 7f53a0c9d640 1 -- 172.21.15.120:0/394719359 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c037070 con 0x7f539c152cc0 2024-09-10T10:29:01.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.572+0000 7f53a0c9d640 1 -- 172.21.15.120:0/394719359 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c032440 con 0x7f539c152cc0 2024-09-10T10:29:01.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.572+0000 7f53a0c9d640 1 -- 172.21.15.120:0/394719359 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c032810 con 0x7f539c152cc0 2024-09-10T10:29:01.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.572+0000 7f53a3f03640 1 -- 172.21.15.120:0/394719359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 msgr2=0x7f539c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:01.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.572+0000 7f53a3f03640 1 --2- 172.21.15.120:0/394719359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 0x7f539c15c7e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f539c0cbf90 tx=0x7f538c02a270 comp rx=0 tx=0).stop 2024-09-10T10:29:01.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.573+0000 7f53a3f03640 1 -- 172.21.15.120:0/394719359 shutdown_connections 2024-09-10T10:29:01.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.573+0000 7f53a3f03640 1 --2- 172.21.15.120:0/394719359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f539c070e00 0x7f539c0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.573+0000 7f53a3f03640 1 --2- 172.21.15.120:0/394719359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 0x7f539c15c7e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.573+0000 7f53a3f03640 1 --2- 172.21.15.120:0/394719359 >> v2:172.21.15.134:3300/0 conn(0x7f539c1528e0 0x7f539c0c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.573+0000 7f53a3f03640 1 -- 172.21.15.120:0/394719359 >> 172.21.15.120:0/394719359 conn(0x7f539c06c240 msgr2=0x7f539c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:01.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.573+0000 7f53a3f03640 1 -- 172.21.15.120:0/394719359 shutdown_connections 2024-09-10T10:29:01.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.573+0000 7f53a3f03640 1 -- 172.21.15.120:0/394719359 wait complete. 2024-09-10T10:29:01.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.574+0000 7f53a3f03640 1 Processor -- start 2024-09-10T10:29:01.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.574+0000 7f53a3f03640 1 -- start start 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.574+0000 7f53a3f03640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f539c070e00 0x7f539c158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.574+0000 7f53a3f03640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f539c1528e0 0x7f539c156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.574+0000 7f53a3f03640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 0x7f539c14d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.574+0000 7f53a3f03640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f539c15c2c0 con 0x7f539c152cc0 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.574+0000 7f53a3f03640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f539c15bfe0 con 0x7f539c070e00 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.574+0000 7f53a3f03640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f539c0c5190 con 0x7f539c1528e0 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a24a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 0x7f539c14d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a24a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 0x7f539c14d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32802/0 (socket says 172.21.15.120:32802) 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a24a0640 1 -- 172.21.15.120:0/1933121954 learned_addr learned my addr 172.21.15.120:0/1933121954 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a149e640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f539c1528e0 0x7f539c156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:01.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a1c9f640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f539c070e00 0x7f539c158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a24a0640 1 -- 172.21.15.120:0/1933121954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f539c1528e0 msgr2=0x7f539c156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a24a0640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f539c1528e0 0x7f539c156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a24a0640 1 -- 172.21.15.120:0/1933121954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f539c070e00 msgr2=0x7f539c158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a24a0640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f539c070e00 0x7f539c158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a24a0640 1 -- 172.21.15.120:0/1933121954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c157420 con 0x7f539c152cc0 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a24a0640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 0x7f539c14d800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5394007e90 tx=0x7f539400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53927fc640 1 -- 172.21.15.120:0/1933121954 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394014420 con 0x7f539c152cc0 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53927fc640 1 -- 172.21.15.120:0/1933121954 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5394019070 con 0x7f539c152cc0 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a3f03640 1 -- 172.21.15.120:0/1933121954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c1f2190 con 0x7f539c152cc0 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53927fc640 1 -- 172.21.15.120:0/1933121954 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394014820 con 0x7f539c152cc0 2024-09-10T10:29:01.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.575+0000 7f53a3f03640 1 -- 172.21.15.120:0/1933121954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c1ee440 con 0x7f539c152cc0 2024-09-10T10:29:01.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.576+0000 7f53a3f03640 1 -- 172.21.15.120:0/1933121954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c0c8df0 con 0x7f539c152cc0 2024-09-10T10:29:01.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.577+0000 7f53927fc640 1 -- 172.21.15.120:0/1933121954 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53940149c0 con 0x7f539c152cc0 2024-09-10T10:29:01.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.577+0000 7f53927fc640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f538006be30 0x7f538006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:01.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.577+0000 7f53927fc640 1 -- 172.21.15.120:0/1933121954 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6986+0+0 (secure 0 0 0) 0x7f539408b9c0 con 0x7f539c152cc0 2024-09-10T10:29:01.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.577+0000 7f53a1c9f640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f538006be30 0x7f538006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:01.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.577+0000 7f53a1c9f640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f538006be30 0x7f538006e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5398006fd0 tx=0x7f5398008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:01.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.581+0000 7f53927fc640 1 -- 172.21.15.120:0/1933121954 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f539408de20 con 0x7f539c152cc0 2024-09-10T10:29:01.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:01.756+0000 7f53a3f03640 1 -- 172.21.15.120:0/1933121954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7f539c070190 con 0x7f539c152cc0 2024-09-10T10:29:02.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.455+0000 7f53927fc640 1 -- 172.21.15.120:0/1933121954 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v75) ==== 113+0+0 (secure 0 0 0) 0x7f53940598b0 con 0x7f539c152cc0 2024-09-10T10:29:02.457 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.6 to 1 (10000) 2024-09-10T10:29:02.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 -- 172.21.15.120:0/1933121954 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f538006be30 msgr2=0x7f538006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:02.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f538006be30 0x7f538006e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5398006fd0 tx=0x7f5398008040 comp rx=0 tx=0).stop 2024-09-10T10:29:02.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 -- 172.21.15.120:0/1933121954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 msgr2=0x7f539c14d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:02.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 0x7f539c14d800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5394007e90 tx=0x7f539400e4b0 comp rx=0 tx=0).stop 2024-09-10T10:29:02.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 -- 172.21.15.120:0/1933121954 shutdown_connections 2024-09-10T10:29:02.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f538006be30 0x7f538006e2f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:02.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f539c152cc0 0x7f539c14d800 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:02.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f539c1528e0 0x7f539c156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:02.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 --2- 172.21.15.120:0/1933121954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f539c070e00 0x7f539c158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:02.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 -- 172.21.15.120:0/1933121954 >> 172.21.15.120:0/1933121954 conn(0x7f539c06c240 msgr2=0x7f539c14f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:02.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 -- 172.21.15.120:0/1933121954 shutdown_connections 2024-09-10T10:29:02.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.458+0000 7f53a3f03640 1 -- 172.21.15.120:0/1933121954 wait complete. 2024-09-10T10:29:02.472 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-10T10:29:02.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.571+0000 7fcac0a00640 1 Processor -- start 2024-09-10T10:29:02.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.571+0000 7fcac0a00640 1 -- start start 2024-09-10T10:29:02.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcac0a00640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcabc154be0 0x7fcabc06e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcac0a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcabc070d60 0x7fcabc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcac0a00640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcabc0c52a0 con 0x7fcabc0c8d60 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcac0a00640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fcabc06ffe0 con 0x7fcabc154be0 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcac0a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcabc0702c0 con 0x7fcabc070d60 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcabad9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcabc154be0 0x7fcabc06e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcab259c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fcabc0c8d60 0x7fcabc15dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:52244/0 (socket says 172.21.15.120:52244) 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcab259c640 1 -- 172.21.15.120:0/3046973908 learned_addr learned my addr 172.21.15.120:0/3046973908 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcabb59e640 1 --2- 172.21.15.120:0/3046973908 >> v2:172.21.15.134:3300/0 conn(0x7fcabc070d60 0x7fcabc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcabad9d640 1 -- 172.21.15.120:0/3046973908 >> v2:172.21.15.134:3300/0 conn(0x7fcabc070d60 msgr2=0x7fcabc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcabad9d640 1 --2- 172.21.15.120:0/3046973908 >> v2:172.21.15.134:3300/0 conn(0x7fcabc070d60 0x7fcabc0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:02.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcabad9d640 1 -- 172.21.15.120:0/3046973908 >> v1:172.21.15.120:6789/0 conn(0x7fcabc0c8d60 legacy=0x7fcabc15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:02.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcabad9d640 1 -- 172.21.15.120:0/3046973908 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc06cec0 con 0x7fcabc154be0 2024-09-10T10:29:02.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.572+0000 7fcaba59c640 1 -- 172.21.15.120:0/3046973908 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689251577 0 0) 0x7fcabc0c52a0 con 0x7fcabc0c8d60 2024-09-10T10:29:02.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.573+0000 7fcabad9d640 1 --2- 172.21.15.120:0/3046973908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc154be0 0x7fcabc06e960 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0cc1a0 tx=0x7fcaac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bd1cd6066c7dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:02.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.573+0000 7fcaba59c640 1 -- 172.21.15.120:0/3046973908 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac037070 con 0x7fcabc154be0 2024-09-10T10:29:02.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.573+0000 7fcaba59c640 1 -- 172.21.15.120:0/3046973908 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaac032440 con 0x7fcabc154be0 2024-09-10T10:29:02.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.573+0000 7fcaba59c640 1 -- 172.21.15.120:0/3046973908 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac02ae60 con 0x7fcabc154be0 2024-09-10T10:29:02.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.574+0000 7fcac0a00640 1 -- 172.21.15.120:0/3046973908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc154be0 msgr2=0x7fcabc06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:02.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.574+0000 7fcac0a00640 1 --2- 172.21.15.120:0/3046973908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc154be0 0x7fcabc06e960 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0cc1a0 tx=0x7fcaac02a110 comp rx=0 tx=0).stop 2024-09-10T10:29:02.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.574+0000 7fcac0a00640 1 -- 172.21.15.120:0/3046973908 shutdown_connections 2024-09-10T10:29:02.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.574+0000 7fcac0a00640 1 --2- 172.21.15.120:0/3046973908 >> v2:172.21.15.134:3300/0 conn(0x7fcabc070d60 0x7fcabc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:02.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.574+0000 7fcac0a00640 1 --2- 172.21.15.120:0/3046973908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc154be0 0x7fcabc06e960 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:02.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.574+0000 7fcac0a00640 1 -- 172.21.15.120:0/3046973908 >> 172.21.15.120:0/3046973908 conn(0x7fcabc06c240 msgr2=0x7fcabc067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:02.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.574+0000 7fcac0a00640 1 -- 172.21.15.120:0/3046973908 shutdown_connections 2024-09-10T10:29:02.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.574+0000 7fcac0a00640 1 -- 172.21.15.120:0/3046973908 wait complete. 2024-09-10T10:29:02.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.575+0000 7fcac0a00640 1 Processor -- start 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.575+0000 7fcac0a00640 1 -- start start 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.576+0000 7fcac0a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc070d60 0x7fcabc1e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.576+0000 7fcac0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcabc0c8d60 0x7fcabc1e6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.576+0000 7fcac0a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcabc154be0 0x7fcabc1f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.576+0000 7fcac0a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc15e3a0 con 0x7fcabc070d60 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.576+0000 7fcac0a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc070750 con 0x7fcabc0c8d60 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.576+0000 7fcac0a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcabc0ce780 con 0x7fcabc154be0 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.576+0000 7fcabad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc070d60 0x7fcabc1e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.576+0000 7fcabad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc070d60 0x7fcabc1e43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32820/0 (socket says 172.21.15.120:32820) 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.576+0000 7fcabad9d640 1 -- 172.21.15.120:0/349054123 learned_addr learned my addr 172.21.15.120:0/349054123 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcab259c640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcabc0c8d60 0x7fcabc1e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:02.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcabb59e640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcabc154be0 0x7fcabc1f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcabad9d640 1 -- 172.21.15.120:0/349054123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcabc154be0 msgr2=0x7fcabc1f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcabad9d640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcabc154be0 0x7fcabc1f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcabad9d640 1 -- 172.21.15.120:0/349054123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcabc0c8d60 msgr2=0x7fcabc1e6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcabad9d640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcabc0c8d60 0x7fcabc1e6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcabad9d640 1 -- 172.21.15.120:0/349054123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc14b970 con 0x7fcabc070d60 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcabad9d640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc070d60 0x7fcabc1e43e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcaac000c00 tx=0x7fcaac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcab3fff640 1 -- 172.21.15.120:0/349054123 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac037040 con 0x7fcabc070d60 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcab3fff640 1 -- 172.21.15.120:0/349054123 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaac042070 con 0x7fcabc070d60 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcac0a00640 1 -- 172.21.15.120:0/349054123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc1578b0 con 0x7fcabc070d60 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcab3fff640 1 -- 172.21.15.120:0/349054123 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac030bd0 con 0x7fcabc070d60 2024-09-10T10:29:02.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.577+0000 7fcac0a00640 1 -- 172.21.15.120:0/349054123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1e4fd0 con 0x7fcabc070d60 2024-09-10T10:29:02.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.578+0000 7fcac0a00640 1 -- 172.21.15.120:0/349054123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcabc06fd70 con 0x7fcabc070d60 2024-09-10T10:29:02.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.578+0000 7fcab3fff640 1 -- 172.21.15.120:0/349054123 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcaac047460 con 0x7fcabc070d60 2024-09-10T10:29:02.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.579+0000 7fcab3fff640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fca9c06be30 0x7fca9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:02.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.579+0000 7fcab3fff640 1 -- 172.21.15.120:0/349054123 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6986+0+0 (secure 0 0 0) 0x7fcaac0ac010 con 0x7fcabc070d60 2024-09-10T10:29:02.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.579+0000 7fcab259c640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fca9c06be30 0x7fca9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:02.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.579+0000 7fcab259c640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fca9c06be30 0x7fca9c06e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0cc380 tx=0x7fcaa4007b90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:02.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.583+0000 7fcab3fff640 1 -- 172.21.15.120:0/349054123 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcabc06fd70 con 0x7fcabc070d60 2024-09-10T10:29:02.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:02.762+0000 7fcac0a00640 1 -- 172.21.15.120:0/349054123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} v 0) -- 0x7fcabc0ce240 con 0x7fcabc070d60 2024-09-10T10:29:03.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.461+0000 7fcab3fff640 1 -- 172.21.15.120:0/349054123 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]=0 set osd.6 primary-affinity to 1 (8655362) v76) ==== 133+0+0 (secure 0 0 0) 0x7fcaac032e20 con 0x7fcabc070d60 2024-09-10T10:29:03.464 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-10T10:29:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.464+0000 7fcac0a00640 1 -- 172.21.15.120:0/349054123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fca9c06be30 msgr2=0x7fca9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.464+0000 7fcac0a00640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fca9c06be30 0x7fca9c06e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0cc380 tx=0x7fcaa4007b90 comp rx=0 tx=0).stop 2024-09-10T10:29:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.464+0000 7fcac0a00640 1 -- 172.21.15.120:0/349054123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc070d60 msgr2=0x7fcabc1e43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.464+0000 7fcac0a00640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc070d60 0x7fcabc1e43e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcaac000c00 tx=0x7fcaac0046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.464+0000 7fcac0a00640 1 -- 172.21.15.120:0/349054123 shutdown_connections 2024-09-10T10:29:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.464+0000 7fcac0a00640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fca9c06be30 0x7fca9c06e2f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.464+0000 7fcac0a00640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcabc154be0 0x7fcabc1f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.464+0000 7fcac0a00640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcabc0c8d60 0x7fcabc1e6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.464+0000 7fcac0a00640 1 --2- 172.21.15.120:0/349054123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcabc070d60 0x7fcabc1e43e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.464+0000 7fcac0a00640 1 -- 172.21.15.120:0/349054123 >> 172.21.15.120:0/349054123 conn(0x7fcabc06c240 msgr2=0x7fcabc153580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.465+0000 7fcac0a00640 1 -- 172.21.15.120:0/349054123 shutdown_connections 2024-09-10T10:29:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.465+0000 7fcac0a00640 1 -- 172.21.15.120:0/349054123 wait complete. 2024-09-10T10:29:03.481 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-10T10:29:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.583+0000 7fb12a52c640 1 Processor -- start 2024-09-10T10:29:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.583+0000 7fb12a52c640 1 -- start start 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.584+0000 7fb12a52c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb124072d20 0x7fb1240c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.584+0000 7fb12a52c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1240c8d50 0x7fb1240c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.584+0000 7fb12a52c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1240c8050 con 0x7fb124073100 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.584+0000 7fb12a52c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb1240c5780 con 0x7fb124072d20 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.584+0000 7fb12a52c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1240c5a60 con 0x7fb1240c8d50 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.584+0000 7fb128ac9640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fb124073100 0x7fb124151550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:52248/0 (socket says 172.21.15.120:52248) 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.584+0000 7fb128ac9640 1 -- 172.21.15.120:0/3073355370 learned_addr learned my addr 172.21.15.120:0/3073355370 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb123fff640 1 --2- 172.21.15.120:0/3073355370 >> v2:172.21.15.134:3300/0 conn(0x7fb124072d20 0x7fb1240c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb1237fe640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1240c8d50 0x7fb1240c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3897300709 0 0) 0x7fb1240c8050 con 0x7fb124073100 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb10c003520 con 0x7fb124073100 2024-09-10T10:29:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3541147374 0 0) 0x7fb10c003520 con 0x7fb124073100 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1240c8d50 msgr2=0x7fb1240c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1240c8d50 0x7fb1240c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 >> v2:172.21.15.134:3300/0 conn(0x7fb124072d20 msgr2=0x7fb1240c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 --2- 172.21.15.120:0/3073355370 >> v2:172.21.15.134:3300/0 conn(0x7fb124072d20 0x7fb1240c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1240c5d40 con 0x7fb124073100 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7fb11c0019f0 con 0x7fb124073100 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 >> v1:172.21.15.120:6789/0 conn(0x7fb124073100 legacy=0x7fb124151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb10c0039b0 0x7fb10c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb10c004800 0x7fb10c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1240c8d50 0x7fb10c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1240c5780 con 0x7fb10c004800 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1240c5a60 con 0x7fb10c0039b0 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb10c00b9f0 con 0x7fb1240c8d50 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb123fff640 1 --2- 172.21.15.120:0/3073355370 >> v2:172.21.15.134:3300/0 conn(0x7fb124072d20 0x7fb1240c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:29:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb1237fe640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb10c004800 0x7fb10c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb128ac9640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1240c8d50 0x7fb10c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb123fff640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb10c0039b0 0x7fb10c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb1237fe640 1 -- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1240c8d50 msgr2=0x7fb10c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb1237fe640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1240c8d50 0x7fb10c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb1237fe640 1 -- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb10c0039b0 msgr2=0x7fb10c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb1237fe640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb10c0039b0 0x7fb10c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb1237fe640 1 -- 172.21.15.120:0/3073355370 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c003520 con 0x7fb10c004800 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb128ac9640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1240c8d50 0x7fb10c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.585+0000 7fb123fff640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb10c0039b0 0x7fb10c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.586+0000 7fb1237fe640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb10c004800 0x7fb10c004c80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb118009ac0 tx=0x7fb10c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68af18cedc508ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.586+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118036070 con 0x7fb10c004800 2024-09-10T10:29:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.586+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb118031400 con 0x7fb10c004800 2024-09-10T10:29:03.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.586+0000 7fb122ffd640 1 -- 172.21.15.120:0/3073355370 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118009690 con 0x7fb10c004800 2024-09-10T10:29:03.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.587+0000 7fb12a52c640 1 -- 172.21.15.120:0/3073355370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb10c004800 msgr2=0x7fb10c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:03.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.587+0000 7fb12a52c640 1 --2- 172.21.15.120:0/3073355370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb10c004800 0x7fb10c004c80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb118009ac0 tx=0x7fb10c005830 comp rx=0 tx=0).stop 2024-09-10T10:29:03.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.587+0000 7fb128ac9640 1 -- 172.21.15.120:0/3073355370 reap_dead start 2024-09-10T10:29:03.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.587+0000 7fb12a52c640 1 -- 172.21.15.120:0/3073355370 shutdown_connections 2024-09-10T10:29:03.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.587+0000 7fb12a52c640 1 -- 172.21.15.120:0/3073355370 >> 172.21.15.120:0/3073355370 conn(0x7fb12406e670 msgr2=0x7fb12406ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:03.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.587+0000 7fb12a52c640 1 -- 172.21.15.120:0/3073355370 shutdown_connections 2024-09-10T10:29:03.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.587+0000 7fb12a52c640 1 -- 172.21.15.120:0/3073355370 wait complete. 2024-09-10T10:29:03.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.588+0000 7fb12a52c640 1 Processor -- start 2024-09-10T10:29:03.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.588+0000 7fb12a52c640 1 -- start start 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb12a52c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb10c004800 0x7fb1241ed170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb12a52c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb124154e60 0x7fb1241f1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb12a52c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb124151590 0x7fb12413aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb12a52c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1240c72c0 con 0x7fb124151590 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb12a52c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1240c5480 con 0x7fb124154e60 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb12a52c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1240c65e0 con 0x7fb10c004800 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb128ac9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb124151590 0x7fb12413aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb128ac9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb124151590 0x7fb12413aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32846/0 (socket says 172.21.15.120:32846) 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb128ac9640 1 -- 172.21.15.120:0/2126462030 learned_addr learned my addr 172.21.15.120:0/2126462030 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb1237fe640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb124154e60 0x7fb1241f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb123fff640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb10c004800 0x7fb1241ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb128ac9640 1 -- 172.21.15.120:0/2126462030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb10c004800 msgr2=0x7fb1241ed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb128ac9640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb10c004800 0x7fb1241ed170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb128ac9640 1 -- 172.21.15.120:0/2126462030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb124154e60 msgr2=0x7fb1241f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb128ac9640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb124154e60 0x7fb1241f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb128ac9640 1 -- 172.21.15.120:0/2126462030 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb124152690 con 0x7fb124151590 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb123fff640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb10c004800 0x7fb1241ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb128ac9640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb124151590 0x7fb12413aae0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb11c000f80 tx=0x7fb11c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb120ff9640 1 -- 172.21.15.120:0/2126462030 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c007a80 con 0x7fb124151590 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb120ff9640 1 -- 172.21.15.120:0/2126462030 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb11c00ed40 con 0x7fb124151590 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb120ff9640 1 -- 172.21.15.120:0/2126462030 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c014670 con 0x7fb124151590 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb12a52c640 1 -- 172.21.15.120:0/2126462030 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb124152970 con 0x7fb124151590 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.589+0000 7fb12a52c640 1 -- 172.21.15.120:0/2126462030 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1241f0c10 con 0x7fb124151590 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.590+0000 7fb12a52c640 1 -- 172.21.15.120:0/2126462030 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1240c6cc0 con 0x7fb124151590 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.591+0000 7fb120ff9640 1 -- 172.21.15.120:0/2126462030 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb11c014810 con 0x7fb124151590 2024-09-10T10:29:03.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.591+0000 7fb120ff9640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb11006be60 0x7fb11006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:03.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.591+0000 7fb120ff9640 1 -- 172.21.15.120:0/2126462030 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6986+0+0 (secure 0 0 0) 0x7fb11c08b240 con 0x7fb124151590 2024-09-10T10:29:03.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.591+0000 7fb123fff640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb11006be60 0x7fb11006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:03.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.592+0000 7fb123fff640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb11006be60 0x7fb11006e320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb114009c90 tx=0x7fb114008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:03.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.596+0000 7fb120ff9640 1 -- 172.21.15.120:0/2126462030 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb11c059660 con 0x7fb124151590 2024-09-10T10:29:03.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:03.776+0000 7fb12a52c640 1 -- 172.21.15.120:0/2126462030 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7fb1240c4f80 con 0x7fb124151590 2024-09-10T10:29:04.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.469+0000 7fb120ff9640 1 -- 172.21.15.120:0/2126462030 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v77) ==== 113+0+0 (secure 0 0 0) 0x7fb11c058f60 con 0x7fb124151590 2024-09-10T10:29:04.471 INFO:teuthology.orchestra.run.smithi120.stderr:reweighted osd.7 to 1 (10000) 2024-09-10T10:29:04.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 -- 172.21.15.120:0/2126462030 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb11006be60 msgr2=0x7fb11006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:04.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb11006be60 0x7fb11006e320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb114009c90 tx=0x7fb114008040 comp rx=0 tx=0).stop 2024-09-10T10:29:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 -- 172.21.15.120:0/2126462030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb124151590 msgr2=0x7fb12413aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb124151590 0x7fb12413aae0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb11c000f80 tx=0x7fb11c00e530 comp rx=0 tx=0).stop 2024-09-10T10:29:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 -- 172.21.15.120:0/2126462030 shutdown_connections 2024-09-10T10:29:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb11006be60 0x7fb11006e320 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb124151590 0x7fb12413aae0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb124154e60 0x7fb1241f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 --2- 172.21.15.120:0/2126462030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb10c004800 0x7fb1241ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 -- 172.21.15.120:0/2126462030 >> 172.21.15.120:0/2126462030 conn(0x7fb12406e670 msgr2=0x7fb12414f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 -- 172.21.15.120:0/2126462030 shutdown_connections 2024-09-10T10:29:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.472+0000 7fb12a52c640 1 -- 172.21.15.120:0/2126462030 wait complete. 2024-09-10T10:29:04.488 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-10T10:29:04.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.589+0000 7f7c009fd640 1 Processor -- start 2024-09-10T10:29:04.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.590+0000 7f7c009fd640 1 -- start start 2024-09-10T10:29:04.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.590+0000 7f7c009fd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7bfc0cc0a0 0x7f7bfc0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:04.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.590+0000 7f7c009fd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7bfc0d0630 0x7f7bfc0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.590+0000 7f7c009fd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc15e660 0x7f7bfc15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.590+0000 7f7c009fd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7bfc15e1d0 con 0x7f7bfc0d0630 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.590+0000 7f7c009fd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7bfc15f030 con 0x7f7bfc0cc0a0 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.590+0000 7f7c009fd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc15ba20 con 0x7f7bfc15e660 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.590+0000 7f7bf3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7bfc0d0630 0x7f7bfc0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bf3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7bfc0d0630 0x7f7bfc0d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32862/0 (socket says 172.21.15.120:32862) 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bf3fff640 1 -- 172.21.15.120:0/4119044942 learned_addr learned my addr 172.21.15.120:0/4119044942 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bfa59c640 1 --2- 172.21.15.120:0/4119044942 >> v2:172.21.15.134:3300/0 conn(0x7f7bfc0cc0a0 0x7f7bfc0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bfad9d640 1 --2- 172.21.15.120:0/4119044942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc15e660 0x7f7bfc15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bf3fff640 1 -- 172.21.15.120:0/4119044942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc15e660 msgr2=0x7f7bfc15eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:04.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bf3fff640 1 --2- 172.21.15.120:0/4119044942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc15e660 0x7f7bfc15eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bf3fff640 1 -- 172.21.15.120:0/4119044942 >> v2:172.21.15.134:3300/0 conn(0x7f7bfc0cc0a0 msgr2=0x7f7bfc0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bf3fff640 1 --2- 172.21.15.120:0/4119044942 >> v2:172.21.15.134:3300/0 conn(0x7f7bfc0cc0a0 0x7f7bfc0cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bf3fff640 1 -- 172.21.15.120:0/4119044942 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc15aea0 con 0x7f7bfc0d0630 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bfa59c640 1 --2- 172.21.15.120:0/4119044942 >> v2:172.21.15.134:3300/0 conn(0x7f7bfc0cc0a0 0x7f7bfc0cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bfad9d640 1 --2- 172.21.15.120:0/4119044942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc15e660 0x7f7bfc15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.591+0000 7f7bf3fff640 1 --2- 172.21.15.120:0/4119044942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc0d0630 0x7f7bfc0d0a50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0caca0 tx=0x7f7be402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5acee5e81fa0a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7bf9d9b640 1 -- 172.21.15.120:0/4119044942 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4037070 con 0x7f7bfc0d0630 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7bf9d9b640 1 -- 172.21.15.120:0/4119044942 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be4032440 con 0x7f7bfc0d0630 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7bf9d9b640 1 -- 172.21.15.120:0/4119044942 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4032810 con 0x7f7bfc0d0630 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7c009fd640 1 -- 172.21.15.120:0/4119044942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc0d0630 msgr2=0x7f7bfc0d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7c009fd640 1 --2- 172.21.15.120:0/4119044942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc0d0630 0x7f7bfc0d0a50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0caca0 tx=0x7f7be402a270 comp rx=0 tx=0).stop 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7c009fd640 1 -- 172.21.15.120:0/4119044942 shutdown_connections 2024-09-10T10:29:04.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7c009fd640 1 --2- 172.21.15.120:0/4119044942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc15e660 0x7f7bfc15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7c009fd640 1 --2- 172.21.15.120:0/4119044942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc0d0630 0x7f7bfc0d0a50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7c009fd640 1 --2- 172.21.15.120:0/4119044942 >> v2:172.21.15.134:3300/0 conn(0x7f7bfc0cc0a0 0x7f7bfc0cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7c009fd640 1 -- 172.21.15.120:0/4119044942 >> 172.21.15.120:0/4119044942 conn(0x7f7bfc06d560 msgr2=0x7f7bfc073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:04.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7c009fd640 1 -- 172.21.15.120:0/4119044942 shutdown_connections 2024-09-10T10:29:04.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.592+0000 7f7c009fd640 1 -- 172.21.15.120:0/4119044942 wait complete. 2024-09-10T10:29:04.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7c009fd640 1 Processor -- start 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7c009fd640 1 -- start start 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7c009fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc0cc0a0 0x7f7bfc1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7c009fd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc0d0630 0x7f7bfc0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7c009fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc15e660 0x7f7bfc0b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7c009fd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc15b740 con 0x7f7bfc15e660 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7c009fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc15b460 con 0x7f7bfc0cc0a0 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7c009fd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc0cbde0 con 0x7f7bfc0d0630 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7bfad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc15e660 0x7f7bfc0b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7bfad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc15e660 0x7f7bfc0b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32874/0 (socket says 172.21.15.120:32874) 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7bfad9d640 1 -- 172.21.15.120:0/2150990496 learned_addr learned my addr 172.21.15.120:0/2150990496 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7bf3fff640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc0d0630 0x7f7bfc0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.594+0000 7f7bfa59c640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc0cc0a0 0x7f7bfc1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7bfad9d640 1 -- 172.21.15.120:0/2150990496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc0d0630 msgr2=0x7f7bfc0b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7bfad9d640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc0d0630 0x7f7bfc0b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7bfad9d640 1 -- 172.21.15.120:0/2150990496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc0cc0a0 msgr2=0x7f7bfc1425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7bfad9d640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc0cc0a0 0x7f7bfc1425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7bfad9d640 1 -- 172.21.15.120:0/2150990496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc15ddc0 con 0x7f7bfc15e660 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7bfa59c640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc0cc0a0 0x7f7bfc1425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7bfad9d640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc15e660 0x7f7bfc0b5d40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400cd50 tx=0x7f7bf400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7bf2ffd640 1 -- 172.21.15.120:0/2150990496 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400f680 con 0x7f7bfc15e660 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7bf2ffd640 1 -- 172.21.15.120:0/2150990496 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf400f840 con 0x7f7bfc15e660 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7bf2ffd640 1 -- 172.21.15.120:0/2150990496 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400fc30 con 0x7f7bfc15e660 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7c009fd640 1 -- 172.21.15.120:0/2150990496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bfc153850 con 0x7f7bfc15e660 2024-09-10T10:29:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.595+0000 7f7c009fd640 1 -- 172.21.15.120:0/2150990496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bfc146c00 con 0x7f7bfc15e660 2024-09-10T10:29:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.596+0000 7f7c009fd640 1 -- 172.21.15.120:0/2150990496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bfc15aaf0 con 0x7f7bfc15e660 2024-09-10T10:29:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.596+0000 7f7bf2ffd640 1 -- 172.21.15.120:0/2150990496 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bf40213f0 con 0x7f7bfc15e660 2024-09-10T10:29:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.597+0000 7f7bf2ffd640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bdc06be60 0x7f7bdc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.597+0000 7f7bf2ffd640 1 -- 172.21.15.120:0/2150990496 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6986+0+0 (secure 0 0 0) 0x7f7bf408c020 con 0x7f7bfc15e660 2024-09-10T10:29:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.597+0000 7f7bfa59c640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bdc06be60 0x7f7bdc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.597+0000 7f7bfa59c640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bdc06be60 0x7f7bdc06e320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7bec006b10 tx=0x7f7bec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.601+0000 7f7bf2ffd640 1 -- 172.21.15.120:0/2150990496 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7bf4014030 con 0x7f7bfc15e660 2024-09-10T10:29:04.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:04.778+0000 7f7c009fd640 1 -- 172.21.15.120:0/2150990496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} v 0) -- 0x7f7bfc153c50 con 0x7f7bfc15e660 2024-09-10T10:29:05.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.476+0000 7f7bf2ffd640 1 -- 172.21.15.120:0/2150990496 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]=0 set osd.7 primary-affinity to 1 (8655362) v78) ==== 133+0+0 (secure 0 0 0) 0x7f7bf405a560 con 0x7f7bfc15e660 2024-09-10T10:29:05.478 INFO:teuthology.orchestra.run.smithi120.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-10T10:29:05.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 -- 172.21.15.120:0/2150990496 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bdc06be60 msgr2=0x7f7bdc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bdc06be60 0x7f7bdc06e320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7bec006b10 tx=0x7f7bec008040 comp rx=0 tx=0).stop 2024-09-10T10:29:05.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 -- 172.21.15.120:0/2150990496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc15e660 msgr2=0x7f7bfc0b5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc15e660 0x7f7bfc0b5d40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400cd50 tx=0x7f7bf400a650 comp rx=0 tx=0).stop 2024-09-10T10:29:05.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 -- 172.21.15.120:0/2150990496 shutdown_connections 2024-09-10T10:29:05.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7bdc06be60 0x7f7bdc06e320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bfc15e660 0x7f7bfc0b5d40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bfc0d0630 0x7f7bfc0b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 --2- 172.21.15.120:0/2150990496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc0cc0a0 0x7f7bfc1425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 -- 172.21.15.120:0/2150990496 >> 172.21.15.120:0/2150990496 conn(0x7f7bfc06d560 msgr2=0x7f7bfc1e3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:05.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 -- 172.21.15.120:0/2150990496 shutdown_connections 2024-09-10T10:29:05.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.479+0000 7f7c009fd640 1 -- 172.21.15.120:0/2150990496 wait complete. 2024-09-10T10:29:05.496 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-10T10:29:05.496 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-10T10:29:05.496 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-10T10:29:05.496 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-10T10:29:05.496 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T10:29:05.496 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:29:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.598+0000 7f4442b4b640 1 Processor -- start 2024-09-10T10:29:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.598+0000 7f4442b4b640 1 -- start start 2024-09-10T10:29:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.599+0000 7f4442b4b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f443c1582d0 0x7f443c0c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.599+0000 7f4442b4b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f443c15afa0 con 0x7f443c157ef0 2024-09-10T10:29:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.599+0000 7f4442b4b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f443c15b280 con 0x7f443c072e00 2024-09-10T10:29:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.599+0000 7f4442b4b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f443c072970 con 0x7f443c1582d0 2024-09-10T10:29:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.599+0000 7f44408e7640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f443c157ef0 0x7f443c0c9a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:52254/0 (socket says 172.21.15.120:52254) 2024-09-10T10:29:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.599+0000 7f44408e7640 1 -- 172.21.15.120:0/2874878155 learned_addr learned my addr 172.21.15.120:0/2874878155 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443bfff640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f443c1582d0 0x7f443c0c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 180466178 0 0) 0x7f443c15afa0 con 0x7f443c157ef0 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4424003520 con 0x7f443c157ef0 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 99118208 0 0) 0x7f4424003520 con 0x7f443c157ef0 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f443c1582d0 msgr2=0x7f443c0c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f443c1582d0 0x7f443c0c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 >> v1:172.21.15.134:6789/0 conn(0x7f443c072e00 legacy=0x7f443c161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c15b560 con 0x7f443c157ef0 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f442c001670 con 0x7f443c157ef0 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 >> v1:172.21.15.120:6789/0 conn(0x7f443c157ef0 legacy=0x7f443c0c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44240039e0 0x7f4424003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f443c1582d0 0x7f4424004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4424004d40 0x7f44240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f443c072970 con 0x7f44240039e0 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f442400b9c0 con 0x7f4424004d40 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f442400bbc0 con 0x7f443c1582d0 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f44408e7640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44240039e0 0x7f4424003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f443bfff640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f443c1582d0 0x7f4424004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f44410e8640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4424004d40 0x7f44240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f44408e7640 1 -- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f443c1582d0 msgr2=0x7f4424004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f44408e7640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f443c1582d0 0x7f4424004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f44408e7640 1 -- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4424004d40 msgr2=0x7f44240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f44408e7640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4424004d40 0x7f44240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.600+0000 7f44408e7640 1 -- 172.21.15.120:0/2874878155 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4424003520 con 0x7f44240039e0 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.601+0000 7f44408e7640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44240039e0 0x7f4424003dc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f442c00e4a0 tx=0x7f4424005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9ec1d9211719ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.601+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c007d50 con 0x7f44240039e0 2024-09-10T10:29:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.601+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442c005af0 con 0x7f44240039e0 2024-09-10T10:29:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.601+0000 7f443b7fe640 1 -- 172.21.15.120:0/2874878155 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c007070 con 0x7f44240039e0 2024-09-10T10:29:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.602+0000 7f4442b4b640 1 -- 172.21.15.120:0/2874878155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44240039e0 msgr2=0x7f4424003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.602+0000 7f4442b4b640 1 --2- 172.21.15.120:0/2874878155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44240039e0 0x7f4424003dc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f442c00e4a0 tx=0x7f4424005830 comp rx=0 tx=0).stop 2024-09-10T10:29:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.602+0000 7f44410e8640 1 -- 172.21.15.120:0/2874878155 reap_dead start 2024-09-10T10:29:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.602+0000 7f4442b4b640 1 -- 172.21.15.120:0/2874878155 shutdown_connections 2024-09-10T10:29:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.602+0000 7f4442b4b640 1 -- 172.21.15.120:0/2874878155 >> 172.21.15.120:0/2874878155 conn(0x7f443c0703e0 msgr2=0x7f443c0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.602+0000 7f4442b4b640 1 -- 172.21.15.120:0/2874878155 shutdown_connections 2024-09-10T10:29:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.602+0000 7f4442b4b640 1 -- 172.21.15.120:0/2874878155 wait complete. 2024-09-10T10:29:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.603+0000 7f4442b4b640 1 Processor -- start 2024-09-10T10:29:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.603+0000 7f4442b4b640 1 -- start start 2024-09-10T10:29:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f4442b4b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44240039e0 0x7f443c14c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f4442b4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f443c0cc320 0x7f443c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f4442b4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f443c1ef580 0x7f443c1efa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f4442b4b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f443c15bb20 con 0x7f443c0cc320 2024-09-10T10:29:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f4442b4b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f443c0caa80 con 0x7f443c1ef580 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f4442b4b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f443c15ac60 con 0x7f44240039e0 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f443bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f443c0cc320 0x7f443c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f443bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f443c0cc320 0x7f443c15a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32890/0 (socket says 172.21.15.120:32890) 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f443bfff640 1 -- 172.21.15.120:0/858008121 learned_addr learned my addr 172.21.15.120:0/858008121 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f44408e7640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44240039e0 0x7f443c14c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f44410e8640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f443c1ef580 0x7f443c1efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f443bfff640 1 -- 172.21.15.120:0/858008121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44240039e0 msgr2=0x7f443c14c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f443bfff640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44240039e0 0x7f443c14c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f443bfff640 1 -- 172.21.15.120:0/858008121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f443c1ef580 msgr2=0x7f443c1efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f443bfff640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f443c1ef580 0x7f443c1efa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f443bfff640 1 -- 172.21.15.120:0/858008121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c155bc0 con 0x7f443c0cc320 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.604+0000 7f443bfff640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f443c0cc320 0x7f443c15a350 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4430011910 tx=0x7f4430011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.605+0000 7f44397fa640 1 -- 172.21.15.120:0/858008121 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443000faf0 con 0x7f443c0cc320 2024-09-10T10:29:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.605+0000 7f44397fa640 1 -- 172.21.15.120:0/858008121 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443000fc90 con 0x7f443c0cc320 2024-09-10T10:29:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.605+0000 7f4442b4b640 1 -- 172.21.15.120:0/858008121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c155600 con 0x7f443c0cc320 2024-09-10T10:29:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.605+0000 7f44397fa640 1 -- 172.21.15.120:0/858008121 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430014440 con 0x7f443c0cc320 2024-09-10T10:29:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.605+0000 7f4442b4b640 1 -- 172.21.15.120:0/858008121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c156b10 con 0x7f443c0cc320 2024-09-10T10:29:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.606+0000 7f4442b4b640 1 -- 172.21.15.120:0/858008121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c15b280 con 0x7f443c0cc320 2024-09-10T10:29:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.606+0000 7f44397fa640 1 -- 172.21.15.120:0/858008121 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4430012040 con 0x7f443c0cc320 2024-09-10T10:29:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.606+0000 7f44397fa640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f442806be60 0x7f442806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.607+0000 7f44397fa640 1 -- 172.21.15.120:0/858008121 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f443008b7f0 con 0x7f443c0cc320 2024-09-10T10:29:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.607+0000 7f44408e7640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f442806be60 0x7f442806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.607+0000 7f44408e7640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f442806be60 0x7f442806e320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f442c035000 tx=0x7f442c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.611+0000 7f44397fa640 1 -- 172.21.15.120:0/858008121 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4430019030 con 0x7f443c0cc320 2024-09-10T10:29:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.778+0000 7f4442b4b640 1 -- 172.21.15.120:0/858008121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f443c0741e0 con 0x7f443c0cc320 2024-09-10T10:29:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.779+0000 7f44397fa640 1 -- 172.21.15.120:0/858008121 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+19282 (secure 0 0 0) 0x7f4430059660 con 0x7f443c0cc320 2024-09-10T10:29:05.782 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:05.782 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":78,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:29:05.476388+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2c","osds":[1,7]},{"pgid":"3.34","osds":[3,1]}],"pg_upmap_items":[{"pgid":"2.1d","mappings":[{"from":6,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":6}]},{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":6}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":3,"to":6},{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2},{"from":3,"to":0}]},{"pgid":"4.1e","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:29:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 -- 172.21.15.120:0/858008121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f442806be60 msgr2=0x7f442806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f442806be60 0x7f442806e320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f442c035000 tx=0x7f442c030000 comp rx=0 tx=0).stop 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 -- 172.21.15.120:0/858008121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f443c0cc320 msgr2=0x7f443c15a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f443c0cc320 0x7f443c15a350 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4430011910 tx=0x7f4430011dd0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 -- 172.21.15.120:0/858008121 shutdown_connections 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f442806be60 0x7f442806e320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f443c1ef580 0x7f443c1efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f443c0cc320 0x7f443c15a350 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 --2- 172.21.15.120:0/858008121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44240039e0 0x7f443c14c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 -- 172.21.15.120:0/858008121 >> 172.21.15.120:0/858008121 conn(0x7f443c0703e0 msgr2=0x7f443c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 -- 172.21.15.120:0/858008121 shutdown_connections 2024-09-10T10:29:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.781+0000 7f4442b4b640 1 -- 172.21.15.120:0/858008121 wait complete. 2024-09-10T10:29:05.795 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T10:29:05.795 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:29:05.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.891+0000 7f471316d640 1 Processor -- start 2024-09-10T10:29:05.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.891+0000 7f471316d640 1 -- start start 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f471316d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f470c0ce5b0 0x7f470c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f471316d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0ce990 0x7f470c157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f471316d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f470c0c9a20 con 0x7f470c06eaa0 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f471316d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f470c0ca400 con 0x7f470c0ce5b0 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f471316d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f470c15ef70 con 0x7f470c0ce990 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f4710f09640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f470c0ce5b0 0x7f470c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f4710f09640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f470c0ce5b0 0x7f470c15ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32900/0 (socket says 172.21.15.120:32900) 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f4710f09640 1 -- 172.21.15.120:0/90622553 learned_addr learned my addr 172.21.15.120:0/90622553 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f4710f09640 1 -- 172.21.15.120:0/90622553 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0ce990 msgr2=0x7f470c157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f4710f09640 1 --2- 172.21.15.120:0/90622553 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0ce990 0x7f470c157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f4710f09640 1 -- 172.21.15.120:0/90622553 >> v1:172.21.15.134:6789/0 conn(0x7f470c06eaa0 legacy=0x7f470c15b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:05.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.892+0000 7f4710f09640 1 -- 172.21.15.120:0/90622553 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c06cf70 con 0x7f470c0ce5b0 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.893+0000 7f4710f09640 1 --2- 172.21.15.120:0/90622553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 0x7f470c15ea30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0098a0 tx=0x7f46fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf732381429d95c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.893+0000 7f470b7fe640 1 -- 172.21.15.120:0/90622553 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc037070 con 0x7f470c0ce5b0 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.893+0000 7f470b7fe640 1 -- 172.21.15.120:0/90622553 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46fc032440 con 0x7f470c0ce5b0 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.893+0000 7f470b7fe640 1 -- 172.21.15.120:0/90622553 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc032810 con 0x7f470c0ce5b0 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.894+0000 7f471316d640 1 -- 172.21.15.120:0/90622553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 msgr2=0x7f470c15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.894+0000 7f471316d640 1 --2- 172.21.15.120:0/90622553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 0x7f470c15ea30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0098a0 tx=0x7f46fc02a270 comp rx=0 tx=0).stop 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.894+0000 7f471316d640 1 -- 172.21.15.120:0/90622553 shutdown_connections 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.894+0000 7f471316d640 1 --2- 172.21.15.120:0/90622553 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0ce990 0x7f470c157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.894+0000 7f471316d640 1 --2- 172.21.15.120:0/90622553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 0x7f470c15ea30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.894+0000 7f471316d640 1 -- 172.21.15.120:0/90622553 >> 172.21.15.120:0/90622553 conn(0x7f470c06c2d0 msgr2=0x7f470c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.894+0000 7f471316d640 1 -- 172.21.15.120:0/90622553 shutdown_connections 2024-09-10T10:29:05.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.894+0000 7f471316d640 1 -- 172.21.15.120:0/90622553 wait complete. 2024-09-10T10:29:05.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.895+0000 7f471316d640 1 Processor -- start 2024-09-10T10:29:05.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.895+0000 7f471316d640 1 -- start start 2024-09-10T10:29:05.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f471316d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470c06eaa0 0x7f470c1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f471316d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 0x7f470c0c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f471316d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0ce990 0x7f470c14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f471316d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f470c15ad50 con 0x7f470c0ce5b0 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f471316d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f470c06cd20 con 0x7f470c06eaa0 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f471316d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f470c15b8f0 con 0x7f470c0ce990 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f470bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 0x7f470c0c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f470bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 0x7f470c0c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32904/0 (socket says 172.21.15.120:32904) 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f470bfff640 1 -- 172.21.15.120:0/1452301648 learned_addr learned my addr 172.21.15.120:0/1452301648 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f470bfff640 1 -- 172.21.15.120:0/1452301648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0ce990 msgr2=0x7f470c14f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f470bfff640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0ce990 0x7f470c14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f4710f09640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470c06eaa0 0x7f470c1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f470bfff640 1 -- 172.21.15.120:0/1452301648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470c06eaa0 msgr2=0x7f470c1521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f470bfff640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470c06eaa0 0x7f470c1521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.896+0000 7f470bfff640 1 -- 172.21.15.120:0/1452301648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c154f20 con 0x7f470c0ce5b0 2024-09-10T10:29:05.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.897+0000 7f470bfff640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 0x7f470c0c3940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f470000c7f0 tx=0x7f470000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.897+0000 7f47097fa640 1 -- 172.21.15.120:0/1452301648 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700016020 con 0x7f470c0ce5b0 2024-09-10T10:29:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.897+0000 7f47097fa640 1 -- 172.21.15.120:0/1452301648 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470000aa50 con 0x7f470c0ce5b0 2024-09-10T10:29:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.897+0000 7f471316d640 1 -- 172.21.15.120:0/1452301648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c15a530 con 0x7f470c0ce5b0 2024-09-10T10:29:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.897+0000 7f47097fa640 1 -- 172.21.15.120:0/1452301648 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470000ae40 con 0x7f470c0ce5b0 2024-09-10T10:29:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.897+0000 7f471316d640 1 -- 172.21.15.120:0/1452301648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c154960 con 0x7f470c0ce5b0 2024-09-10T10:29:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.898+0000 7f471316d640 1 -- 172.21.15.120:0/1452301648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470c0c8eb0 con 0x7f470c0ce5b0 2024-09-10T10:29:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.898+0000 7f47097fa640 1 -- 172.21.15.120:0/1452301648 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4700019020 con 0x7f470c0ce5b0 2024-09-10T10:29:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.899+0000 7f47097fa640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f46f806be30 0x7f46f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.899+0000 7f47097fa640 1 -- 172.21.15.120:0/1452301648 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f470008b4a0 con 0x7f470c0ce5b0 2024-09-10T10:29:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.899+0000 7f4710f09640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f46f806be30 0x7f46f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.899+0000 7f4710f09640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f46f806be30 0x7f46f806e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f46fc034000 tx=0x7f46fc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:05.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:05.904+0000 7f47097fa640 1 -- 172.21.15.120:0/1452301648 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f470c0c8eb0 con 0x7f470c0ce5b0 2024-09-10T10:29:06.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.071+0000 7f471316d640 1 -- 172.21.15.120:0/1452301648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f470c15a700 con 0x7f470c0ce5b0 2024-09-10T10:29:06.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.072+0000 7f47097fa640 1 -- 172.21.15.120:0/1452301648 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+19282 (secure 0 0 0) 0x7f4700020030 con 0x7f470c0ce5b0 2024-09-10T10:29:06.075 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:06.075 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":78,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:29:05.476388+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2c","osds":[1,7]},{"pgid":"3.34","osds":[3,1]}],"pg_upmap_items":[{"pgid":"2.1d","mappings":[{"from":6,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":6}]},{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":6}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":3,"to":6},{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2},{"from":3,"to":0}]},{"pgid":"4.1e","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:29:06.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.074+0000 7f471316d640 1 -- 172.21.15.120:0/1452301648 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f46f806be30 msgr2=0x7f46f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.074+0000 7f471316d640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f46f806be30 0x7f46f806e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f46fc034000 tx=0x7f46fc02e000 comp rx=0 tx=0).stop 2024-09-10T10:29:06.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.074+0000 7f471316d640 1 -- 172.21.15.120:0/1452301648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 msgr2=0x7f470c0c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.074+0000 7f471316d640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 0x7f470c0c3940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f470000c7f0 tx=0x7f470000ccb0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.074+0000 7f471316d640 1 -- 172.21.15.120:0/1452301648 shutdown_connections 2024-09-10T10:29:06.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.074+0000 7f471316d640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f46f806be30 0x7f46f806e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.074+0000 7f471316d640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0ce990 0x7f470c14f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.074+0000 7f471316d640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f470c0ce5b0 0x7f470c0c3940 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.074+0000 7f471316d640 1 --2- 172.21.15.120:0/1452301648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470c06eaa0 0x7f470c1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.074+0000 7f471316d640 1 -- 172.21.15.120:0/1452301648 >> 172.21.15.120:0/1452301648 conn(0x7f470c06c2d0 msgr2=0x7f470c0cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.075+0000 7f471316d640 1 -- 172.21.15.120:0/1452301648 shutdown_connections 2024-09-10T10:29:06.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.075+0000 7f471316d640 1 -- 172.21.15.120:0/1452301648 wait complete. 2024-09-10T10:29:06.088 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T10:29:06.089 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T10:29:06.089 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T10:29:06.090 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T10:29:06.090 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T10:29:06.091 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T10:29:06.091 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T10:29:06.091 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T10:29:06.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.255+0000 7f88d26c9640 1 Processor -- start 2024-09-10T10:29:06.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.255+0000 7f88d26c9640 1 -- start start 2024-09-10T10:29:06.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f2643fff640 1 Processor -- start 2024-09-10T10:29:06.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f88d26c9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f88cc150c10 0x7f88cc072d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f88d26c9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f88cc0c9650 0x7f88cc071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f88d26c9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88cc073260 con 0x7f88cc072410 2024-09-10T10:29:06.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f88d26c9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f88cc1569d0 con 0x7f88cc0c9650 2024-09-10T10:29:06.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f88d26c9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f88cc06dbd0 con 0x7f88cc150c10 2024-09-10T10:29:06.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f88cb7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f88cc0c9650 0x7f88cc071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f88cb7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f88cc0c9650 0x7f88cc071ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32912/0 (socket says 172.21.15.120:32912) 2024-09-10T10:29:06.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f88cb7fe640 1 -- 172.21.15.120:0/2816787055 learned_addr learned my addr 172.21.15.120:0/2816787055 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f88cbfff640 1 --2- 172.21.15.120:0/2816787055 >> v2:172.21.15.134:3300/0 conn(0x7f88cc150c10 0x7f88cc072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f2643fff640 1 -- start start 2024-09-10T10:29:06.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.256+0000 7f21e9ee6640 1 Processor -- start 2024-09-10T10:29:06.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.257+0000 7f88cbfff640 1 -- 172.21.15.120:0/2816787055 >> v2:172.21.15.120:3300/0 conn(0x7f88cc0c9650 msgr2=0x7f88cc071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.257+0000 7f88cbfff640 1 --2- 172.21.15.120:0/2816787055 >> v2:172.21.15.120:3300/0 conn(0x7f88cc0c9650 0x7f88cc071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.257+0000 7f88cbfff640 1 -- 172.21.15.120:0/2816787055 >> v1:172.21.15.120:6789/0 conn(0x7f88cc072410 legacy=0x7f88cc156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.257+0000 7f88cbfff640 1 -- 172.21.15.120:0/2816787055 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc06deb0 con 0x7f88cc150c10 2024-09-10T10:29:06.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.257+0000 7f2643fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f26440c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2643fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f26440c8430 0x7f2644071630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.257+0000 7f21e9ee6640 1 -- start start 2024-09-10T10:29:06.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2643fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2644071b70 0x7f26441522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f21e9ee6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f21e40cd2a0 0x7f21e40ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2642ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f26440c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2643fff640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f2644073d70 con 0x7f2644071b70 2024-09-10T10:29:06.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2643fff640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2644072000 con 0x7f26440c8430 2024-09-10T10:29:06.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2643fff640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f26440c77f0 con 0x7f26440c6ed0 2024-09-10T10:29:06.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2642ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f26440c72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:57246/0 (socket says 172.21.15.120:57246) 2024-09-10T10:29:06.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2642ffd640 1 -- 172.21.15.120:0/602184196 learned_addr learned my addr 172.21.15.120:0/602184196 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f21e9ee6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f21e40cd680 0x7f21e40ca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f26437fe640 1 --2- 172.21.15.120:0/602184196 >> v2:172.21.15.120:3300/0 conn(0x7f2644071b70 0x7f26441522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2642ffd640 1 -- 172.21.15.120:0/602184196 >> v2:172.21.15.134:3300/0 conn(0x7f26440c8430 msgr2=0x7f2644071630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:29:06.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2642ffd640 1 --2- 172.21.15.120:0/602184196 >> v2:172.21.15.134:3300/0 conn(0x7f26440c8430 0x7f2644071630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2642ffd640 1 -- 172.21.15.120:0/602184196 >> v2:172.21.15.120:3300/0 conn(0x7f2644071b70 msgr2=0x7f26441522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2642ffd640 1 --2- 172.21.15.120:0/602184196 >> v2:172.21.15.120:3300/0 conn(0x7f2644071b70 0x7f26441522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f2642ffd640 1 -- 172.21.15.120:0/602184196 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26441586e0 con 0x7f26440c6ed0 2024-09-10T10:29:06.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f21e37fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f21e40cd2a0 0x7f21e40ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f21e37fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f21e40cd2a0 0x7f21e40ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:46630/0 (socket says 172.21.15.120:46630) 2024-09-10T10:29:06.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f21e2ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f21e40cd680 0x7f21e40ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.258+0000 7f21e2ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f21e40cd680 0x7f21e40ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32932/0 (socket says 172.21.15.120:32932) 2024-09-10T10:29:06.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7f21e37fe640 1 -- 172.21.15.120:0/42402968 learned_addr learned my addr 172.21.15.120:0/42402968 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7f88cbfff640 1 --2- 172.21.15.120:0/2816787055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88cc150c10 0x7f88cc072d20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0cd270 tx=0x7f88c402cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b74a0387ddf612 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7fec88c58640 1 Processor -- start 2024-09-10T10:29:06.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7f21e9ee6640 1 -- 172.21.15.120:0/42402968 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e40c8cc0 con 0x7f21e415b2a0 2024-09-10T10:29:06.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7f21e9ee6640 1 -- 172.21.15.120:0/42402968 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f21e40c8a90 con 0x7f21e40cd680 2024-09-10T10:29:06.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7f21e9ee6640 1 -- 172.21.15.120:0/42402968 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f21e40c9b90 con 0x7f21e40cd2a0 2024-09-10T10:29:06.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7f21e37fe640 1 -- 172.21.15.120:0/42402968 >> v2:172.21.15.120:3300/0 conn(0x7f21e40cd680 msgr2=0x7f21e40ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7f21e37fe640 1 --2- 172.21.15.120:0/42402968 >> v2:172.21.15.120:3300/0 conn(0x7f21e40cd680 0x7f21e40ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7f21e37fe640 1 -- 172.21.15.120:0/42402968 >> v1:172.21.15.120:6789/0 conn(0x7f21e415b2a0 legacy=0x7f21e415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7f21e37fe640 1 -- 172.21.15.120:0/42402968 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e40c8f80 con 0x7f21e40cd2a0 2024-09-10T10:29:06.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7f88caffd640 1 -- 172.21.15.120:0/2816787055 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4034070 con 0x7f88cc150c10 2024-09-10T10:29:06.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7f88caffd640 1 -- 172.21.15.120:0/2816787055 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c400a4e0 con 0x7f88cc150c10 2024-09-10T10:29:06.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.259+0000 7fec88c58640 1 -- start start 2024-09-10T10:29:06.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7f2642ffd640 1 --2- 172.21.15.120:0/602184196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f26440c72b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26440ccee0 tx=0x7f263802a280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e5a5af222bd151 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7f88caffd640 1 -- 172.21.15.120:0/2816787055 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c400a8b0 con 0x7f88cc150c10 2024-09-10T10:29:06.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7fec88c58640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec840c7c30 0x7fec8415f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7fec88c58640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fec840c8010 0x7fec8406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7fec88c58640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec840c7100 con 0x7fec84067d10 2024-09-10T10:29:06.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7fec88c58640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fec8415fb00 con 0x7fec840c8010 2024-09-10T10:29:06.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7fec88c58640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec840c7660 con 0x7fec840c7c30 2024-09-10T10:29:06.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7f2641ffb640 1 -- 172.21.15.120:0/602184196 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638037070 con 0x7f26440c6ed0 2024-09-10T10:29:06.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7f2641ffb640 1 -- 172.21.15.120:0/602184196 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2638032440 con 0x7f26440c6ed0 2024-09-10T10:29:06.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7f2641ffb640 1 -- 172.21.15.120:0/602184196 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638032810 con 0x7f26440c6ed0 2024-09-10T10:29:06.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7fe7b8874640 1 Processor -- start 2024-09-10T10:29:06.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7fec83fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fec84067d10 0x7fec841587c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:41892/0 (socket says 172.21.15.120:41892) 2024-09-10T10:29:06.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7fec83fff640 1 -- 172.21.15.120:0/600811986 learned_addr learned my addr 172.21.15.120:0/600811986 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:06.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.260+0000 7f21e37fe640 1 --2- 172.21.15.120:0/42402968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e40cd2a0 0x7f21e40ca020 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cac40 tx=0x7f21d802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd0e74232b9eddf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f88d26c9640 1 -- 172.21.15.120:0/2816787055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88cc150c10 msgr2=0x7f88cc072d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f88d26c9640 1 --2- 172.21.15.120:0/2816787055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88cc150c10 0x7f88cc072d20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0cd270 tx=0x7f88c402cc00 comp rx=0 tx=0).stop 2024-09-10T10:29:06.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7fe7b8874640 1 -- start start 2024-09-10T10:29:06.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7fec837fe640 1 --2- 172.21.15.120:0/600811986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec840c7c30 0x7fec8415f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7fec82ffd640 1 --2- 172.21.15.120:0/600811986 >> v2:172.21.15.120:3300/0 conn(0x7fec840c8010 0x7fec8406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f21e27fc640 1 -- 172.21.15.120:0/42402968 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8037070 con 0x7f21e40cd2a0 2024-09-10T10:29:06.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f21e27fc640 1 -- 172.21.15.120:0/42402968 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d8032440 con 0x7f21e40cd2a0 2024-09-10T10:29:06.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f21e27fc640 1 -- 172.21.15.120:0/42402968 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8032810 con 0x7f21e40cd2a0 2024-09-10T10:29:06.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7f21e9ee6640 1 -- 172.21.15.120:0/42402968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e40cd2a0 msgr2=0x7f21e40ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7f21e9ee6640 1 --2- 172.21.15.120:0/42402968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e40cd2a0 0x7f21e40ca020 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cac40 tx=0x7f21d802a270 comp rx=0 tx=0).stop 2024-09-10T10:29:06.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f88d26c9640 1 -- 172.21.15.120:0/2816787055 shutdown_connections 2024-09-10T10:29:06.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f88d26c9640 1 --2- 172.21.15.120:0/2816787055 >> v2:172.21.15.120:3300/0 conn(0x7f88cc0c9650 0x7f88cc071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f88d26c9640 1 --2- 172.21.15.120:0/2816787055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88cc150c10 0x7f88cc072d20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f88d26c9640 1 -- 172.21.15.120:0/2816787055 >> 172.21.15.120:0/2816787055 conn(0x7f88cc06f3c0 msgr2=0x7f88cc06f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f88d26c9640 1 -- 172.21.15.120:0/2816787055 shutdown_connections 2024-09-10T10:29:06.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f88d26c9640 1 -- 172.21.15.120:0/2816787055 wait complete. 2024-09-10T10:29:06.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f2643fff640 1 -- 172.21.15.120:0/602184196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 msgr2=0x7f26440c72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f2643fff640 1 --2- 172.21.15.120:0/602184196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f26440c72b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26440ccee0 tx=0x7f263802a280 comp rx=0 tx=0).stop 2024-09-10T10:29:06.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f2643fff640 1 -- 172.21.15.120:0/602184196 shutdown_connections 2024-09-10T10:29:06.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f2643fff640 1 --2- 172.21.15.120:0/602184196 >> v2:172.21.15.120:3300/0 conn(0x7f2644071b70 0x7f26441522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f2643fff640 1 --2- 172.21.15.120:0/602184196 >> v2:172.21.15.134:3300/0 conn(0x7f26440c8430 0x7f2644071630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f2643fff640 1 --2- 172.21.15.120:0/602184196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f26440c72b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f2643fff640 1 -- 172.21.15.120:0/602184196 >> 172.21.15.120:0/602184196 conn(0x7f26440c89b0 msgr2=0x7f26440c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f2643fff640 1 -- 172.21.15.120:0/602184196 shutdown_connections 2024-09-10T10:29:06.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7f2643fff640 1 -- 172.21.15.120:0/602184196 wait complete. 2024-09-10T10:29:06.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fe7b8874640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe7b40cc0c0 0x7fe7b415aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7fec82ffd640 1 -- 172.21.15.120:0/600811986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec840c7c30 msgr2=0x7fec8415f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7fec82ffd640 1 --2- 172.21.15.120:0/600811986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec840c7c30 0x7fec8415f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7fec82ffd640 1 -- 172.21.15.120:0/600811986 >> v1:172.21.15.134:6789/0 conn(0x7fec84067d10 legacy=0x7fec841587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.261+0000 7fec82ffd640 1 -- 172.21.15.120:0/600811986 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8406bd40 con 0x7fec840c8010 2024-09-10T10:29:06.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fe7b8874640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe7b40cb2f0 0x7fe7b40cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fe7b8874640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7b415e9d0 0x7fe7b40cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7f23584af640 1 Processor -- start 2024-09-10T10:29:06.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fe7b259c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe7b40cc0c0 0x7fe7b415aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fe7abfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe7b40cb2f0 0x7fe7b40cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fe7abfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe7b40cb2f0 0x7fe7b40cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:46652/0 (socket says 172.21.15.120:46652) 2024-09-10T10:29:06.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fe7abfff640 1 -- 172.21.15.120:0/4291170610 learned_addr learned my addr 172.21.15.120:0/4291170610 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fe7b259c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe7b40cc0c0 0x7fe7b415aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32954/0 (socket says 172.21.15.120:32954) 2024-09-10T10:29:06.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fe7b2d9d640 1 --2- 172.21.15.120:0/4291170610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7b415e9d0 0x7fe7b40cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fec82ffd640 1 --2- 172.21.15.120:0/600811986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840c8010 0x7fec8406c7e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fec740097d0 tx=0x7fec7402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7f5194378d55d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7fe7b8874640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b40cbd00 con 0x7fe7b40cc0c0 2024-09-10T10:29:06.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7fe7b8874640 1 -- 172.21.15.120:0/4291170610 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b415ba70 con 0x7fe7b40cb2f0 2024-09-10T10:29:06.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.262+0000 7f23584af640 1 -- start start 2024-09-10T10:29:06.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7fe7b8874640 1 -- 172.21.15.120:0/4291170610 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b415e540 con 0x7fe7b415e9d0 2024-09-10T10:29:06.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7f88d26c9640 1 Processor -- start 2024-09-10T10:29:06.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7f21e9ee6640 1 -- 172.21.15.120:0/42402968 shutdown_connections 2024-09-10T10:29:06.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7f21e9ee6640 1 --2- 172.21.15.120:0/42402968 >> v2:172.21.15.120:3300/0 conn(0x7f21e40cd680 0x7f21e40ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7f21e9ee6640 1 --2- 172.21.15.120:0/42402968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e40cd2a0 0x7f21e40ca020 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7f21e9ee6640 1 -- 172.21.15.120:0/42402968 >> 172.21.15.120:0/42402968 conn(0x7f21e406cd20 msgr2=0x7f21e406d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7fefa0ca1640 1 Processor -- start 2024-09-10T10:29:06.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7fefa0ca1640 1 -- start start 2024-09-10T10:29:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7f21e9ee6640 1 -- 172.21.15.120:0/42402968 shutdown_connections 2024-09-10T10:29:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7f88d26c9640 1 -- start start 2024-09-10T10:29:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f01c9d8d640 1 Processor -- start 2024-09-10T10:29:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f23584af640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f235015caa0 0x7f235015f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f23584af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23500b7410 0x7f235015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f2643fff640 1 Processor -- start 2024-09-10T10:29:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7fec827fc640 1 -- 172.21.15.120:0/600811986 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74037070 con 0x7fec840c8010 2024-09-10T10:29:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7fec827fc640 1 -- 172.21.15.120:0/600811986 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec74032440 con 0x7fec840c8010 2024-09-10T10:29:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7fec827fc640 1 -- 172.21.15.120:0/600811986 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74032850 con 0x7fec840c8010 2024-09-10T10:29:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f2643fff640 1 -- start start 2024-09-10T10:29:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7fe7abfff640 1 -- 172.21.15.120:0/4291170610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7b415e9d0 msgr2=0x7fe7b40cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7fe7abfff640 1 --2- 172.21.15.120:0/4291170610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7b415e9d0 0x7fe7b40cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7fe7abfff640 1 -- 172.21.15.120:0/4291170610 >> v2:172.21.15.120:3300/0 conn(0x7fe7b40cc0c0 msgr2=0x7fe7b415aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7fe7abfff640 1 --2- 172.21.15.120:0/4291170610 >> v2:172.21.15.120:3300/0 conn(0x7fe7b40cc0c0 0x7fe7b415aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7fe7abfff640 1 -- 172.21.15.120:0/4291170610 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b415b1d0 con 0x7fe7b40cb2f0 2024-09-10T10:29:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7fe7abfff640 1 --2- 172.21.15.120:0/4291170610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b40cb2f0 0x7fe7b40cc4a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40ce570 tx=0x7fe79c02a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ed94ea31e177f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f01c9d8d640 1 -- start start 2024-09-10T10:29:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f23584af640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23500cfbc0 con 0x7f235015c6c0 2024-09-10T10:29:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f23584af640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f235015f1c0 con 0x7f235015caa0 2024-09-10T10:29:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f23584af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f23500cfe60 con 0x7f23500b7410 2024-09-10T10:29:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f2355a4a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f235015caa0 0x7f235015f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f2355a4a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f235015caa0 0x7f235015f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:46662/0 (socket says 172.21.15.120:46662) 2024-09-10T10:29:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f2355a4a640 1 -- 172.21.15.120:0/1526245980 learned_addr learned my addr 172.21.15.120:0/1526245980 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7fefa0ca1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940d8910 0x7fef940df120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fe7b1d9b640 1 -- 172.21.15.120:0/4291170610 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c037070 con 0x7fe7b40cb2f0 2024-09-10T10:29:06.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.264+0000 7f88d26c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88cc072410 0x7f88cc1fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f88d26c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88cc0c9650 0x7f88cc1e0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f88d26c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88cc150c10 0x7f88cc1e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f88d26c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc06e750 con 0x7f88cc072410 2024-09-10T10:29:06.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f88d26c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc06e470 con 0x7f88cc0c9650 2024-09-10T10:29:06.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f88d26c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc072a10 con 0x7f88cc150c10 2024-09-10T10:29:06.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940d8910 0x7fef940df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fe7b1d9b640 1 -- 172.21.15.120:0/4291170610 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c032440 con 0x7fe7b40cb2f0 2024-09-10T10:29:06.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fe7b1d9b640 1 -- 172.21.15.120:0/4291170610 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c032810 con 0x7fe7b40cb2f0 2024-09-10T10:29:06.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.263+0000 7f21e9ee6640 1 -- 172.21.15.120:0/42402968 wait complete. 2024-09-10T10:29:06.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940d8910 0x7fef940df120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:57272/0 (socket says 172.21.15.120:57272) 2024-09-10T10:29:06.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f88d0c66640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88cc150c10 0x7f88cc1e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fef9affd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fef94005da0 0x7fef9404a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:41896/0 (socket says 172.21.15.120:41896) 2024-09-10T10:29:06.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fef9affd640 1 -- 172.21.15.120:0/3733361332 learned_addr learned my addr 172.21.15.120:0/3733361332 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:06.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f88d0c66640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88cc150c10 0x7f88cc1e0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:57288/0 (socket says 172.21.15.120:57288) 2024-09-10T10:29:06.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f88d0c66640 1 -- 172.21.15.120:0/1437953482 learned_addr learned my addr 172.21.15.120:0/1437953482 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fefa0ca1640 1 --2- 172.21.15.120:0/3733361332 >> v2:172.21.15.134:3300/0 conn(0x7fef94050250 0x7fef94048fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f2356a4c640 1 --2- 172.21.15.120:0/1526245980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23500b7410 0x7f235015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fec88c58640 1 -- 172.21.15.120:0/600811986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840c8010 msgr2=0x7fec8406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fec88c58640 1 --2- 172.21.15.120:0/600811986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840c8010 0x7fec8406c7e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fec740097d0 tx=0x7fec7402a200 comp rx=0 tx=0).stop 2024-09-10T10:29:06.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fefa0ca1640 1 -- 172.21.15.120:0/3733361332 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef94049af0 con 0x7fef94005da0 2024-09-10T10:29:06.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f21e9ee6640 1 Processor -- start 2024-09-10T10:29:06.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f2643fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2644071b70 0x7f264414e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fefa0ca1640 1 -- 172.21.15.120:0/3733361332 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fef9404a3e0 con 0x7fef94050250 2024-09-10T10:29:06.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fefa0ca1640 1 -- 172.21.15.120:0/3733361332 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fef9404f090 con 0x7fef940d8910 2024-09-10T10:29:06.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f2643fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f2644156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f2355a4a640 1 -- 172.21.15.120:0/1526245980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23500b7410 msgr2=0x7f235015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fef9bfff640 1 --2- 172.21.15.120:0/3733361332 >> v2:172.21.15.134:3300/0 conn(0x7fef94050250 0x7fef94048fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f2643fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26440c8430 0x7f26440cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1262439633 0 0) 0x7fef94049af0 con 0x7fef94005da0 2024-09-10T10:29:06.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f2355a4a640 1 --2- 172.21.15.120:0/1526245980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23500b7410 0x7f235015fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f2355a4a640 1 -- 172.21.15.120:0/1526245980 >> v1:172.21.15.134:6789/0 conn(0x7f235015c6c0 legacy=0x7f2350163110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f2355249640 1 -- 172.21.15.120:0/1526245980 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1735350026 0 0) 0x7f23500cfbc0 con 0x7f235015c6c0 2024-09-10T10:29:06.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f01c9d8d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f01c4150ca0 0x7f01c4157e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef8c003520 con 0x7fef94005da0 2024-09-10T10:29:06.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f2355a4a640 1 -- 172.21.15.120:0/1526245980 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23500d0990 con 0x7f235015caa0 2024-09-10T10:29:06.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f26427fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f2644156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.265+0000 7f21e9ee6640 1 -- start start 2024-09-10T10:29:06.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fec88c58640 1 -- 172.21.15.120:0/600811986 shutdown_connections 2024-09-10T10:29:06.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fec88c58640 1 --2- 172.21.15.120:0/600811986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840c8010 0x7fec8406c7e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fec88c58640 1 --2- 172.21.15.120:0/600811986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec840c7c30 0x7fec8415f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fec88c58640 1 -- 172.21.15.120:0/600811986 >> 172.21.15.120:0/600811986 conn(0x7fec840cc660 msgr2=0x7fec840cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f01bbfff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f01c415ac00 0x7f01c40cc4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:52298/0 (socket says 172.21.15.120:52298) 2024-09-10T10:29:06.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f01bbfff640 1 -- 172.21.15.120:0/1928298429 learned_addr learned my addr 172.21.15.120:0/1928298429 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:06.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 657496882 0 0) 0x7fef8c003520 con 0x7fef94005da0 2024-09-10T10:29:06.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f88cb7fe640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88cc0c9650 0x7f88cc1e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f01c9d8d640 1 --2- 172.21.15.120:0/1928298429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c40c9bf0 0x7f01c40c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f2356a4c640 1 --2- 172.21.15.120:0/1526245980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23500b7410 0x7f235015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:06.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940d8910 msgr2=0x7fef940df120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fec88c58640 1 -- 172.21.15.120:0/600811986 shutdown_connections 2024-09-10T10:29:06.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fe7b8874640 1 -- 172.21.15.120:0/4291170610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b40cb2f0 msgr2=0x7fe7b40cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f26427fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f2644156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:57292/0 (socket says 172.21.15.120:57292) 2024-09-10T10:29:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f26427fc640 1 -- 172.21.15.120:0/1323854643 learned_addr learned my addr 172.21.15.120:0/1323854643 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f2642ffd640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2644071b70 0x7f264414e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f26437fe640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26440c8430 0x7f26440cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fe7b8874640 1 --2- 172.21.15.120:0/4291170610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b40cb2f0 0x7fe7b40cc4a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40ce570 tx=0x7fe79c02a1d0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f2643fff640 1 -- 172.21.15.120:0/1323854643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2644158f80 con 0x7f2644071b70 2024-09-10T10:29:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f01c9d8d640 1 -- 172.21.15.120:0/1928298429 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c4158380 con 0x7f01c415ac00 2024-09-10T10:29:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f2643fff640 1 -- 172.21.15.120:0/1323854643 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2644158ca0 con 0x7f26440c8430 2024-09-10T10:29:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f01c9d8d640 1 -- 172.21.15.120:0/1928298429 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f01c40c8b50 con 0x7f01c4150ca0 2024-09-10T10:29:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f2643fff640 1 -- 172.21.15.120:0/1323854643 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2644073fe0 con 0x7f26440c6ed0 2024-09-10T10:29:06.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fe7b8874640 1 -- 172.21.15.120:0/4291170610 shutdown_connections 2024-09-10T10:29:06.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f01c3fff640 1 --2- 172.21.15.120:0/1928298429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c40c9bf0 0x7f01c40c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940d8910 0x7fef940df120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 >> v2:172.21.15.134:3300/0 conn(0x7fef94050250 msgr2=0x7fef94048fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 --2- 172.21.15.120:0/3733361332 >> v2:172.21.15.134:3300/0 conn(0x7fef94050250 0x7fef94048fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9404f370 con 0x7fef94005da0 2024-09-10T10:29:06.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9b7fe640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940d8910 0x7fef940df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:06.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7fef88001670 con 0x7fef94005da0 2024-09-10T10:29:06.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fe7b8874640 1 --2- 172.21.15.120:0/4291170610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7b415e9d0 0x7fe7b40cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 >> v1:172.21.15.134:6789/0 conn(0x7fef94005da0 legacy=0x7fef9404a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef8c0039b0 0x7fef8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef8c004800 0x7fef8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940d8910 0x7fef8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef9404a3e0 con 0x7fef8c0039b0 2024-09-10T10:29:06.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fe7b8874640 1 --2- 172.21.15.120:0/4291170610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b40cb2f0 0x7fe7b40cc4a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef9404f090 con 0x7fef8c004800 2024-09-10T10:29:06.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fef8c00ba40 con 0x7fef940d8910 2024-09-10T10:29:06.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fe7b8874640 1 --2- 172.21.15.120:0/4291170610 >> v2:172.21.15.120:3300/0 conn(0x7fe7b40cc0c0 0x7fe7b415aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fe7b8874640 1 -- 172.21.15.120:0/4291170610 >> 172.21.15.120:0/4291170610 conn(0x7fe7b406d560 msgr2=0x7fe7b406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9bfff640 1 --2- 172.21.15.120:0/3733361332 >> v2:172.21.15.134:3300/0 conn(0x7fef94050250 0x7fef94048fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:29:06.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f88d0c66640 1 -- 172.21.15.120:0/1437953482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88cc0c9650 msgr2=0x7f88cc1e0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f88d0c66640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88cc0c9650 0x7f88cc1e0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f88d0c66640 1 -- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88cc072410 msgr2=0x7f88cc1fa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f88d0c66640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88cc072410 0x7f88cc1fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f88d0c66640 1 -- 172.21.15.120:0/1437953482 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc0ca680 con 0x7f88cc150c10 2024-09-10T10:29:06.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fef9affd640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef8c004800 0x7fef8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f01c9d8d640 1 -- 172.21.15.120:0/1928298429 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01c40c8e30 con 0x7f01c40c9bf0 2024-09-10T10:29:06.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7fef9b7fe640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef8c0039b0 0x7fef8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f21e9ee6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e40cd2a0 0x7f21e40c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f21e9ee6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21e40cd680 0x7f21e40c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f26427fc640 1 -- 172.21.15.120:0/1323854643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26440c8430 msgr2=0x7f26440cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f26427fc640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26440c8430 0x7f26440cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f26427fc640 1 -- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2644071b70 msgr2=0x7f264414e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f26427fc640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2644071b70 0x7f264414e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f26427fc640 1 -- 172.21.15.120:0/1323854643 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2644157040 con 0x7f26440c6ed0 2024-09-10T10:29:06.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fe7b8874640 1 -- 172.21.15.120:0/4291170610 shutdown_connections 2024-09-10T10:29:06.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f01c2ffd640 1 -- 172.21.15.120:0/1928298429 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 960166863 0 0) 0x7f01c4158380 con 0x7f01c415ac00 2024-09-10T10:29:06.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f01c2ffd640 1 -- 172.21.15.120:0/1928298429 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01b0005730 con 0x7f01c415ac00 2024-09-10T10:29:06.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7fe7b8874640 1 -- 172.21.15.120:0/4291170610 wait complete. 2024-09-10T10:29:06.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f01c3fff640 1 -- 172.21.15.120:0/1928298429 >> v2:172.21.15.134:3300/0 conn(0x7f01c4150ca0 msgr2=0x7f01c4157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7fef9b7fe640 1 -- 172.21.15.120:0/3733361332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940d8910 msgr2=0x7fef8c0051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:29:06.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7fef9b7fe640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940d8910 0x7fef8c0051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7fef9b7fe640 1 -- 172.21.15.120:0/3733361332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef8c004800 msgr2=0x7fef8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7fef9b7fe640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef8c004800 0x7fef8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f26427fc640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f2644156b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f263400b4a0 tx=0x7f263400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7fef9b7fe640 1 -- 172.21.15.120:0/3733361332 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9404f370 con 0x7fef8c0039b0 2024-09-10T10:29:06.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f21e37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e40cd2a0 0x7f21e40c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f2355a4a640 1 --2- 172.21.15.120:0/1526245980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f235015caa0 0x7f235015f6c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2340009870 tx=0x7f234002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57364065f8cb2a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f01c2ffd640 1 -- 172.21.15.120:0/1928298429 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2808934633 0 0) 0x7f01b0005730 con 0x7f01c415ac00 2024-09-10T10:29:06.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f262b7fe640 1 -- 172.21.15.120:0/1323854643 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634014070 con 0x7f26440c6ed0 2024-09-10T10:29:06.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f262b7fe640 1 -- 172.21.15.120:0/1323854643 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2634010610 con 0x7f26440c6ed0 2024-09-10T10:29:06.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f2643fff640 1 -- 172.21.15.120:0/1323854643 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26440c5210 con 0x7f26440c6ed0 2024-09-10T10:29:06.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f262b7fe640 1 -- 172.21.15.120:0/1323854643 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340109e0 con 0x7f26440c6ed0 2024-09-10T10:29:06.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f21e37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e40cd2a0 0x7f21e40c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:46712/0 (socket says 172.21.15.120:46712) 2024-09-10T10:29:06.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7f01c37fe640 1 --2- 172.21.15.120:0/1928298429 >> v2:172.21.15.134:3300/0 conn(0x7f01c4150ca0 0x7f01c4157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f2355249640 1 -- 172.21.15.120:0/1526245980 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340037070 con 0x7f235015caa0 2024-09-10T10:29:06.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f21e2ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21e40cd680 0x7f21e40c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f21e2ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21e40cd680 0x7f21e40c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:57326/0 (socket says 172.21.15.120:57326) 2024-09-10T10:29:06.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f01c3fff640 1 --2- 172.21.15.120:0/1928298429 >> v2:172.21.15.134:3300/0 conn(0x7f01c4150ca0 0x7f01c4157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f01c3fff640 1 -- 172.21.15.120:0/1928298429 >> v1:172.21.15.120:6789/0 conn(0x7f01c415ac00 legacy=0x7f01c40cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f01c3fff640 1 -- 172.21.15.120:0/1928298429 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c40c9110 con 0x7f01c40c9bf0 2024-09-10T10:29:06.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7f2355249640 1 -- 172.21.15.120:0/1526245980 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2340032440 con 0x7f235015caa0 2024-09-10T10:29:06.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.267+0000 7f2643fff640 1 -- 172.21.15.120:0/1323854643 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f26440c3500 con 0x7f26440c6ed0 2024-09-10T10:29:06.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7f21e37fe640 1 -- 172.21.15.120:0/1592600644 learned_addr learned my addr 172.21.15.120:0/1592600644 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7f21e9ee6640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e415b2a0 0x7f21e40c2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f21e40c9820 con 0x7f21e415b2a0 2024-09-10T10:29:06.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21e40c9540 con 0x7f21e40cd2a0 2024-09-10T10:29:06.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.266+0000 7fec88c58640 1 -- 172.21.15.120:0/600811986 wait complete. 2024-09-10T10:29:06.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7fec88c58640 1 Processor -- start 2024-09-10T10:29:06.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f21e4073730 con 0x7f21e40cd680 2024-09-10T10:29:06.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7f21e3fff640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e415b2a0 0x7f21e40c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7f01c37fe640 1 --2- 172.21.15.120:0/1928298429 >> v2:172.21.15.134:3300/0 conn(0x7f01c4150ca0 0x7f01c4157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:06.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7fec88c58640 1 -- start start 2024-09-10T10:29:06.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.268+0000 7f2355249640 1 -- 172.21.15.120:0/1526245980 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340032810 con 0x7f235015caa0 2024-09-10T10:29:06.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f23584af640 1 -- 172.21.15.120:0/1526245980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f235015caa0 msgr2=0x7f235015f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f23584af640 1 --2- 172.21.15.120:0/1526245980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f235015caa0 0x7f235015f6c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2340009870 tx=0x7f234002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f88d0c66640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88cc150c10 0x7f88cc1e0a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c550 tx=0x7f88c000ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f2643fff640 1 -- 172.21.15.120:0/1323854643 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f26440c68b0 con 0x7f26440c6ed0 2024-09-10T10:29:06.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7fef9b7fe640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef8c0039b0 0x7fef8c003dd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fef900099e0 tx=0x7fef8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47748a41ca70cb53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f23584af640 1 -- 172.21.15.120:0/1526245980 shutdown_connections 2024-09-10T10:29:06.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f23584af640 1 --2- 172.21.15.120:0/1526245980 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23500b7410 0x7f235015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f23584af640 1 --2- 172.21.15.120:0/1526245980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f235015caa0 0x7f235015f6c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f23584af640 1 -- 172.21.15.120:0/1526245980 >> 172.21.15.120:0/1526245980 conn(0x7f235006f3a0 msgr2=0x7f235006f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f21e37fe640 1 -- 172.21.15.120:0/1592600644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21e40cd680 msgr2=0x7f21e40c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f21e37fe640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21e40cd680 0x7f21e40c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f21e37fe640 1 -- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e415b2a0 msgr2=0x7f21e40c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f21e37fe640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21e415b2a0 0x7f21e40c2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f21e37fe640 1 -- 172.21.15.120:0/1592600644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e40c3780 con 0x7f21e40cd2a0 2024-09-10T10:29:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7fe7b8874640 1 Processor -- start 2024-09-10T10:29:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7fe7b8874640 1 -- start start 2024-09-10T10:29:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f23584af640 1 -- 172.21.15.120:0/1526245980 shutdown_connections 2024-09-10T10:29:06.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f23584af640 1 -- 172.21.15.120:0/1526245980 wait complete. 2024-09-10T10:29:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f01c3fff640 1 --2- 172.21.15.120:0/1928298429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c40c9bf0 0x7f01c40c76d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01bc007d70 tx=0x7f01bc02a9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=914b8015226025a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f88c8ff9640 1 -- 172.21.15.120:0/1437953482 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0017070 con 0x7f88cc150c10 2024-09-10T10:29:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f88c8ff9640 1 -- 172.21.15.120:0/1437953482 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c00035f0 con 0x7f88cc150c10 2024-09-10T10:29:06.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f88c8ff9640 1 -- 172.21.15.120:0/1437953482 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00039e0 con 0x7f88cc150c10 2024-09-10T10:29:06.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90037070 con 0x7fef8c0039b0 2024-09-10T10:29:06.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef90032440 con 0x7fef8c0039b0 2024-09-10T10:29:06.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7fef9a7fc640 1 -- 172.21.15.120:0/3733361332 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9000f040 con 0x7fef8c0039b0 2024-09-10T10:29:06.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7fec88c58640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec84067d10 0x7fec840c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f01c2ffd640 1 -- 172.21.15.120:0/1928298429 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac001670 con 0x7f01c40c9bf0 2024-09-10T10:29:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f01c2ffd640 1 -- 172.21.15.120:0/1928298429 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01bc035070 con 0x7f01c40c9bf0 2024-09-10T10:29:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f01c2ffd640 1 -- 172.21.15.120:0/1928298429 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc00a500 con 0x7f01c40c9bf0 2024-09-10T10:29:06.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7fec88c58640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840c7c30 0x7fec841505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f21e37fe640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e40cd2a0 0x7f21e40c3ba0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21d8000c00 tx=0x7f21d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f21cffff640 1 -- 172.21.15.120:0/1592600644 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8037040 con 0x7f21e40cd2a0 2024-09-10T10:29:06.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f21cffff640 1 -- 172.21.15.120:0/1592600644 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d8042070 con 0x7f21e40cd2a0 2024-09-10T10:29:06.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e414eab0 con 0x7f21e40cd2a0 2024-09-10T10:29:06.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e414f050 con 0x7f21e40cd2a0 2024-09-10T10:29:06.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f262b7fe640 1 -- 172.21.15.120:0/1323854643 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2634017070 con 0x7f26440c6ed0 2024-09-10T10:29:06.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f262b7fe640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f263006be60 0x7f263006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7fe7b8874640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7b40cb2f0 0x7fe7b414b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f21cffff640 1 -- 172.21.15.120:0/1592600644 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8030940 con 0x7f21e40cd2a0 2024-09-10T10:29:06.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7fe7b8874640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7b40cc0c0 0x7fe7b40c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7f262b7fe640 1 -- 172.21.15.120:0/1323854643 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f263408a900 con 0x7f26440c6ed0 2024-09-10T10:29:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7f262b7fe640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f2630075c70 0x7f2630078110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7f262b7fe640 1 -- 172.21.15.120:0/1323854643 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26300787e0 con 0x7f2630075c70 2024-09-10T10:29:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7f262b7fe640 1 -- 172.21.15.120:0/1323854643 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f263408f2a0 con 0x7f26440c6ed0 2024-09-10T10:29:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fefa0ca1640 1 -- 172.21.15.120:0/3733361332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef8c0039b0 msgr2=0x7fef8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fefa0ca1640 1 --2- 172.21.15.120:0/3733361332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef8c0039b0 0x7fef8c003dd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fef900099e0 tx=0x7fef8c005830 comp rx=0 tx=0).stop 2024-09-10T10:29:06.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b8874640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b415e9d0 0x7fe7b40c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b8874640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b415e1d0 con 0x7fe7b40cc0c0 2024-09-10T10:29:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b8874640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b415b790 con 0x7fe7b415e9d0 2024-09-10T10:29:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b8874640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b40c5560 con 0x7fe7b40cb2f0 2024-09-10T10:29:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b415e9d0 0x7fe7b40c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b415e9d0 0x7fe7b40c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:46724/0 (socket says 172.21.15.120:46724) 2024-09-10T10:29:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b2d9d640 1 -- 172.21.15.120:0/2305688815 learned_addr learned my addr 172.21.15.120:0/2305688815 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7f01c9d8d640 1 -- 172.21.15.120:0/1928298429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c40c9bf0 msgr2=0x7f01c40c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7f01c9d8d640 1 --2- 172.21.15.120:0/1928298429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c40c9bf0 0x7f01c40c76d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01bc007d70 tx=0x7f01bc02a9d0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.269+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc1e2430 con 0x7f88cc150c10 2024-09-10T10:29:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.270+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc1e26f0 con 0x7f88cc150c10 2024-09-10T10:29:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7f23584af640 1 Processor -- start 2024-09-10T10:29:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fef9bfff640 1 -- 172.21.15.120:0/3733361332 reap_dead start 2024-09-10T10:29:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fefa0ca1640 1 -- 172.21.15.120:0/3733361332 shutdown_connections 2024-09-10T10:29:06.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fefa0ca1640 1 -- 172.21.15.120:0/3733361332 >> 172.21.15.120:0/3733361332 conn(0x7fef94054070 msgr2=0x7fef94054450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88cc0c3e10 con 0x7f88cc150c10 2024-09-10T10:29:06.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fec837fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec84067d10 0x7fec840c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fec88c58640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec840c8010 0x7fec840c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7f26437fe640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f2630075c70 0x7f2630078110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fec88c58640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec8406c2c0 con 0x7fec840c7c30 2024-09-10T10:29:06.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fec88c58640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec8406bfe0 con 0x7fec840c8010 2024-09-10T10:29:06.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7f23584af640 1 -- start start 2024-09-10T10:29:06.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fec88c58640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec840c67e0 con 0x7fec84067d10 2024-09-10T10:29:06.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fec837fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec84067d10 0x7fec840c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:57338/0 (socket says 172.21.15.120:57338) 2024-09-10T10:29:06.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fec837fe640 1 -- 172.21.15.120:0/3134724605 learned_addr learned my addr 172.21.15.120:0/3134724605 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fefa0ca1640 1 -- 172.21.15.120:0/3733361332 shutdown_connections 2024-09-10T10:29:06.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fefa0ca1640 1 -- 172.21.15.120:0/3733361332 wait complete. 2024-09-10T10:29:06.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b259c640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7b40cb2f0 0x7fe7b414b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b2d9d640 1 -- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7b40cb2f0 msgr2=0x7fe7b414b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b2d9d640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7b40cb2f0 0x7fe7b414b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b2d9d640 1 -- 172.21.15.120:0/2305688815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7b40cc0c0 msgr2=0x7fe7b40c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b2d9d640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7b40cc0c0 0x7fe7b40c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fe7b2d9d640 1 -- 172.21.15.120:0/2305688815 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b415cd10 con 0x7fe7b415e9d0 2024-09-10T10:29:06.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.271+0000 7fec83fff640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec840c8010 0x7fec840c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7f2642ffd640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f263006be60 0x7f263006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fe7b2d9d640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b415e9d0 0x7fe7b40c9da0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00efc0 tx=0x7fe7ac00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fec82ffd640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840c7c30 0x7fec841505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fec837fe640 1 -- 172.21.15.120:0/3134724605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec840c8010 msgr2=0x7fec840c4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fec837fe640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec840c8010 0x7fec840c4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fec837fe640 1 -- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840c7c30 msgr2=0x7fec841505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fec837fe640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840c7c30 0x7fec841505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fec837fe640 1 -- 172.21.15.120:0/3134724605 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec840c4870 con 0x7fec84067d10 2024-09-10T10:29:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fec83fff640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec840c8010 0x7fec840c4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fec82ffd640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840c7c30 0x7fec841505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fec837fe640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec84067d10 0x7fec840c5820 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec7800e9b0 tx=0x7fec7800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21e40c79a0 con 0x7f21e40cd2a0 2024-09-10T10:29:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7f88c8ff9640 1 -- 172.21.15.120:0/1437953482 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88c0003b80 con 0x7f88cc150c10 2024-09-10T10:29:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fe7aaffd640 1 -- 172.21.15.120:0/2305688815 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac018020 con 0x7fe7b415e9d0 2024-09-10T10:29:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fe7aaffd640 1 -- 172.21.15.120:0/2305688815 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ac014480 con 0x7fe7b415e9d0 2024-09-10T10:29:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fe7b8874640 1 -- 172.21.15.120:0/2305688815 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b415dc10 con 0x7fe7b415e9d0 2024-09-10T10:29:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7fe7b8874640 1 -- 172.21.15.120:0/2305688815 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b40ca940 con 0x7fe7b415e9d0 2024-09-10T10:29:06.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7f01c9d8d640 1 -- 172.21.15.120:0/1928298429 shutdown_connections 2024-09-10T10:29:06.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7f01c9d8d640 1 --2- 172.21.15.120:0/1928298429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c40c9bf0 0x7f01c40c76d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7f01c9d8d640 1 --2- 172.21.15.120:0/1928298429 >> v2:172.21.15.134:3300/0 conn(0x7f01c4150ca0 0x7f01c4157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7f01c9d8d640 1 -- 172.21.15.120:0/1928298429 >> 172.21.15.120:0/1928298429 conn(0x7f01c406bc10 msgr2=0x7f01c406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.272+0000 7f2642ffd640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f263006be60 0x7f263006e320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2638034000 tx=0x7f263802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f01c9d8d640 1 -- 172.21.15.120:0/1928298429 shutdown_connections 2024-09-10T10:29:06.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f23584af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23500b7410 0x7f23500d5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7fe7aaffd640 1 -- 172.21.15.120:0/2305688815 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac014870 con 0x7fe7b415e9d0 2024-09-10T10:29:06.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f23584af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f235015c6c0 0x7f23500d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7fec6bfff640 1 -- 172.21.15.120:0/3134724605 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78017070 con 0x7fec84067d10 2024-09-10T10:29:06.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7fec6bfff640 1 -- 172.21.15.120:0/3134724605 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7800c9a0 con 0x7fec84067d10 2024-09-10T10:29:06.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7fec88c58640 1 -- 172.21.15.120:0/3134724605 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec840c5eb0 con 0x7fec84067d10 2024-09-10T10:29:06.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7fec88c58640 1 -- 172.21.15.120:0/3134724605 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fec841fd5e0 con 0x7fec84067d10 2024-09-10T10:29:06.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f88c8ff9640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f88b806be30 0x7f88b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f88c8ff9640 1 -- 172.21.15.120:0/1437953482 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f88c008a760 con 0x7f88cc150c10 2024-09-10T10:29:06.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f88cbfff640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f88b806be30 0x7f88b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f88c8ff9640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f88b8075cd0 0x7f88b8078170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7fefa0ca1640 1 Processor -- start 2024-09-10T10:29:06.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7fefa0ca1640 1 -- start start 2024-09-10T10:29:06.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f01c9d8d640 1 -- 172.21.15.120:0/1928298429 wait complete. 2024-09-10T10:29:06.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f23584af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23500cafe0 0x7f2350161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f2355a4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f235015c6c0 0x7f23500d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f2355a4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f235015c6c0 0x7f23500d2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:57368/0 (socket says 172.21.15.120:57368) 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f2355a4a640 1 -- 172.21.15.120:0/1955758498 learned_addr learned my addr 172.21.15.120:0/1955758498 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23500d12c0 con 0x7f23500b7410 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f2356a4c640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23500cafe0 0x7f2350161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f235624b640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23500b7410 0x7f23500d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23500d1140 con 0x7f23500cafe0 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f23500d1440 con 0x7f235015c6c0 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f2356a4c640 1 -- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f235015c6c0 msgr2=0x7f23500d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f2356a4c640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f235015c6c0 0x7f23500d2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f2356a4c640 1 -- 172.21.15.120:0/1955758498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23500b7410 msgr2=0x7f23500d5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f26437fe640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f2630075c70 0x7f2630078110 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f2356a4c640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23500b7410 0x7f23500d5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f88cb7fe640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f88b8075cd0 0x7f88b8078170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f88c8ff9640 1 -- 172.21.15.120:0/1437953482 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88b8078840 con 0x7f88b8075cd0 2024-09-10T10:29:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f88cbfff640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f88b806be30 0x7f88b806e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88c4036000 tx=0x7f88c4031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f88c8ff9640 1 -- 172.21.15.120:0/1437953482 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f88c0091050 con 0x7f88cc150c10 2024-09-10T10:29:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f88cb7fe640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f88b8075cd0 0x7f88b8078170 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7fec6bfff640 1 -- 172.21.15.120:0/3134724605 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7800cd90 con 0x7fec84067d10 2024-09-10T10:29:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.273+0000 7f21cffff640 1 -- 172.21.15.120:0/1592600644 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21d8030ae0 con 0x7f21e40cd2a0 2024-09-10T10:29:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f21cffff640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f21c806beb0 0x7f21c806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f21cffff640 1 -- 172.21.15.120:0/1592600644 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f21d80abb90 con 0x7f21e40cd2a0 2024-09-10T10:29:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f2356a4c640 1 -- 172.21.15.120:0/1955758498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23500d43d0 con 0x7f23500cafe0 2024-09-10T10:29:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f235624b640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23500b7410 0x7f23500d5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7f2356a4c640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23500cafe0 0x7f2350161710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c5f0 tx=0x7f234c00cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7f01c9d8d640 1 Processor -- start 2024-09-10T10:29:06.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7fefa0ca1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef94177d40 0x7fef94178120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7f01c9d8d640 1 -- start start 2024-09-10T10:29:06.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7fe7aaffd640 1 -- 172.21.15.120:0/2305688815 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7ac005d10 con 0x7fe7b415e9d0 2024-09-10T10:29:06.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7f2346ffd640 1 -- 172.21.15.120:0/1955758498 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c015070 con 0x7f23500cafe0 2024-09-10T10:29:06.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7f2346ffd640 1 -- 172.21.15.120:0/1955758498 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c011820 con 0x7f23500cafe0 2024-09-10T10:29:06.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7f2346ffd640 1 -- 172.21.15.120:0/1955758498 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c011c10 con 0x7f23500cafe0 2024-09-10T10:29:06.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f21cffff640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f21c8075cd0 0x7f21c8078170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.274+0000 7f21cffff640 1 -- 172.21.15.120:0/1592600644 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21c8078840 con 0x7f21c8075cd0 2024-09-10T10:29:06.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7f21e3fff640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f21c8075cd0 0x7f21c8078170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7f21e2ffd640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f21c806beb0 0x7f21c806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7f21e3fff640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f21c8075cd0 0x7f21c8078170 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.275+0000 7f21e2ffd640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f21c806beb0 0x7f21c806e370 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f21e4150eb0 tx=0x7f21d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7f88c8ff9640 1 -- 172.21.15.120:0/1437953482 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f88b8078840 con 0x7f88b8075cd0 2024-09-10T10:29:06.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7f262b7fe640 1 -- 172.21.15.120:0/1323854643 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f26300787e0 con 0x7f2630075c70 2024-09-10T10:29:06.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7f21cffff640 1 -- 172.21.15.120:0/1592600644 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f21d8075b40 con 0x7f21e40cd2a0 2024-09-10T10:29:06.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fefa0ca1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef94178660 0x7fef9404c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fefa0ca1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef9404c7a0 0x7fef9404d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef94177d40 0x7fef94178120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fefa0ca1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef940d8cf0 con 0x7fef94178660 2024-09-10T10:29:06.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fefa0ca1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef9404a070 con 0x7fef9404c7a0 2024-09-10T10:29:06.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fefa0ca1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fef94050700 con 0x7fef94177d40 2024-09-10T10:29:06.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef94177d40 0x7fef94178120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:57382/0 (socket says 172.21.15.120:57382) 2024-09-10T10:29:06.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fef9b7fe640 1 -- 172.21.15.120:0/1722837195 learned_addr learned my addr 172.21.15.120:0/1722837195 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fe7aaffd640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe79406be80 0x7fe79406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fef9affd640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef94178660 0x7fef9404c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fec69ffb640 1 -- 172.21.15.120:0/3134724605 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec64001030 con 0x7fec84067d10 2024-09-10T10:29:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7f21cffff640 1 -- 172.21.15.120:0/1592600644 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21c8078840 con 0x7f21c8075cd0 2024-09-10T10:29:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fe7b8874640 1 -- 172.21.15.120:0/2305688815 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7b415c270 con 0x7fe7b415e9d0 2024-09-10T10:29:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7f01c9d8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c40c9bf0 0x7f01c4154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7fef9bfff640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef9404c7a0 0x7fef9404d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7f01c9d8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01c4150ca0 0x7f01c4159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7f01c9d8d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c415ac00 0x7f01c41585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c9d8d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01c4151080 con 0x7f01c4150ca0 2024-09-10T10:29:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23501f8c70 con 0x7f23500cafe0 2024-09-10T10:29:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.276+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23500d4950 con 0x7f23500cafe0 2024-09-10T10:29:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c9d8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01c40c96d0 con 0x7f01c40c9bf0 2024-09-10T10:29:06.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fef9b7fe640 1 -- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef9404c7a0 msgr2=0x7fef9404d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c9d8d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01c40ccc00 con 0x7f01c415ac00 2024-09-10T10:29:06.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fef9b7fe640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef9404c7a0 0x7fef9404d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fef9b7fe640 1 -- 172.21.15.120:0/1722837195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef94178660 msgr2=0x7fef9404c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fef9b7fe640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef94178660 0x7fef9404c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c415ac00 0x7f01c41585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c415ac00 0x7f01c41585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:57396/0 (socket says 172.21.15.120:57396) 2024-09-10T10:29:06.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c3fff640 1 -- 172.21.15.120:0/3148007956 learned_addr learned my addr 172.21.15.120:0/3148007956 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fef9b7fe640 1 -- 172.21.15.120:0/1722837195 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef940dcff0 con 0x7fef94177d40 2024-09-10T10:29:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c37fe640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c40c9bf0 0x7f01c4154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fef9bfff640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef9404c7a0 0x7fef9404d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fe7aaffd640 1 -- 172.21.15.120:0/2305688815 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fe7ac08c1c0 con 0x7fe7b415e9d0 2024-09-10T10:29:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fe7aaffd640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe794075c90 0x7fe794078130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fe7aaffd640 1 -- 172.21.15.120:0/2305688815 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7ac00be50 con 0x7fe794075c90 2024-09-10T10:29:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fe7aaffd640 1 -- 172.21.15.120:0/2305688815 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fe7ac093050 con 0x7fe7b415e9d0 2024-09-10T10:29:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01bbfff640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01c4150ca0 0x7f01c4159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2350163110 con 0x7f23500cafe0 2024-09-10T10:29:06.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c37fe640 1 -- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c415ac00 msgr2=0x7f01c41585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c37fe640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01c415ac00 0x7f01c41585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c37fe640 1 -- 172.21.15.120:0/3148007956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01c4150ca0 msgr2=0x7f01c4159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c37fe640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01c4150ca0 0x7f01c4159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01c37fe640 1 -- 172.21.15.120:0/3148007956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c41ec790 con 0x7f01c40c9bf0 2024-09-10T10:29:06.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7f01bbfff640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01c4150ca0 0x7f01c4159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:06.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.277+0000 7fec6bfff640 1 -- 172.21.15.120:0/3134724605 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec7801a070 con 0x7fec84067d10 2024-09-10T10:29:06.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7fe7b259c640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe79406be80 0x7fe79406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7fef9b7fe640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef94177d40 0x7fef94178120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef900099e0 tx=0x7fef90009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7fef87fff640 1 -- 172.21.15.120:0/1722837195 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90042020 con 0x7fef94177d40 2024-09-10T10:29:06.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7fefa0ca1640 1 -- 172.21.15.120:0/1722837195 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef940de720 con 0x7fef94177d40 2024-09-10T10:29:06.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7fec6bfff640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec7006bf00 0x7fec7006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7fec6bfff640 1 -- 172.21.15.120:0/3134724605 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fec7808b390 con 0x7fec84067d10 2024-09-10T10:29:06.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.279+0000 7fec6bfff640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fec70075d90 0x7fec70078230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7fefa0ca1640 1 -- 172.21.15.120:0/1722837195 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fef940dd960 con 0x7fef94177d40 2024-09-10T10:29:06.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.279+0000 7fef87fff640 1 -- 172.21.15.120:0/1722837195 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9003d690 con 0x7fef94177d40 2024-09-10T10:29:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.279+0000 7fec83fff640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fec70075d90 0x7fec70078230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.279+0000 7fec6bfff640 1 -- 172.21.15.120:0/3134724605 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec70078900 con 0x7fec70075d90 2024-09-10T10:29:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.279+0000 7fec6bfff640 1 -- 172.21.15.120:0/3134724605 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fec78092050 con 0x7fec84067d10 2024-09-10T10:29:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7f01c37fe640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c40c9bf0 0x7f01c4154a90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f01b4006d80 tx=0x7f01b400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7f01c0ff9640 1 -- 172.21.15.120:0/3148007956 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b401d070 con 0x7f01c40c9bf0 2024-09-10T10:29:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7f01c0ff9640 1 -- 172.21.15.120:0/3148007956 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b40077c0 con 0x7f01c40c9bf0 2024-09-10T10:29:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.278+0000 7f01c0ff9640 1 -- 172.21.15.120:0/3148007956 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4007b90 con 0x7f01c40c9bf0 2024-09-10T10:29:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.280+0000 7fec83fff640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fec70075d90 0x7fec70078230 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.280+0000 7fec82ffd640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec7006bf00 0x7fec7006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.280+0000 7fec82ffd640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec7006bf00 0x7fec7006e3c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fec74034000 tx=0x7fec7402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.280+0000 7fe7b259c640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe79406be80 0x7fe79406e340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009a50 tx=0x7fe7a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.280+0000 7f01c9d8d640 1 -- 172.21.15.120:0/3148007956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c4157b10 con 0x7f01c40c9bf0 2024-09-10T10:29:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.280+0000 7f01c9d8d640 1 -- 172.21.15.120:0/3148007956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c4157db0 con 0x7f01c40c9bf0 2024-09-10T10:29:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.280+0000 7fef87fff640 1 -- 172.21.15.120:0/1722837195 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90037040 con 0x7fef94177d40 2024-09-10T10:29:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.280+0000 7fefa0ca1640 1 -- 172.21.15.120:0/1722837195 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef940df120 con 0x7fef94177d40 2024-09-10T10:29:06.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.281+0000 7fef87fff640 1 -- 172.21.15.120:0/1722837195 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef90045020 con 0x7fef94177d40 2024-09-10T10:29:06.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.281+0000 7fec6bfff640 1 -- 172.21.15.120:0/3134724605 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec70078900 con 0x7fec70075d90 2024-09-10T10:29:06.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.281+0000 7fe7abfff640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe794075c90 0x7fe794078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.281+0000 7f01c9d8d640 1 -- 172.21.15.120:0/3148007956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01c40c85a0 con 0x7f01c40c9bf0 2024-09-10T10:29:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.281+0000 7fef87fff640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fef8006be60 0x7fef8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.282+0000 7fef9affd640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fef8006be60 0x7fef8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.282+0000 7fe7abfff640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe794075c90 0x7fe794078130 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.282+0000 7f01c0ff9640 1 -- 172.21.15.120:0/3148007956 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01b4020070 con 0x7f01c40c9bf0 2024-09-10T10:29:06.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.282+0000 7f2346ffd640 1 -- 172.21.15.120:0/1955758498 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f234c018070 con 0x7f23500cafe0 2024-09-10T10:29:06.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.282+0000 7fef9affd640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fef8006be60 0x7fef8006e320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef88007ca0 tx=0x7fef88007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.282+0000 7fef87fff640 1 -- 172.21.15.120:0/1722837195 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fef900abae0 con 0x7fef94177d40 2024-09-10T10:29:06.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7fe7aaffd640 1 -- 172.21.15.120:0/2305688815 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7ac00be50 con 0x7fe794075c90 2024-09-10T10:29:06.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7fef87fff640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fef80075d00 0x7fef800781a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7fef87fff640 1 -- 172.21.15.120:0/1722837195 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef9003d690 con 0x7fef80075d00 2024-09-10T10:29:06.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7fef87fff640 1 -- 172.21.15.120:0/1722837195 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fef900b0260 con 0x7fef94177d40 2024-09-10T10:29:06.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7f01c0ff9640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01a406be80 0x7f01a406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7fef9bfff640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fef80075d00 0x7fef800781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7f2346ffd640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233406bf00 0x7f233406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7f01c0ff9640 1 -- 172.21.15.120:0/3148007956 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f01b4093e30 con 0x7f01c40c9bf0 2024-09-10T10:29:06.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7f01c0ff9640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f01a4075d20 0x7f01a40781c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7f01c0ff9640 1 -- 172.21.15.120:0/3148007956 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01a4078890 con 0x7f01a4075d20 2024-09-10T10:29:06.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7f01c0ff9640 1 -- 172.21.15.120:0/3148007956 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f01b4096490 con 0x7f01c40c9bf0 2024-09-10T10:29:06.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.283+0000 7f01bbfff640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01a406be80 0x7f01a406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.284+0000 7fef9bfff640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fef80075d00 0x7fef800781a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.284+0000 7f235624b640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233406bf00 0x7f233406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.284+0000 7f01c3fff640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f01a4075d20 0x7f01a40781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.284+0000 7f2346ffd640 1 -- 172.21.15.120:0/1955758498 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f234c08b6c0 con 0x7f23500cafe0 2024-09-10T10:29:06.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.284+0000 7f2346ffd640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2334075da0 0x7f2334078240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.284+0000 7fef87fff640 1 -- 172.21.15.120:0/1722837195 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef9003d690 con 0x7fef80075d00 2024-09-10T10:29:06.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.284+0000 7f2346ffd640 1 -- 172.21.15.120:0/1955758498 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f234c009f20 con 0x7f2334075da0 2024-09-10T10:29:06.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.285+0000 7f01c3fff640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f01a4075d20 0x7f01a40781c0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.284+0000 7f2346ffd640 1 -- 172.21.15.120:0/1955758498 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f234c08bab0 con 0x7f23500cafe0 2024-09-10T10:29:06.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.285+0000 7f235624b640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233406bf00 0x7f233406e3c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23480012e0 tx=0x7f2348009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.285+0000 7f2355a4a640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2334075da0 0x7f2334078240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.285+0000 7f01bbfff640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01a406be80 0x7f01a406e340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0017e0 tx=0x7f01ac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.285+0000 7f01c0ff9640 1 -- 172.21.15.120:0/3148007956 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01a4078890 con 0x7f01a4075d20 2024-09-10T10:29:06.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.286+0000 7f2355a4a640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2334075da0 0x7f2334078240 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.286+0000 7f2346ffd640 1 -- 172.21.15.120:0/1955758498 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f234c009f20 con 0x7f2334075da0 2024-09-10T10:29:06.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.304+0000 7f2643fff640 1 -- 172.21.15.120:0/1323854643 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f26440c5da0 con 0x7f2630075c70 2024-09-10T10:29:06.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.304+0000 7f262b7fe640 1 -- 172.21.15.120:0/1323854643 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f26440c5da0 con 0x7f2630075c70 2024-09-10T10:29:06.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.305+0000 7f26297fa640 1 -- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f2630075c70 msgr2=0x7f2630078110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.305+0000 7f26297fa640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f2630075c70 0x7f2630078110 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.305+0000 7f26297fa640 1 -- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f263006be60 msgr2=0x7f263006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.305+0000 7f26297fa640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f263006be60 0x7f263006e320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2638034000 tx=0x7f263802e000 comp rx=0 tx=0).stop 2024-09-10T10:29:06.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.305+0000 7f26297fa640 1 -- 172.21.15.120:0/1323854643 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 msgr2=0x7f2644156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.305+0000 7f26297fa640 1 --2- 172.21.15.120:0/1323854643 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26440c6ed0 0x7f2644156b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f263400b4a0 tx=0x7f263400b960 comp rx=0 tx=0).stop 2024-09-10T10:29:06.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.305+0000 7f26437fe640 1 -- 172.21.15.120:0/1323854643 reap_dead start 2024-09-10T10:29:06.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.305+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f88cc0718c0 con 0x7f88b8075cd0 2024-09-10T10:29:06.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.306+0000 7f26297fa640 1 -- 172.21.15.120:0/1323854643 shutdown_connections 2024-09-10T10:29:06.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.306+0000 7f26297fa640 1 -- 172.21.15.120:0/1323854643 >> 172.21.15.120:0/1323854643 conn(0x7f26440c89b0 msgr2=0x7f264406d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.306+0000 7f26297fa640 1 -- 172.21.15.120:0/1323854643 shutdown_connections 2024-09-10T10:29:06.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.306+0000 7f26297fa640 1 -- 172.21.15.120:0/1323854643 wait complete. 2024-09-10T10:29:06.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.306+0000 7f88c8ff9640 1 -- 172.21.15.120:0/1437953482 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f88cc0718c0 con 0x7f88b8075cd0 2024-09-10T10:29:06.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.307+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21e40c7490 con 0x7f21c8075cd0 2024-09-10T10:29:06.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.307+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f88b8075cd0 msgr2=0x7f88b8078170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.307+0000 7f88d26c9640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f88b8075cd0 0x7f88b8078170 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.307+0000 7f21cffff640 1 -- 172.21.15.120:0/1592600644 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21e40c7490 con 0x7f21c8075cd0 2024-09-10T10:29:06.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.307+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f88b806be30 msgr2=0x7f88b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.308+0000 7f88d26c9640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f88b806be30 0x7f88b806e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88c4036000 tx=0x7f88c4031000 comp rx=0 tx=0).stop 2024-09-10T10:29:06.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.308+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88cc150c10 msgr2=0x7f88cc1e0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.308+0000 7f88d26c9640 1 --2- 172.21.15.120:0/1437953482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88cc150c10 0x7f88cc1e0a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c550 tx=0x7f88c000ca10 comp rx=0 tx=0).stop 2024-09-10T10:29:06.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.308+0000 7f88d0c66640 1 -- 172.21.15.120:0/1437953482 reap_dead start 2024-09-10T10:29:06.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.308+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f21c8075cd0 msgr2=0x7f21c8078170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.308+0000 7f21e9ee6640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f21c8075cd0 0x7f21c8078170 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.308+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 shutdown_connections 2024-09-10T10:29:06.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.308+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 >> 172.21.15.120:0/1437953482 conn(0x7f88cc06f3c0 msgr2=0x7f88cc151b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7fec69ffb640 1 -- 172.21.15.120:0/3134724605 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec64002dd0 con 0x7fec70075d90 2024-09-10T10:29:06.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 shutdown_connections 2024-09-10T10:29:06.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7f88d26c9640 1 -- 172.21.15.120:0/1437953482 wait complete. 2024-09-10T10:29:06.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f21c806beb0 msgr2=0x7f21c806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7f21e9ee6640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f21c806beb0 0x7f21c806e370 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f21e4150eb0 tx=0x7f21d0008040 comp rx=0 tx=0).stop 2024-09-10T10:29:06.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e40cd2a0 msgr2=0x7f21e40c3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7f21e9ee6640 1 --2- 172.21.15.120:0/1592600644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e40cd2a0 0x7f21e40c3ba0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21d8000c00 tx=0x7f21d80046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7f21e3fff640 1 -- 172.21.15.120:0/1592600644 reap_dead start 2024-09-10T10:29:06.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7fec6bfff640 1 -- 172.21.15.120:0/3134724605 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fec64002dd0 con 0x7fec70075d90 2024-09-10T10:29:06.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 shutdown_connections 2024-09-10T10:29:06.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.309+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 >> 172.21.15.120:0/1592600644 conn(0x7f21e406cd20 msgr2=0x7f21e4070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 shutdown_connections 2024-09-10T10:29:06.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7f21e9ee6640 1 -- 172.21.15.120:0/1592600644 wait complete. 2024-09-10T10:29:06.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7fec88c58640 1 -- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fec70075d90 msgr2=0x7fec70078230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7fec88c58640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7fec70075d90 0x7fec70078230 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7fec88c58640 1 -- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec7006bf00 msgr2=0x7fec7006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7fec88c58640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fec7006bf00 0x7fec7006e3c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fec74034000 tx=0x7fec7402e000 comp rx=0 tx=0).stop 2024-09-10T10:29:06.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7fec88c58640 1 -- 172.21.15.120:0/3134724605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec84067d10 msgr2=0x7fec840c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7fec88c58640 1 --2- 172.21.15.120:0/3134724605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec84067d10 0x7fec840c5820 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec7800e9b0 tx=0x7fec7800ee70 comp rx=0 tx=0).stop 2024-09-10T10:29:06.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7fec83fff640 1 -- 172.21.15.120:0/3134724605 reap_dead start 2024-09-10T10:29:06.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7fec88c58640 1 -- 172.21.15.120:0/3134724605 shutdown_connections 2024-09-10T10:29:06.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.310+0000 7fec88c58640 1 -- 172.21.15.120:0/3134724605 >> 172.21.15.120:0/3134724605 conn(0x7fec840cc660 msgr2=0x7fec84153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.311+0000 7fec88c58640 1 -- 172.21.15.120:0/3134724605 shutdown_connections 2024-09-10T10:29:06.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.311+0000 7fec88c58640 1 -- 172.21.15.120:0/3134724605 wait complete. 2024-09-10T10:29:06.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.313+0000 7fe7b8874640 1 -- 172.21.15.120:0/2305688815 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7b4068ef0 con 0x7fe794075c90 2024-09-10T10:29:06.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.313+0000 7fefa0ca1640 1 -- 172.21.15.120:0/1722837195 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef940ddde0 con 0x7fef80075d00 2024-09-10T10:29:06.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.313+0000 7fe7aaffd640 1 -- 172.21.15.120:0/2305688815 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe7b4068ef0 con 0x7fe794075c90 2024-09-10T10:29:06.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.314+0000 7f01c9d8d640 1 -- 172.21.15.120:0/3148007956 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f01c414ae40 con 0x7f01a4075d20 2024-09-10T10:29:06.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.314+0000 7fe7a8ff9640 1 -- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe794075c90 msgr2=0x7fe794078130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.314+0000 7fe7a8ff9640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7fe794075c90 0x7fe794078130 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.314+0000 7fef87fff640 1 -- 172.21.15.120:0/1722837195 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fef940ddde0 con 0x7fef80075d00 2024-09-10T10:29:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.314+0000 7fe7a8ff9640 1 -- 172.21.15.120:0/2305688815 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe79406be80 msgr2=0x7fe79406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.314+0000 7fe7a8ff9640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe79406be80 0x7fe79406e340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009a50 tx=0x7fe7a4008040 comp rx=0 tx=0).stop 2024-09-10T10:29:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.314+0000 7fe7a8ff9640 1 -- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b415e9d0 msgr2=0x7fe7b40c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.314+0000 7fe7a8ff9640 1 --2- 172.21.15.120:0/2305688815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b415e9d0 0x7fe7b40c9da0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00efc0 tx=0x7fe7ac00c490 comp rx=0 tx=0).stop 2024-09-10T10:29:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.314+0000 7fe7b2d9d640 1 -- 172.21.15.120:0/2305688815 reap_dead start 2024-09-10T10:29:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fe7a8ff9640 1 -- 172.21.15.120:0/2305688815 shutdown_connections 2024-09-10T10:29:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fe7a8ff9640 1 -- 172.21.15.120:0/2305688815 >> 172.21.15.120:0/2305688815 conn(0x7fe7b406d560 msgr2=0x7fe7b40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fef85ffb640 1 -- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fef80075d00 msgr2=0x7fef800781a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fef85ffb640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fef80075d00 0x7fef800781a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fef85ffb640 1 -- 172.21.15.120:0/1722837195 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fef8006be60 msgr2=0x7fef8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fef85ffb640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fef8006be60 0x7fef8006e320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef88007ca0 tx=0x7fef88007c10 comp rx=0 tx=0).stop 2024-09-10T10:29:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fef85ffb640 1 -- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef94177d40 msgr2=0x7fef94178120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fef85ffb640 1 --2- 172.21.15.120:0/1722837195 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef94177d40 0x7fef94178120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef900099e0 tx=0x7fef90009190 comp rx=0 tx=0).stop 2024-09-10T10:29:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fef9bfff640 1 -- 172.21.15.120:0/1722837195 reap_dead start 2024-09-10T10:29:06.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fef85ffb640 1 -- 172.21.15.120:0/1722837195 shutdown_connections 2024-09-10T10:29:06.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fef85ffb640 1 -- 172.21.15.120:0/1722837195 >> 172.21.15.120:0/1722837195 conn(0x7fef94054070 msgr2=0x7fef94006970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fe7a8ff9640 1 -- 172.21.15.120:0/2305688815 shutdown_connections 2024-09-10T10:29:06.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7fe7a8ff9640 1 -- 172.21.15.120:0/2305688815 wait complete. 2024-09-10T10:29:06.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.315+0000 7f01c0ff9640 1 -- 172.21.15.120:0/3148007956 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f01c414ae40 con 0x7f01a4075d20 2024-09-10T10:29:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7fef85ffb640 1 -- 172.21.15.120:0/1722837195 shutdown_connections 2024-09-10T10:29:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7f01b9ffb640 1 -- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f01a4075d20 msgr2=0x7f01a40781c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7f01b9ffb640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7f01a4075d20 0x7f01a40781c0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7f01b9ffb640 1 -- 172.21.15.120:0/3148007956 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01a406be80 msgr2=0x7f01a406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7f01b9ffb640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f01a406be80 0x7f01a406e340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0017e0 tx=0x7f01ac009040 comp rx=0 tx=0).stop 2024-09-10T10:29:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7f01b9ffb640 1 -- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c40c9bf0 msgr2=0x7f01c4154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7f01b9ffb640 1 --2- 172.21.15.120:0/3148007956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c40c9bf0 0x7f01c4154a90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f01b4006d80 tx=0x7f01b400cbf0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7f01c3fff640 1 -- 172.21.15.120:0/3148007956 reap_dead start 2024-09-10T10:29:06.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7f01b9ffb640 1 -- 172.21.15.120:0/3148007956 shutdown_connections 2024-09-10T10:29:06.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7f01b9ffb640 1 -- 172.21.15.120:0/3148007956 >> 172.21.15.120:0/3148007956 conn(0x7f01c406bc10 msgr2=0x7f01c414fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7fef85ffb640 1 -- 172.21.15.120:0/1722837195 wait complete. 2024-09-10T10:29:06.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.316+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2350163110 con 0x7f2334075da0 2024-09-10T10:29:06.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.317+0000 7f2346ffd640 1 -- 172.21.15.120:0/1955758498 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2350163110 con 0x7f2334075da0 2024-09-10T10:29:06.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.317+0000 7f01b9ffb640 1 -- 172.21.15.120:0/3148007956 shutdown_connections 2024-09-10T10:29:06.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.317+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2334075da0 msgr2=0x7f2334078240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.317+0000 7f23584af640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f2334075da0 0x7f2334078240 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.317+0000 7f01b9ffb640 1 -- 172.21.15.120:0/3148007956 wait complete. 2024-09-10T10:29:06.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.318+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233406bf00 msgr2=0x7f233406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.318+0000 7f23584af640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f233406bf00 0x7f233406e3c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23480012e0 tx=0x7f2348009040 comp rx=0 tx=0).stop 2024-09-10T10:29:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.318+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23500cafe0 msgr2=0x7f2350161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.318+0000 7f23584af640 1 --2- 172.21.15.120:0/1955758498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23500cafe0 0x7f2350161710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c5f0 tx=0x7f234c00cab0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.318+0000 7f2356a4c640 1 -- 172.21.15.120:0/1955758498 reap_dead start 2024-09-10T10:29:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.318+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 shutdown_connections 2024-09-10T10:29:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.318+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 >> 172.21.15.120:0/1955758498 conn(0x7f235006f3a0 msgr2=0x7f235015f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.318+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 shutdown_connections 2024-09-10T10:29:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.318+0000 7f23584af640 1 -- 172.21.15.120:0/1955758498 wait complete. 2024-09-10T10:29:06.384 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:06.384 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T10:29:06.384 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:06.385 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:06.385 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T10:29:06.386 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T10:29:06.387 INFO:teuthology.orchestra.run.smithi120.stdout:55834574883 2024-09-10T10:29:06.387 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T10:29:06.388 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:06.388 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T10:29:06.389 INFO:teuthology.orchestra.run.smithi120.stdout:55834574883 2024-09-10T10:29:06.389 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T10:29:06.390 INFO:teuthology.orchestra.run.smithi120.stdout:55834574883 2024-09-10T10:29:06.390 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T10:29:06.391 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:06.391 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T10:29:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.590+0000 7f5dff59e640 1 Processor -- start 2024-09-10T10:29:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.590+0000 7f5dff59e640 1 -- start start 2024-09-10T10:29:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.592+0000 7f86faeac640 1 Processor -- start 2024-09-10T10:29:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.592+0000 7f86faeac640 1 -- start start 2024-09-10T10:29:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.592+0000 7f5dff59e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5df80d5850 0x7f5df8007730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.592+0000 7f5dff59e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df80dc3a0 con 0x7f5df804d310 2024-09-10T10:29:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.592+0000 7f5dff59e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df80dc0c0 con 0x7f5df8047a80 2024-09-10T10:29:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.592+0000 7f5dff59e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f5df80dbe90 con 0x7f5df80d5850 2024-09-10T10:29:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.592+0000 7f5dfed9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f5df8047a80 0x7f5df80e5d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:41910/0 (socket says 172.21.15.120:41910) 2024-09-10T10:29:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.592+0000 7f5dfdd9b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f5df804d310 0x7f5df8045d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:52308/0 (socket says 172.21.15.120:52308) 2024-09-10T10:29:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.592+0000 7f5dfed9d640 1 -- 172.21.15.120:0/703164212 learned_addr learned my addr 172.21.15.120:0/703164212 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.592+0000 7f5dfe59c640 1 --2- 172.21.15.120:0/703164212 >> v2:172.21.15.120:3300/0 conn(0x7f5df80d5850 0x7f5df8007730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 827351799 0 0) 0x7f5df80dc3a0 con 0x7f5df804d310 2024-09-10T10:29:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5de8003520 con 0x7f5df804d310 2024-09-10T10:29:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f86faeac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec0d8bb0 0x7f86ec04ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678332053 0 0) 0x7f5df80dc0c0 con 0x7f5df8047a80 2024-09-10T10:29:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5df80dc3a0 con 0x7f5df8047a80 2024-09-10T10:29:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 967924467 0 0) 0x7f5df80dc3a0 con 0x7f5df8047a80 2024-09-10T10:29:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 >> v2:172.21.15.120:3300/0 conn(0x7f5df80d5850 msgr2=0x7f5df8007730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f86faeac640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f86ec007340 0x7f86ec005d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f86faeac640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86ec050520 con 0x7f86ec04f870 2024-09-10T10:29:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f86faeac640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f86ec050800 con 0x7f86ec007340 2024-09-10T10:29:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f86faeac640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f86ec04f070 con 0x7f86ec0d8bb0 2024-09-10T10:29:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 --2- 172.21.15.120:0/703164212 >> v2:172.21.15.120:3300/0 conn(0x7f5df80d5850 0x7f5df8007730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 >> v1:172.21.15.120:6789/0 conn(0x7f5df804d310 legacy=0x7f5df8045d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df8046cc0 con 0x7f5df8047a80 2024-09-10T10:29:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f5e00050800 con 0x7f5df8047a80 2024-09-10T10:29:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 >> v1:172.21.15.134:6789/0 conn(0x7f5df8047a80 legacy=0x7f5df80e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5de8003b40 0x7f5de8003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5de8004800 0x7f5de8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5de800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5de800c030 con 0x7f5de80051c0 2024-09-10T10:29:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5de800c1e0 con 0x7f5de8003b40 2024-09-10T10:29:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5de800c3e0 con 0x7f5de8004800 2024-09-10T10:29:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfe59c640 1 --2- 172.21.15.120:0/703164212 >> v2:172.21.15.120:3300/0 conn(0x7f5df80d5850 0x7f5df8007730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f5dfed9d640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5de800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f86f3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f86ec007340 0x7f86ec005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f5dfdd9b640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5de8004800 0x7f5de8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f86f9449640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f86ec04f870 0x7f86ec00e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:41926/0 (socket says 172.21.15.120:41926) 2024-09-10T10:29:06.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f86f9449640 1 -- 172.21.15.120:0/4027575688 learned_addr learned my addr 172.21.15.120:0/4027575688 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:06.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f5dfe59c640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5de8003b40 0x7f5de8003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f86f8c48640 1 --2- 172.21.15.120:0/4027575688 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec0d8bb0 0x7f86ec04ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f42acf49640 1 Processor -- start 2024-09-10T10:29:06.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.593+0000 7f42acf49640 1 -- start start 2024-09-10T10:29:06.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f86f3fff640 1 --2- 172.21.15.120:0/4027575688 >> v2:172.21.15.134:3300/0 conn(0x7f86ec007340 0x7f86ec005d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:46758/0 (socket says 172.21.15.120:46758) 2024-09-10T10:29:06.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f86f37fe640 1 -- 172.21.15.120:0/4027575688 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 521199635 0 0) 0x7f86ec050520 con 0x7f86ec04f870 2024-09-10T10:29:06.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f04013cf640 1 Processor -- start 2024-09-10T10:29:06.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f86f37fe640 1 -- 172.21.15.120:0/4027575688 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86e4003520 con 0x7f86ec04f870 2024-09-10T10:29:06.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f04013cf640 1 -- start start 2024-09-10T10:29:06.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42acf49640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f42a00d6a60 0x7f42a00dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f86f8c48640 1 -- 172.21.15.120:0/4027575688 >> v2:172.21.15.134:3300/0 conn(0x7f86ec007340 msgr2=0x7f86ec005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f86f37fe640 1 -- 172.21.15.120:0/4027575688 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1588907280 0 0) 0x7f86e4003520 con 0x7f86ec04f870 2024-09-10T10:29:06.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42acf49640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f42a00dd060 0x7f42a00dca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f86f8c48640 1 --2- 172.21.15.120:0/4027575688 >> v2:172.21.15.134:3300/0 conn(0x7f86ec007340 0x7f86ec005d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f86f8c48640 1 -- 172.21.15.120:0/4027575688 >> v1:172.21.15.134:6789/0 conn(0x7f86ec04f870 legacy=0x7f86ec00e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f86f8c48640 1 -- 172.21.15.120:0/4027575688 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec04f350 con 0x7f86ec0d8bb0 2024-09-10T10:29:06.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42acf49640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42a004ca20 con 0x7f42a00d6680 2024-09-10T10:29:06.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42acf49640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f42a004c7f0 con 0x7f42a00d6a60 2024-09-10T10:29:06.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42acf49640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f42a004cce0 con 0x7f42a00dd060 2024-09-10T10:29:06.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42a659c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f42a00d6680 0x7f42a0046f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:41932/0 (socket says 172.21.15.120:41932) 2024-09-10T10:29:06.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42a659c640 1 -- 172.21.15.120:0/2634767984 learned_addr learned my addr 172.21.15.120:0/2634767984 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:06.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42a6d9d640 1 --2- 172.21.15.120:0/2634767984 >> v2:172.21.15.134:3300/0 conn(0x7f42a00dd060 0x7f42a00dca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42a6d9d640 1 -- 172.21.15.120:0/2634767984 >> v2:172.21.15.120:3300/0 conn(0x7f42a00d6a60 msgr2=0x7f42a00dc4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:29:06.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42a6d9d640 1 --2- 172.21.15.120:0/2634767984 >> v2:172.21.15.120:3300/0 conn(0x7f42a00d6a60 0x7f42a00dc4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42a6d9d640 1 -- 172.21.15.120:0/2634767984 >> v1:172.21.15.134:6789/0 conn(0x7f42a00d6680 legacy=0x7f42a0046f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f42a6d9d640 1 -- 172.21.15.120:0/2634767984 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a004cfc0 con 0x7f42a00dd060 2024-09-10T10:29:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f5dfed9d640 1 -- 172.21.15.120:0/703164212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5de8004800 msgr2=0x7f5de8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f5dfed9d640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5de8004800 0x7f5de8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f5dfed9d640 1 -- 172.21.15.120:0/703164212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5de8003b40 msgr2=0x7f5de8003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f5dfed9d640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5de8003b40 0x7f5de8003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f5dfed9d640 1 -- 172.21.15.120:0/703164212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df8046cc0 con 0x7f5de80051c0 2024-09-10T10:29:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.594+0000 7f5dfe59c640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5de8003b40 0x7f5de8003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f5dfed9d640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5de800b9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5e00000b60 tx=0x7f5de8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52311e88e4d44982 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f5dfed9d640 1 -- 172.21.15.120:0/703164212 reap_dead start 2024-09-10T10:29:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e000b7070 con 0x7f5de80051c0 2024-09-10T10:29:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.595+0000 7f5dfd59a640 1 -- 172.21.15.120:0/703164212 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e000b2400 con 0x7f5de80051c0 2024-09-10T10:29:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f04013cf640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f03fc150c10 0x7f03fc072d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f86f3fff640 1 --2- 172.21.15.120:0/4027575688 >> v2:172.21.15.134:3300/0 conn(0x7f86ec007340 0x7f86ec005d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:29:06.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f04013cf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc0c9650 0x7f03fc071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f04013cf640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f03fc072410 0x7f03fc156e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f04013cf640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f03fc1569d0 con 0x7f03fc150c10 2024-09-10T10:29:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f04013cf640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f03fc073260 con 0x7f03fc072410 2024-09-10T10:29:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f04013cf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc06dbd0 con 0x7f03fc0c9650 2024-09-10T10:29:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f4acbfff640 1 Processor -- start 2024-09-10T10:29:06.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f03faffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f03fc150c10 0x7f03fc072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f1acf59e640 1 Processor -- start 2024-09-10T10:29:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f1acf59e640 1 -- start start 2024-09-10T10:29:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f5dff59e640 1 -- 172.21.15.120:0/703164212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 msgr2=0x7f5de800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f5dff59e640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5de800b9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5e00000b60 tx=0x7f5de8005830 comp rx=0 tx=0).stop 2024-09-10T10:29:06.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f03faffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f03fc150c10 0x7f03fc072d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33064/0 (socket says 172.21.15.120:33064) 2024-09-10T10:29:06.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f03faffd640 1 -- 172.21.15.120:0/937097505 learned_addr learned my addr 172.21.15.120:0/937097505 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f03fb7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f03fc072410 0x7f03fc156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f5dff59e640 1 -- 172.21.15.120:0/703164212 shutdown_connections 2024-09-10T10:29:06.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f5dff59e640 1 --2- 172.21.15.120:0/703164212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5de800b9c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f5dff59e640 1 -- 172.21.15.120:0/703164212 >> 172.21.15.120:0/703164212 conn(0x7f5df804d830 msgr2=0x7f5df804dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f86f8c48640 1 --2- 172.21.15.120:0/4027575688 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec0d8bb0 0x7f86ec04ea00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86f40518b0 tx=0x7f86f40639f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbed675c4e870432 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f86f37fe640 1 -- 172.21.15.120:0/4027575688 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40af070 con 0x7f86ec0d8bb0 2024-09-10T10:29:06.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f86f37fe640 1 -- 172.21.15.120:0/4027575688 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f40ab4d0 con 0x7f86ec0d8bb0 2024-09-10T10:29:06.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f86f37fe640 1 -- 172.21.15.120:0/4027575688 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0001670 con 0x7f86ec0d8bb0 2024-09-10T10:29:06.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f03f27fc640 1 --2- 172.21.15.120:0/937097505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc0c9650 0x7f03fc071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f03fb7fe640 1 -- 172.21.15.120:0/937097505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc0c9650 msgr2=0x7f03fc071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f03fb7fe640 1 --2- 172.21.15.120:0/937097505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc0c9650 0x7f03fc071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f03fb7fe640 1 -- 172.21.15.120:0/937097505 >> v2:172.21.15.120:3300/0 conn(0x7f03fc150c10 msgr2=0x7f03fc072d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f03fb7fe640 1 --2- 172.21.15.120:0/937097505 >> v2:172.21.15.120:3300/0 conn(0x7f03fc150c10 0x7f03fc072d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f03fb7fe640 1 -- 172.21.15.120:0/937097505 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc06deb0 con 0x7f03fc072410 2024-09-10T10:29:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f1acf59e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1ac80dce90 0x7f1ac8005a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f42a6d9d640 1 --2- 172.21.15.120:0/2634767984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a00dd060 0x7f42a00dca10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f42a80518b0 tx=0x7f42a80639f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7610097cd57f767b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.596+0000 7f4acbfff640 1 -- start start 2024-09-10T10:29:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f4acbfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4acc06d450 0x7f4acc0c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f4acbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 0x7f4acc15e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f4acbfff640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4acc0b6610 con 0x7f4acc06d830 2024-09-10T10:29:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f4acbfff640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4acc0b5850 con 0x7f4acc06d450 2024-09-10T10:29:06.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f4acbfff640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4acc0cc120 con 0x7f4acc0b3bf0 2024-09-10T10:29:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f42a559a640 1 -- 172.21.15.120:0/2634767984 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a80bc070 con 0x7f42a00dd060 2024-09-10T10:29:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f1acf59e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac80e6ae0 con 0x7f1ac80e6030 2024-09-10T10:29:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f1acf59e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac80e6dc0 con 0x7f1ac804aa40 2024-09-10T10:29:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.597+0000 7f1acf59e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac8045660 con 0x7f1ac80dce90 2024-09-10T10:29:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f1ace59c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1ac80dce90 0x7f1ac8005a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f1ace59c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1ac80dce90 0x7f1ac8005a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:46792/0 (socket says 172.21.15.120:46792) 2024-09-10T10:29:06.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f1ace59c640 1 -- 172.21.15.120:0/3577736014 learned_addr learned my addr 172.21.15.120:0/3577736014 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f42a559a640 1 -- 172.21.15.120:0/2634767984 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a80b7640 con 0x7f42a00dd060 2024-09-10T10:29:06.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f42a559a640 1 -- 172.21.15.120:0/2634767984 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a80b7a70 con 0x7f42a00dd060 2024-09-10T10:29:06.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f1ace59c640 1 -- 172.21.15.120:0/3577736014 >> v1:172.21.15.134:6789/0 conn(0x7f1ac804aa40 legacy=0x7f1ac8046390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:29:06.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f5dff59e640 1 -- 172.21.15.120:0/703164212 shutdown_connections 2024-09-10T10:29:06.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f4aca7fc640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f4acc06d830 0x7f4acc0b34e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:41946/0 (socket says 172.21.15.120:41946) 2024-09-10T10:29:06.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f4aca7fc640 1 -- 172.21.15.120:0/3064313359 learned_addr learned my addr 172.21.15.120:0/3064313359 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:06.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f5dff59e640 1 -- 172.21.15.120:0/703164212 wait complete. 2024-09-10T10:29:06.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f1ace59c640 1 -- 172.21.15.120:0/3577736014 >> v1:172.21.15.120:6789/0 conn(0x7f1ac80e6030 legacy=0x7f1ac8044b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:29:06.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f86faeac640 1 -- 172.21.15.120:0/4027575688 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec0d8bb0 msgr2=0x7f86ec04ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f86faeac640 1 --2- 172.21.15.120:0/4027575688 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec0d8bb0 0x7f86ec04ea00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86f40518b0 tx=0x7f86f40639f0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f86faeac640 1 -- 172.21.15.120:0/4027575688 shutdown_connections 2024-09-10T10:29:06.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f86faeac640 1 --2- 172.21.15.120:0/4027575688 >> v2:172.21.15.134:3300/0 conn(0x7f86ec007340 0x7f86ec005d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f86faeac640 1 --2- 172.21.15.120:0/4027575688 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec0d8bb0 0x7f86ec04ea00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7fc571415640 1 Processor -- start 2024-09-10T10:29:06.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f86faeac640 1 -- 172.21.15.120:0/4027575688 >> 172.21.15.120:0/4027575688 conn(0x7f86ec054470 msgr2=0x7f86ec054880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7fc571415640 1 -- start start 2024-09-10T10:29:06.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f86faeac640 1 -- 172.21.15.120:0/4027575688 shutdown_connections 2024-09-10T10:29:06.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f86faeac640 1 -- 172.21.15.120:0/4027575688 wait complete. 2024-09-10T10:29:06.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f03fb7fe640 1 --2- 172.21.15.120:0/937097505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc072410 0x7f03fc156e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f03fc069a10 tx=0x7f03f402ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a5bfe7331ce0b2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.598+0000 7f1ace59c640 1 -- 172.21.15.120:0/3577736014 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac80458b0 con 0x7f1ac80dce90 2024-09-10T10:29:06.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f4acaffd640 1 --2- 172.21.15.120:0/3064313359 >> v2:172.21.15.120:3300/0 conn(0x7f4acc06d450 0x7f4acc0c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f42acf49640 1 -- 172.21.15.120:0/2634767984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a00dd060 msgr2=0x7f42a00dca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f42acf49640 1 --2- 172.21.15.120:0/2634767984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a00dd060 0x7f42a00dca10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f42a80518b0 tx=0x7f42a80639f0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f42acf49640 1 -- 172.21.15.120:0/2634767984 shutdown_connections 2024-09-10T10:29:06.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f42acf49640 1 --2- 172.21.15.120:0/2634767984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a00dd060 0x7f42a00dca10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f42acf49640 1 --2- 172.21.15.120:0/2634767984 >> v2:172.21.15.120:3300/0 conn(0x7f42a00d6a60 0x7f42a00dc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f42acf49640 1 -- 172.21.15.120:0/2634767984 >> 172.21.15.120:0/2634767984 conn(0x7f42a004e8f0 msgr2=0x7f42a004ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f42acf49640 1 -- 172.21.15.120:0/2634767984 shutdown_connections 2024-09-10T10:29:06.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f42acf49640 1 -- 172.21.15.120:0/2634767984 wait complete. 2024-09-10T10:29:06.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f03fa7fc640 1 -- 172.21.15.120:0/937097505 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f402f720 con 0x7f03fc072410 2024-09-10T10:29:06.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f03fa7fc640 1 -- 172.21.15.120:0/937097505 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f402f8c0 con 0x7f03fc072410 2024-09-10T10:29:06.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f03fa7fc640 1 -- 172.21.15.120:0/937097505 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f402fc90 con 0x7f03fc072410 2024-09-10T10:29:06.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f4acb7fe640 1 --2- 172.21.15.120:0/3064313359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 0x7f4acc15e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f4acaffd640 1 -- 172.21.15.120:0/3064313359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 msgr2=0x7f4acc15e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f4acaffd640 1 --2- 172.21.15.120:0/3064313359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 0x7f4acc15e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f4acaffd640 1 -- 172.21.15.120:0/3064313359 >> v1:172.21.15.134:6789/0 conn(0x7f4acc06d830 legacy=0x7f4acc0b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f4acaffd640 1 -- 172.21.15.120:0/3064313359 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc0c6ff0 con 0x7f4acc06d450 2024-09-10T10:29:06.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.599+0000 7f4acb7fe640 1 --2- 172.21.15.120:0/3064313359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 0x7f4acc15e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:06.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7f5dff59e640 1 Processor -- start 2024-09-10T10:29:06.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7fc571415640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc56404d310 0x7fc564045d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7fc571415640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 0x7fc564046290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7fc56bfff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fc5640d5850 0x7fc564007730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:52318/0 (socket says 172.21.15.120:52318) 2024-09-10T10:29:06.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7fc56bfff640 1 -- 172.21.15.120:0/3364494456 learned_addr learned my addr 172.21.15.120:0/3364494456 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:06.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7f5dff59e640 1 -- start start 2024-09-10T10:29:06.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7f04013cf640 1 -- 172.21.15.120:0/937097505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc072410 msgr2=0x7f03fc156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7f04013cf640 1 --2- 172.21.15.120:0/937097505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc072410 0x7f03fc156e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f03fc069a10 tx=0x7f03f402ede0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7fc571415640 1 -- 172.21.15.120:0/3364494456 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5640dc3a0 con 0x7fc5640d5850 2024-09-10T10:29:06.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7fc571415640 1 -- 172.21.15.120:0/3364494456 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc5640dbe90 con 0x7fc56404d310 2024-09-10T10:29:06.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7fc571415640 1 -- 172.21.15.120:0/3364494456 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc5640dc0c0 con 0x7fc564047a80 2024-09-10T10:29:06.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7fc570c14640 1 --2- 172.21.15.120:0/3364494456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 0x7fc564046290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7f1ace59c640 1 --2- 172.21.15.120:0/3577736014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac80dce90 0x7f1ac8005a00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009870 tx=0x7f1ac402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb529ec16d22bc5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.600+0000 7f4acaffd640 1 --2- 172.21.15.120:0/3064313359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4acc06d450 0x7f4acc0c5250 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00097a0 tx=0x7f4ac002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3541328441b321ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f86faeac640 1 Processor -- start 2024-09-10T10:29:06.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f86faeac640 1 -- start start 2024-09-10T10:29:06.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7fc56b7fe640 1 --2- 172.21.15.120:0/3364494456 >> v2:172.21.15.134:3300/0 conn(0x7fc56404d310 0x7fc564045d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f1acd59a640 1 -- 172.21.15.120:0/3577736014 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4037070 con 0x7f1ac80dce90 2024-09-10T10:29:06.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f1acd59a640 1 -- 172.21.15.120:0/3577736014 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac4032440 con 0x7f1ac80dce90 2024-09-10T10:29:06.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f1acd59a640 1 -- 172.21.15.120:0/3577736014 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4032810 con 0x7f1ac80dce90 2024-09-10T10:29:06.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7fc56affd640 1 -- 172.21.15.120:0/3364494456 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2873123246 0 0) 0x7fc5640dc3a0 con 0x7fc5640d5850 2024-09-10T10:29:06.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7fc56affd640 1 -- 172.21.15.120:0/3364494456 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc55c005730 con 0x7fc5640d5850 2024-09-10T10:29:06.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f42acf49640 1 Processor -- start 2024-09-10T10:29:06.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f04013cf640 1 -- 172.21.15.120:0/937097505 shutdown_connections 2024-09-10T10:29:06.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f04013cf640 1 --2- 172.21.15.120:0/937097505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc072410 0x7f03fc156e60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f04013cf640 1 --2- 172.21.15.120:0/937097505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc0c9650 0x7f03fc071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f04013cf640 1 --2- 172.21.15.120:0/937097505 >> v2:172.21.15.120:3300/0 conn(0x7f03fc150c10 0x7f03fc072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f04013cf640 1 -- 172.21.15.120:0/937097505 >> 172.21.15.120:0/937097505 conn(0x7f03fc06f3c0 msgr2=0x7f03fc06f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f4ac9ffb640 1 -- 172.21.15.120:0/3064313359 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0037070 con 0x7f4acc06d450 2024-09-10T10:29:06.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f4ac9ffb640 1 -- 172.21.15.120:0/3064313359 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac0032440 con 0x7f4acc06d450 2024-09-10T10:29:06.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f4ac9ffb640 1 -- 172.21.15.120:0/3064313359 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0032810 con 0x7f4acc06d450 2024-09-10T10:29:06.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f5dff59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5df8165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f5dff59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df8165530 0x7f5df80cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f5dff59e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5df8048130 0x7f5df8007670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f5dff59e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8047560 con 0x7f5de80051c0 2024-09-10T10:29:06.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f5dfe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5df8165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7fc570c14640 1 -- 172.21.15.120:0/3364494456 >> v2:172.21.15.134:3300/0 conn(0x7fc56404d310 msgr2=0x7fc564045d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7fc570c14640 1 --2- 172.21.15.120:0/3364494456 >> v2:172.21.15.134:3300/0 conn(0x7fc56404d310 0x7fc564045d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7fc570c14640 1 -- 172.21.15.120:0/3364494456 >> v1:172.21.15.120:6789/0 conn(0x7fc5640d5850 legacy=0x7fc564007730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f5dfe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5df8165ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33088/0 (socket says 172.21.15.120:33088) 2024-09-10T10:29:06.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f5dfe59c640 1 -- 172.21.15.120:0/2499303219 learned_addr learned my addr 172.21.15.120:0/2499303219 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7fc570c14640 1 -- 172.21.15.120:0/3364494456 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc564046cc0 con 0x7fc564047a80 2024-09-10T10:29:06.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8047280 con 0x7f5df8165530 2024-09-10T10:29:06.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.601+0000 7f42acf49640 1 -- start start 2024-09-10T10:29:06.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5df80468c0 con 0x7f5df8048130 2024-09-10T10:29:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f5dfed9d640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5df8048130 0x7f5df8007670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f1acf59e640 1 -- 172.21.15.120:0/3577736014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac80dce90 msgr2=0x7f1ac8005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7fd321bf1640 1 Processor -- start 2024-09-10T10:29:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f1acf59e640 1 --2- 172.21.15.120:0/3577736014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac80dce90 0x7f1ac8005a00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009870 tx=0x7f1ac402a270 comp rx=0 tx=0).stop 2024-09-10T10:29:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7fd321bf1640 1 -- start start 2024-09-10T10:29:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f5dfdd9b640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df8165530 0x7f5df80cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f42acf49640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42a00d6680 0x7f42a00d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f86faeac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec007340 0x7f86ec0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f4acbfff640 1 -- 172.21.15.120:0/3064313359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4acc06d450 msgr2=0x7f4acc0c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f4acbfff640 1 --2- 172.21.15.120:0/3064313359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4acc06d450 0x7f4acc0c5250 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00097a0 tx=0x7f4ac002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f42acf49640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42a00d6a60 0x7f42a0174010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f5dfed9d640 1 -- 172.21.15.120:0/2499303219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df8165530 msgr2=0x7f5df80cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f4acbfff640 1 -- 172.21.15.120:0/3064313359 shutdown_connections 2024-09-10T10:29:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f4acbfff640 1 --2- 172.21.15.120:0/3064313359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 0x7f4acc15e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f4acbfff640 1 --2- 172.21.15.120:0/3064313359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4acc06d450 0x7f4acc0c5250 secure :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00097a0 tx=0x7f4ac002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f4acbfff640 1 -- 172.21.15.120:0/3064313359 >> 172.21.15.120:0/3064313359 conn(0x7f4acc0d0250 msgr2=0x7f4acc0d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f42acf49640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a00dd060 0x7f42a0050f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f42acf49640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42a0047690 con 0x7f42a00d6680 2024-09-10T10:29:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f42a659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42a00d6680 0x7f42a00d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f42a659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42a00d6680 0x7f42a00d6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33094/0 (socket says 172.21.15.120:33094) 2024-09-10T10:29:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f42a659c640 1 -- 172.21.15.120:0/269290847 learned_addr learned my addr 172.21.15.120:0/269290847 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f42acf49640 1 -- 172.21.15.120:0/269290847 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42a004d580 con 0x7f42a00dd060 2024-09-10T10:29:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f42acf49640 1 -- 172.21.15.120:0/269290847 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f42a0047810 con 0x7f42a00d6a60 2024-09-10T10:29:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f86faeac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec04f870 0x7f86ec037cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86f8c48640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec007340 0x7f86ec0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f1acf59e640 1 -- 172.21.15.120:0/3577736014 shutdown_connections 2024-09-10T10:29:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f1acf59e640 1 --2- 172.21.15.120:0/3577736014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac80dce90 0x7f1ac8005a00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f1acf59e640 1 -- 172.21.15.120:0/3577736014 >> 172.21.15.120:0/3577736014 conn(0x7f1ac804fc50 msgr2=0x7f1ac8050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f1acf59e640 1 -- 172.21.15.120:0/3577736014 shutdown_connections 2024-09-10T10:29:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f1acf59e640 1 -- 172.21.15.120:0/3577736014 wait complete. 2024-09-10T10:29:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86f8c48640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec007340 0x7f86ec0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33098/0 (socket says 172.21.15.120:33098) 2024-09-10T10:29:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f42a5d9b640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42a00d6a60 0x7f42a0174010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f4acbfff640 1 -- 172.21.15.120:0/3064313359 shutdown_connections 2024-09-10T10:29:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86f8c48640 1 -- 172.21.15.120:0/2328868872 learned_addr learned my addr 172.21.15.120:0/2328868872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86faeac640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86ec0d8bb0 0x7f86ec038230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f42a6d9d640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a00dd060 0x7f42a0050f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f04013cf640 1 -- 172.21.15.120:0/937097505 shutdown_connections 2024-09-10T10:29:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.602+0000 7f04013cf640 1 -- 172.21.15.120:0/937097505 wait complete. 2024-09-10T10:29:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec049340 con 0x7f86ec007340 2024-09-10T10:29:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f4acbfff640 1 -- 172.21.15.120:0/3064313359 wait complete. 2024-09-10T10:29:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec0491c0 con 0x7f86ec0d8bb0 2024-09-10T10:29:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86f3fff640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec04f870 0x7f86ec037cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f86ec00eae0 con 0x7f86ec04f870 2024-09-10T10:29:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7fc570c14640 1 --2- 172.21.15.120:0/3364494456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 0x7fc564046290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc56c02dd50 tx=0x7fc56c0a6ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6948c0fca9f7519 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86f9449640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86ec0d8bb0 0x7f86ec038230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f42a659c640 1 -- 172.21.15.120:0/269290847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42a00d6a60 msgr2=0x7f42a0174010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f42a659c640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42a00d6a60 0x7f42a0174010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f42a659c640 1 -- 172.21.15.120:0/269290847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a00dd060 msgr2=0x7f42a0050f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f42a659c640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a00dd060 0x7f42a0050f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86f8c48640 1 -- 172.21.15.120:0/2328868872 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec04f870 msgr2=0x7f86ec037cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86f8c48640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec04f870 0x7f86ec037cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f5dfed9d640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df8165530 0x7f5df80cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f5dfed9d640 1 -- 172.21.15.120:0/2499303219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 msgr2=0x7f5df8165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f5dfed9d640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5df8165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f5dfed9d640 1 -- 172.21.15.120:0/2499303219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df8165990 con 0x7f5df8048130 2024-09-10T10:29:06.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f5dfe59c640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5df8165ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:06.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86f8c48640 1 -- 172.21.15.120:0/2328868872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86ec0d8bb0 msgr2=0x7f86ec038230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86f8c48640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86ec0d8bb0 0x7f86ec038230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f5dfed9d640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5df8048130 0x7f5df8007670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e00074ad0 tx=0x7f5e00074480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f86f8c48640 1 -- 172.21.15.120:0/2328868872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec1749b0 con 0x7f86ec007340 2024-09-10T10:29:06.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.603+0000 7f42a659c640 1 -- 172.21.15.120:0/269290847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a0008de0 con 0x7f42a00d6680 2024-09-10T10:29:06.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f42a6d9d640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a00dd060 0x7f42a0050f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:06.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f42a659c640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42a00d6680 0x7f42a00d6ff0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c480 tx=0x7f429c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f5deeffd640 1 -- 172.21.15.120:0/2499303219 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0004ffb0 con 0x7f5df8048130 2024-09-10T10:29:06.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f86f8c48640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec007340 0x7f86ec0ce820 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f86f402dd50 tx=0x7f86f4074750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df80cc000 con 0x7f5df8048130 2024-09-10T10:29:06.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7fd321bf1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c152a30 0x7fd31c15c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7fd321bf1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd31c158800 0x7fd31c0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7fd321bf1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd31c0c93b0 0x7fd31c0c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7fd321bf1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fd31c15ccc0 con 0x7fd31c158800 2024-09-10T10:29:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7fd321bf1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd31c0c7640 con 0x7fd31c0c93b0 2024-09-10T10:29:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7fd321bf1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c06ed30 con 0x7fd31c152a30 2024-09-10T10:29:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f5deeffd640 1 -- 172.21.15.120:0/2499303219 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e00050520 con 0x7f5df8048130 2024-09-10T10:29:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f04013cf640 1 Processor -- start 2024-09-10T10:29:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f5deeffd640 1 -- 172.21.15.120:0/2499303219 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0004ffb0 con 0x7f5df8048130 2024-09-10T10:29:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f1acf59e640 1 Processor -- start 2024-09-10T10:29:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f86f17fa640 1 -- 172.21.15.120:0/2328868872 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40c4070 con 0x7f86ec007340 2024-09-10T10:29:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df8049d70 con 0x7f5df8048130 2024-09-10T10:29:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f1acf59e640 1 -- start start 2024-09-10T10:29:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7fd31bfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd31c158800 0x7fd31c0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7fd31bfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd31c158800 0x7fd31c0c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33102/0 (socket says 172.21.15.120:33102) 2024-09-10T10:29:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7fd31bfff640 1 -- 172.21.15.120:0/483045156 learned_addr learned my addr 172.21.15.120:0/483045156 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f86f17fa640 1 -- 172.21.15.120:0/2328868872 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f40b5a10 con 0x7f86ec007340 2024-09-10T10:29:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f04013cf640 1 -- start start 2024-09-10T10:29:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7fd320bef640 1 --2- 172.21.15.120:0/483045156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c152a30 0x7fd31c15c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7fd31bfff640 1 -- 172.21.15.120:0/483045156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c152a30 msgr2=0x7fd31c15c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7fd31bfff640 1 --2- 172.21.15.120:0/483045156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c152a30 0x7fd31c15c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7fd31bfff640 1 -- 172.21.15.120:0/483045156 >> v2:172.21.15.134:3300/0 conn(0x7fd31c0c93b0 msgr2=0x7fd31c0c7ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:29:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7fd31bfff640 1 --2- 172.21.15.120:0/483045156 >> v2:172.21.15.134:3300/0 conn(0x7fd31c0c93b0 0x7fd31c0c7ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7fd31bfff640 1 -- 172.21.15.120:0/483045156 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c158f90 con 0x7fd31c158800 2024-09-10T10:29:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7f4acbfff640 1 Processor -- start 2024-09-10T10:29:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7f4acbfff640 1 -- start start 2024-09-10T10:29:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f4296ffd640 1 -- 172.21.15.120:0/269290847 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c017070 con 0x7f42a00d6680 2024-09-10T10:29:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.604+0000 7f4296ffd640 1 -- 172.21.15.120:0/269290847 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429c005ce0 con 0x7f42a00d6680 2024-09-10T10:29:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7f4296ffd640 1 -- 172.21.15.120:0/269290847 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c0072d0 con 0x7f42a00d6680 2024-09-10T10:29:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7f42acf49640 1 -- 172.21.15.120:0/269290847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a00090c0 con 0x7f42a00d6680 2024-09-10T10:29:06.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7f86f17fa640 1 -- 172.21.15.120:0/2328868872 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40af040 con 0x7f86ec007340 2024-09-10T10:29:06.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ec174c70 con 0x7f86ec007340 2024-09-10T10:29:06.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ec175230 con 0x7f86ec007340 2024-09-10T10:29:06.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.605+0000 7f42acf49640 1 -- 172.21.15.120:0/269290847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a0009680 con 0x7f42a00d6680 2024-09-10T10:29:06.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f04013cf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc072410 0x7f03fc1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f1acf59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ac804aa40 0x7f1ac8049940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f04013cf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc0c9650 0x7f03fc14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f4acbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4acc06d830 0x7f4acc158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f04013cf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03fc150c10 0x7f03fc1deca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f1acf59e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ac80dce90 0x7f1ac80cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f1acf59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac80e6030 0x7f1ac80cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f1ace59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ac804aa40 0x7f1ac8049940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f4acbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 0x7f4acc0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f1ace59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ac804aa40 0x7f1ac8049940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33110/0 (socket says 172.21.15.120:33110) 2024-09-10T10:29:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f1acf59e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac80dd270 con 0x7f1ac804aa40 2024-09-10T10:29:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f1acf59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8045f00 con 0x7f1ac80e6030 2024-09-10T10:29:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f1acf59e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac8045240 con 0x7f1ac80dce90 2024-09-10T10:29:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f1ace59c640 1 -- 172.21.15.120:0/3360430997 learned_addr learned my addr 172.21.15.120:0/3360430997 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f4acbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4acc0c8920 0x7f4acc0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f04013cf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc06e750 con 0x7f03fc150c10 2024-09-10T10:29:06.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f04013cf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc06e470 con 0x7f03fc0c9650 2024-09-10T10:29:06.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f4acaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4acc06d830 0x7f4acc158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f04013cf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc072a10 con 0x7f03fc072410 2024-09-10T10:29:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f4acbfff640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc0cc370 con 0x7f4acc0c8920 2024-09-10T10:29:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f03faffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc072410 0x7f03fc1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f4acbfff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc0cc6e0 con 0x7f4acc06d830 2024-09-10T10:29:06.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f03faffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc072410 0x7f03fc1511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:57480/0 (socket says 172.21.15.120:57480) 2024-09-10T10:29:06.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f03faffd640 1 -- 172.21.15.120:0/1465606301 learned_addr learned my addr 172.21.15.120:0/1465606301 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f4acbfff640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4acc0b5360 con 0x7f4acc0b3bf0 2024-09-10T10:29:06.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f4acaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4acc06d830 0x7f4acc158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:46844/0 (socket says 172.21.15.120:46844) 2024-09-10T10:29:06.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f4acaffd640 1 -- 172.21.15.120:0/1908891658 learned_addr learned my addr 172.21.15.120:0/1908891658 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7fc56affd640 1 -- 172.21.15.120:0/3364494456 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0a88f0 con 0x7fc564047a80 2024-09-10T10:29:06.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7fc56affd640 1 -- 172.21.15.120:0/3364494456 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c0a8a90 con 0x7fc564047a80 2024-09-10T10:29:06.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7fc56affd640 1 -- 172.21.15.120:0/3364494456 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0b2400 con 0x7fc564047a80 2024-09-10T10:29:06.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f5deeffd640 1 -- 172.21.15.120:0/2499303219 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e000b15b0 con 0x7f5df8048130 2024-09-10T10:29:06.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f4294ff9640 1 -- 172.21.15.120:0/269290847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a0046f80 con 0x7f42a00d6680 2024-09-10T10:29:06.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f03fb7fe640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03fc150c10 0x7f03fc1deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f03f27fc640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc0c9650 0x7f03fc14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f1acdd9b640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ac80dce90 0x7f1ac80cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.606+0000 7f4aca7fc640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 0x7f4acc0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f4acaffd640 1 -- 172.21.15.120:0/1908891658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 msgr2=0x7f4acc0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f4acaffd640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 0x7f4acc0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f4acaffd640 1 -- 172.21.15.120:0/1908891658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4acc0c8920 msgr2=0x7f4acc0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f4acaffd640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4acc0c8920 0x7f4acc0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f4acaffd640 1 -- 172.21.15.120:0/1908891658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc0c92f0 con 0x7f4acc06d830 2024-09-10T10:29:06.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f4aca7fc640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 0x7f4acc0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:06.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f4acaffd640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4acc06d830 0x7f4acc158d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00098c0 tx=0x7f4ac00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f1aced9d640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac80e6030 0x7f1ac80cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f5deeffd640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5de006c030 0x7f5de006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de4005290 con 0x7f5df8048130 2024-09-10T10:29:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f5deeffd640 1 -- 172.21.15.120:0/2499303219 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f5e00126980 con 0x7f5df8048130 2024-09-10T10:29:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f03fb7fe640 1 -- 172.21.15.120:0/1465606301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc072410 msgr2=0x7f03fc1511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f03fb7fe640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc072410 0x7f03fc1511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f03fb7fe640 1 -- 172.21.15.120:0/1465606301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc0c9650 msgr2=0x7f03fc14f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f03fb7fe640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc0c9650 0x7f03fc14f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7f03fb7fe640 1 -- 172.21.15.120:0/1465606301 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc1ed160 con 0x7f03fc150c10 2024-09-10T10:29:06.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f03f27fc640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc0c9650 0x7f03fc14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:06.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7fd31bfff640 1 --2- 172.21.15.120:0/483045156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 0x7fd31c0c8e70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0cbe40 tx=0x7fd31002a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d93ebbfde69aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7fd31b7fe640 1 -- 172.21.15.120:0/483045156 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310037070 con 0x7fd31c158800 2024-09-10T10:29:06.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7fd31b7fe640 1 -- 172.21.15.120:0/483045156 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd310032440 con 0x7fd31c158800 2024-09-10T10:29:06.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.607+0000 7fd31b7fe640 1 -- 172.21.15.120:0/483045156 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310037070 con 0x7fd31c158800 2024-09-10T10:29:06.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f5dfe59c640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5de006c030 0x7f5de006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f1acdd9b640 1 -- 172.21.15.120:0/3360430997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac80e6030 msgr2=0x7f1ac80cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f1acdd9b640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac80e6030 0x7f1ac80cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f1acdd9b640 1 -- 172.21.15.120:0/3360430997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ac804aa40 msgr2=0x7f1ac8049940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f1acdd9b640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ac804aa40 0x7f1ac8049940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f1acdd9b640 1 -- 172.21.15.120:0/3360430997 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac80ceb50 con 0x7f1ac80dce90 2024-09-10T10:29:06.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f1aced9d640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac80e6030 0x7f1ac80cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:29:06.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f03fb7fe640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03fc150c10 0x7f03fc1deca0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f03f4035000 tx=0x7f03f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f4296ffd640 1 -- 172.21.15.120:0/269290847 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f429c01a070 con 0x7f42a00d6680 2024-09-10T10:29:06.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f1acdd9b640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ac80dce90 0x7f1ac80cf970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0051790 tx=0x7f1ad0069890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7fc571415640 1 -- 172.21.15.120:0/3364494456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 msgr2=0x7fc564046290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7fc571415640 1 --2- 172.21.15.120:0/3364494456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 0x7fc564046290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc56c02dd50 tx=0x7fc56c0a6ad0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7fc571415640 1 -- 172.21.15.120:0/3364494456 shutdown_connections 2024-09-10T10:29:06.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7fc571415640 1 --2- 172.21.15.120:0/3364494456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 0x7fc564046290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f4ab37fe640 1 -- 172.21.15.120:0/1908891658 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0037070 con 0x7f4acc06d830 2024-09-10T10:29:06.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7fc571415640 1 --2- 172.21.15.120:0/3364494456 >> v2:172.21.15.134:3300/0 conn(0x7fc56404d310 0x7fc564045d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7fc571415640 1 -- 172.21.15.120:0/3364494456 >> 172.21.15.120:0/3364494456 conn(0x7fc56404d830 msgr2=0x7fc56404dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7f4ab37fe640 1 -- 172.21.15.120:0/1908891658 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac0042070 con 0x7f4acc06d830 2024-09-10T10:29:06.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7f4acbfff640 1 -- 172.21.15.120:0/1908891658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc157cc0 con 0x7f4acc06d830 2024-09-10T10:29:06.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.608+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ec0d5e60 con 0x7f86ec007340 2024-09-10T10:29:06.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7f5dfe59c640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5de006c030 0x7f5de006e4f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5df0006b10 tx=0x7f5df0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7fc571415640 1 -- 172.21.15.120:0/3364494456 shutdown_connections 2024-09-10T10:29:06.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7f4296ffd640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f428806bf00 0x7f428806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7f4ab37fe640 1 -- 172.21.15.120:0/1908891658 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0030c00 con 0x7f4acc06d830 2024-09-10T10:29:06.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7f4acbfff640 1 -- 172.21.15.120:0/1908891658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc158280 con 0x7f4acc06d830 2024-09-10T10:29:06.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7f86f17fa640 1 -- 172.21.15.120:0/2328868872 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86f40c7070 con 0x7f86ec007340 2024-09-10T10:29:06.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7fc571415640 1 -- 172.21.15.120:0/3364494456 wait complete. 2024-09-10T10:29:06.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7f4296ffd640 1 -- 172.21.15.120:0/269290847 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f429c08a650 con 0x7f42a00d6680 2024-09-10T10:29:06.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7f42a5d9b640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f428806bf00 0x7f428806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7f42a5d9b640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f428806bf00 0x7f428806e3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42a00d5aa0 tx=0x7f4298009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7fd321bf1640 1 -- 172.21.15.120:0/483045156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 msgr2=0x7fd31c0c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.609+0000 7fd321bf1640 1 --2- 172.21.15.120:0/483045156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 0x7fd31c0c8e70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0cbe40 tx=0x7fd31002a280 comp rx=0 tx=0).stop 2024-09-10T10:29:06.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7fd321bf1640 1 -- 172.21.15.120:0/483045156 shutdown_connections 2024-09-10T10:29:06.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7fd321bf1640 1 --2- 172.21.15.120:0/483045156 >> v2:172.21.15.134:3300/0 conn(0x7fd31c0c93b0 0x7fd31c0c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7fd321bf1640 1 --2- 172.21.15.120:0/483045156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 0x7fd31c0c8e70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7fd321bf1640 1 --2- 172.21.15.120:0/483045156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c152a30 0x7fd31c15c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7fd321bf1640 1 -- 172.21.15.120:0/483045156 >> 172.21.15.120:0/483045156 conn(0x7fd31c067950 msgr2=0x7fd31c067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:06.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7f86f17fa640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f86d806bf30 0x7f86d806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7fd321bf1640 1 -- 172.21.15.120:0/483045156 shutdown_connections 2024-09-10T10:29:06.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7fd321bf1640 1 -- 172.21.15.120:0/483045156 wait complete. 2024-09-10T10:29:06.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7f86f17fa640 1 -- 172.21.15.120:0/2328868872 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f86f4127fe0 con 0x7f86ec007340 2024-09-10T10:29:06.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7f86f3fff640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f86d806bf30 0x7f86d806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7f4acbfff640 1 -- 172.21.15.120:0/1908891658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4acc0b4b80 con 0x7f4acc06d830 2024-09-10T10:29:06.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.610+0000 7f4ab37fe640 1 -- 172.21.15.120:0/1908891658 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ac0047460 con 0x7f4acc06d830 2024-09-10T10:29:06.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.611+0000 7f86f3fff640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f86d806bf30 0x7f86d806e3f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009560 tx=0x7f86e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.611+0000 7f4ab37fe640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ab806be80 0x7f4ab806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.611+0000 7fc571415640 1 Processor -- start 2024-09-10T10:29:06.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.611+0000 7f4aca7fc640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ab806be80 0x7f4ab806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.611+0000 7f4ab37fe640 1 -- 172.21.15.120:0/1908891658 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f4ac00abab0 con 0x7f4acc06d830 2024-09-10T10:29:06.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.611+0000 7fc571415640 1 -- start start 2024-09-10T10:29:06.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.611+0000 7f4aca7fc640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ab806be80 0x7f4ab806e340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4acc1f55d0 tx=0x7f4ac4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fd321bf1640 1 Processor -- start 2024-09-10T10:29:06.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fd321bf1640 1 -- start start 2024-09-10T10:29:06.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc571415640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 0x7fc564173ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc571415640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56404d310 0x7fc5641740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc571415640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5640d5850 0x7fc5640d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc571415640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc564047560 con 0x7fc5640d5850 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc571415640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc564047280 con 0x7fc56404d310 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc571415640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc5640468c0 con 0x7fc564047a80 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc570c14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5640d5850 0x7fc5640d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc56b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56404d310 0x7fc5641740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc570c14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5640d5850 0x7fc5640d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33122/0 (socket says 172.21.15.120:33122) 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc570c14640 1 -- 172.21.15.120:0/1870047094 learned_addr learned my addr 172.21.15.120:0/1870047094 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.612+0000 7fc56bfff640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 0x7fc564173ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fc570c14640 1 -- 172.21.15.120:0/1870047094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 msgr2=0x7fc564173ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fc570c14640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 0x7fc564173ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fc570c14640 1 -- 172.21.15.120:0/1870047094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56404d310 msgr2=0x7fc5641740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fc570c14640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56404d310 0x7fc5641740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fc570c14640 1 -- 172.21.15.120:0/1870047094 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56404add0 con 0x7fc5640d5850 2024-09-10T10:29:06.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fc570c14640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5640d5850 0x7fc5640d01b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc56c06a450 tx=0x7fc56c074480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd321bf1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c0c93b0 0x7fd31c154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd321bf1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd31c152a30 0x7fd31c153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd321bf1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 0x7fd31c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd3213f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 0x7fd31c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd3213f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 0x7fd31c0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33124/0 (socket says 172.21.15.120:33124) 2024-09-10T10:29:06.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd3213f0640 1 -- 172.21.15.120:0/4152550880 learned_addr learned my addr 172.21.15.120:0/4152550880 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:06.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd320bef640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c0c93b0 0x7fd31c154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd321bf1640 1 -- 172.21.15.120:0/4152550880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0c72d0 con 0x7fd31c158800 2024-09-10T10:29:06.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd321bf1640 1 -- 172.21.15.120:0/4152550880 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c159550 con 0x7fd31c152a30 2024-09-10T10:29:06.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd321bf1640 1 -- 172.21.15.120:0/4152550880 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c158c40 con 0x7fd31c0c93b0 2024-09-10T10:29:06.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7fd31bfff640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd31c152a30 0x7fd31c153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7f1abeffd640 1 -- 172.21.15.120:0/3360430997 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0091070 con 0x7f1ac80dce90 2024-09-10T10:29:06.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7f5deeffd640 1 -- 172.21.15.120:0/2499303219 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5e000f4e20 con 0x7f5df8048130 2024-09-10T10:29:06.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7f1abeffd640 1 -- 172.21.15.120:0/3360430997 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad0071840 con 0x7f1ac80dce90 2024-09-10T10:29:06.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7f1abeffd640 1 -- 172.21.15.120:0/3360430997 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0071d30 con 0x7f1ac80dce90 2024-09-10T10:29:06.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.613+0000 7f1acf59e640 1 -- 172.21.15.120:0/3360430997 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac80e4fe0 con 0x7f1ac80dce90 2024-09-10T10:29:06.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7f1acf59e640 1 -- 172.21.15.120:0/3360430997 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac80cedc0 con 0x7f1ac80dce90 2024-09-10T10:29:06.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd320bef640 1 -- 172.21.15.120:0/4152550880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd31c152a30 msgr2=0x7fd31c153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd320bef640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd31c152a30 0x7fd31c153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd320bef640 1 -- 172.21.15.120:0/4152550880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 msgr2=0x7fd31c0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd320bef640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 0x7fd31c0ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd320bef640 1 -- 172.21.15.120:0/4152550880 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c15c8e0 con 0x7fd31c0c93b0 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd3213f0640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 0x7fd31c0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd320bef640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c0c93b0 0x7fd31c154bb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00e8a0 tx=0x7fd30c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd3197fa640 1 -- 172.21.15.120:0/4152550880 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c017070 con 0x7fd31c0c93b0 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd3197fa640 1 -- 172.21.15.120:0/4152550880 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c00c940 con 0x7fd31c0c93b0 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd3197fa640 1 -- 172.21.15.120:0/4152550880 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c009e60 con 0x7fd31c0c93b0 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.614+0000 7fd321bf1640 1 -- 172.21.15.120:0/4152550880 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c1f5e80 con 0x7fd31c0c93b0 2024-09-10T10:29:06.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.615+0000 7fd321bf1640 1 -- 172.21.15.120:0/4152550880 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c0c6670 con 0x7fd31c0c93b0 2024-09-10T10:29:06.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.615+0000 7f1acf59e640 1 -- 172.21.15.120:0/3360430997 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac804a7d0 con 0x7f1ac80dce90 2024-09-10T10:29:06.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7f1abeffd640 1 -- 172.21.15.120:0/3360430997 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ad0094070 con 0x7f1ac80dce90 2024-09-10T10:29:06.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7fd302ffd640 1 -- 172.21.15.120:0/4152550880 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f4005290 con 0x7fd31c0c93b0 2024-09-10T10:29:06.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7fd3197fa640 1 -- 172.21.15.120:0/4152550880 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd30c01a070 con 0x7fd31c0c93b0 2024-09-10T10:29:06.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7f1abeffd640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ab006bf00 0x7f1ab006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7fc568ff9640 1 -- 172.21.15.120:0/1870047094 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0bd650 con 0x7fc5640d5850 2024-09-10T10:29:06.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.617+0000 7fc571415640 1 -- 172.21.15.120:0/1870047094 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56404b0b0 con 0x7fc5640d5850 2024-09-10T10:29:06.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.617+0000 7fc571415640 1 -- 172.21.15.120:0/1870047094 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5640d2600 con 0x7fc5640d5850 2024-09-10T10:29:06.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.617+0000 7f4296ffd640 1 -- 172.21.15.120:0/269290847 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f429c0584a0 con 0x7f42a00d6680 2024-09-10T10:29:06.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7f03f3fff640 1 -- 172.21.15.120:0/1465606301 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4046020 con 0x7f03fc150c10 2024-09-10T10:29:06.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7f04013cf640 1 -- 172.21.15.120:0/1465606301 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc14f350 con 0x7f03fc150c10 2024-09-10T10:29:06.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7f04013cf640 1 -- 172.21.15.120:0/1465606301 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc14df60 con 0x7f03fc150c10 2024-09-10T10:29:06.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.617+0000 7fd3197fa640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd30806be60 0x7fd30806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7f1abeffd640 1 -- 172.21.15.120:0/3360430997 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f1ad01072d0 con 0x7f1ac80dce90 2024-09-10T10:29:06.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7f1ace59c640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ab006bf00 0x7f1ab006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.616+0000 7f4ab37fe640 1 -- 172.21.15.120:0/1908891658 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4ac0079fb0 con 0x7f4acc06d830 2024-09-10T10:29:06.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.617+0000 7fd3197fa640 1 -- 172.21.15.120:0/4152550880 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fd30c08b390 con 0x7fd31c0c93b0 2024-09-10T10:29:06.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.617+0000 7fd31bfff640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd30806be60 0x7fd30806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.618+0000 7fd31bfff640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd30806be60 0x7fd30806e320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd310034000 tx=0x7fd31002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.618+0000 7f1ace59c640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ab006bf00 0x7f1ab006e3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ac402a2b0 tx=0x7f1ac402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.618+0000 7f04013cf640 1 -- 172.21.15.120:0/1465606301 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03fc0c6cb0 con 0x7f03fc150c10 2024-09-10T10:29:06.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.618+0000 7fc571415640 1 -- 172.21.15.120:0/1870047094 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc56404c5f0 con 0x7fc5640d5850 2024-09-10T10:29:06.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.622+0000 7f03f3fff640 1 -- 172.21.15.120:0/1465606301 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4009a00 con 0x7f03fc150c10 2024-09-10T10:29:06.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.622+0000 7f1abeffd640 1 -- 172.21.15.120:0/3360430997 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1ac804a7d0 con 0x7f1ac80dce90 2024-09-10T10:29:06.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.622+0000 7f03f3fff640 1 -- 172.21.15.120:0/1465606301 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f402f720 con 0x7f03fc150c10 2024-09-10T10:29:06.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.622+0000 7fc568ff9640 1 -- 172.21.15.120:0/1870047094 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c0a8950 con 0x7fc5640d5850 2024-09-10T10:29:06.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.622+0000 7fc568ff9640 1 -- 172.21.15.120:0/1870047094 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0c3070 con 0x7fc5640d5850 2024-09-10T10:29:06.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.623+0000 7f86f17fa640 1 -- 172.21.15.120:0/2328868872 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86f40ba070 con 0x7f86ec007340 2024-09-10T10:29:06.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.624+0000 7f03f3fff640 1 -- 172.21.15.120:0/1465606301 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03f4049020 con 0x7f03fc150c10 2024-09-10T10:29:06.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.625+0000 7f03f3fff640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03dc06be30 0x7f03dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.625+0000 7f03f3fff640 1 -- 172.21.15.120:0/1465606301 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f03f40b0eb0 con 0x7f03fc150c10 2024-09-10T10:29:06.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.625+0000 7fd3197fa640 1 -- 172.21.15.120:0/4152550880 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd30c059260 con 0x7fd31c0c93b0 2024-09-10T10:29:06.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.625+0000 7f03faffd640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03dc06be30 0x7f03dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.626+0000 7fc568ff9640 1 -- 172.21.15.120:0/1870047094 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc56c063230 con 0x7fc5640d5850 2024-09-10T10:29:06.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.627+0000 7fc568ff9640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc55006be30 0x7fc55006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:06.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.627+0000 7fc568ff9640 1 -- 172.21.15.120:0/1870047094 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fc56c128800 con 0x7fc5640d5850 2024-09-10T10:29:06.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.627+0000 7fc56bfff640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc55006be30 0x7fc55006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:06.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.626+0000 7f03faffd640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03dc06be30 0x7f03dc06e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0094a0 tx=0x7f03ec009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.628+0000 7fc56bfff640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc55006be30 0x7fc55006e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc560005880 tx=0x7fc560009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.632+0000 7f03f3fff640 1 -- 172.21.15.120:0/1465606301 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f03f4050030 con 0x7f03fc150c10 2024-09-10T10:29:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:06.641+0000 7fc568ff9640 1 -- 172.21.15.120:0/1870047094 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc56c0ae030 con 0x7fc5640d5850 2024-09-10T10:29:07.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.009+0000 7f4294ff9640 1 -- 172.21.15.120:0/269290847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f42a004c1d0 con 0x7f42a00d6680 2024-09-10T10:29:07.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.010+0000 7f4296ffd640 1 -- 172.21.15.120:0/269290847 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f429c05c7a0 con 0x7f42a00d6680 2024-09-10T10:29:07.011 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:07.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.013+0000 7f42acf49640 1 -- 172.21.15.120:0/269290847 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f428806bf00 msgr2=0x7f428806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.014+0000 7f42acf49640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f428806bf00 0x7f428806e3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42a00d5aa0 tx=0x7f4298009060 comp rx=0 tx=0).stop 2024-09-10T10:29:07.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.014+0000 7f42acf49640 1 -- 172.21.15.120:0/269290847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42a00d6680 msgr2=0x7f42a00d6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.014+0000 7f42acf49640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42a00d6680 0x7f42a00d6ff0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c480 tx=0x7f429c00c940 comp rx=0 tx=0).stop 2024-09-10T10:29:07.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.014+0000 7f42acf49640 1 -- 172.21.15.120:0/269290847 shutdown_connections 2024-09-10T10:29:07.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.014+0000 7f42acf49640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f428806bf00 0x7f428806e3c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.014+0000 7f42acf49640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a00dd060 0x7f42a0050f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.015+0000 7f42acf49640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42a00d6a60 0x7f42a0174010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.015+0000 7f42acf49640 1 --2- 172.21.15.120:0/269290847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42a00d6680 0x7f42a00d6ff0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.015+0000 7f42acf49640 1 -- 172.21.15.120:0/269290847 >> 172.21.15.120:0/269290847 conn(0x7f42a004e8f0 msgr2=0x7f42a00d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:07.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.015+0000 7f42acf49640 1 -- 172.21.15.120:0/269290847 shutdown_connections 2024-09-10T10:29:07.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.015+0000 7f42acf49640 1 -- 172.21.15.120:0/269290847 wait complete. 2024-09-10T10:29:07.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.016+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5de4005580 con 0x7f5df8048130 2024-09-10T10:29:07.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.017+0000 7f5deeffd640 1 -- 172.21.15.120:0/2499303219 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5e000f4e20 con 0x7f5df8048130 2024-09-10T10:29:07.018 INFO:teuthology.orchestra.run.smithi120.stdout:55834574881 2024-09-10T10:29:07.022 INFO:teuthology.orchestra.run.smithi120.stdout:55834574881 2024-09-10T10:29:07.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.018+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f86ec04e4b0 con 0x7f86ec007340 2024-09-10T10:29:07.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.019+0000 7f86f17fa640 1 -- 172.21.15.120:0/2328868872 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f86f40f6500 con 0x7f86ec007340 2024-09-10T10:29:07.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.020+0000 7f4acbfff640 1 -- 172.21.15.120:0/1908891658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4acc0c9a70 con 0x7f4acc06d830 2024-09-10T10:29:07.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5de006c030 msgr2=0x7f5de006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5de006c030 0x7f5de006e4f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5df0006b10 tx=0x7f5df0008040 comp rx=0 tx=0).stop 2024-09-10T10:29:07.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5df8048130 msgr2=0x7f5df8007670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5df8048130 0x7f5df8007670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e00074ad0 tx=0x7f5e00074480 comp rx=0 tx=0).stop 2024-09-10T10:29:07.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 shutdown_connections 2024-09-10T10:29:07.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5de006c030 0x7f5de006e4f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5df8048130 0x7f5df8007670 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df8165530 0x7f5df80cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 --2- 172.21.15.120:0/2499303219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5de80051c0 0x7f5df8165ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 >> 172.21.15.120:0/2499303219 conn(0x7f5df804d830 msgr2=0x7f5df8045d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:07.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 shutdown_connections 2024-09-10T10:29:07.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f4ab37fe640 1 -- 172.21.15.120:0/1908891658 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4ac00798b0 con 0x7f4acc06d830 2024-09-10T10:29:07.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.021+0000 7f5dff59e640 1 -- 172.21.15.120:0/2499303219 wait complete. 2024-09-10T10:29:07.025 INFO:teuthology.orchestra.run.smithi120.stdout:55834574881 2024-09-10T10:29:07.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.022+0000 7f1acf59e640 1 -- 172.21.15.120:0/3360430997 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1ac8005a00 con 0x7f1ac80dce90 2024-09-10T10:29:07.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.022+0000 7f1abeffd640 1 -- 172.21.15.120:0/3360430997 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ac80e4fe0 con 0x7f1ac80dce90 2024-09-10T10:29:07.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.022+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f86d806bf30 msgr2=0x7f86d806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.022+0000 7f86faeac640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f86d806bf30 0x7f86d806e3f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009560 tx=0x7f86e8008040 comp rx=0 tx=0).stop 2024-09-10T10:29:07.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.022+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec007340 msgr2=0x7f86ec0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.022+0000 7f86faeac640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec007340 0x7f86ec0ce820 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f86f402dd50 tx=0x7f86f4074750 comp rx=0 tx=0).stop 2024-09-10T10:29:07.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.023+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 shutdown_connections 2024-09-10T10:29:07.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.023+0000 7f86faeac640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f86d806bf30 0x7f86d806e3f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.023+0000 7f86faeac640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86ec0d8bb0 0x7f86ec038230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.023+0000 7f86faeac640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ec04f870 0x7f86ec037cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.023+0000 7f86faeac640 1 --2- 172.21.15.120:0/2328868872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86ec007340 0x7f86ec0ce820 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.023+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 >> 172.21.15.120:0/2328868872 conn(0x7f86ec054470 msgr2=0x7f86ec004a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:07.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.023+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 shutdown_connections 2024-09-10T10:29:07.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.023+0000 7f86faeac640 1 -- 172.21.15.120:0/2328868872 wait complete. 2024-09-10T10:29:07.027 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:07.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.025+0000 7f4acbfff640 1 -- 172.21.15.120:0/1908891658 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ab806be80 msgr2=0x7f4ab806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.025+0000 7f4acbfff640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ab806be80 0x7f4ab806e340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4acc1f55d0 tx=0x7f4ac4009040 comp rx=0 tx=0).stop 2024-09-10T10:29:07.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.025+0000 7f4acbfff640 1 -- 172.21.15.120:0/1908891658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4acc06d830 msgr2=0x7f4acc158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.025+0000 7f4acbfff640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4acc06d830 0x7f4acc158d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00098c0 tx=0x7f4ac00046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.026+0000 7f4acbfff640 1 -- 172.21.15.120:0/1908891658 shutdown_connections 2024-09-10T10:29:07.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.026+0000 7f4acbfff640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f4ab806be80 0x7f4ab806e340 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.026+0000 7f4acbfff640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4acc0c8920 0x7f4acc0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.026+0000 7f4acbfff640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4acc0b3bf0 0x7f4acc0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.026+0000 7f4acbfff640 1 --2- 172.21.15.120:0/1908891658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4acc06d830 0x7f4acc158d60 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00098c0 tx=0x7f4ac00046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.026+0000 7f4acbfff640 1 -- 172.21.15.120:0/1908891658 >> 172.21.15.120:0/1908891658 conn(0x7f4acc0d0250 msgr2=0x7f4acc15ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:07.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.026+0000 7fd302ffd640 1 -- 172.21.15.120:0/4152550880 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd2f4005580 con 0x7fd31c0c93b0 2024-09-10T10:29:07.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.026+0000 7f4acbfff640 1 -- 172.21.15.120:0/1908891658 shutdown_connections 2024-09-10T10:29:07.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.026+0000 7f4acbfff640 1 -- 172.21.15.120:0/1908891658 wait complete. 2024-09-10T10:29:07.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.026+0000 7fd3197fa640 1 -- 172.21.15.120:0/4152550880 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd30c05d560 con 0x7fd31c0c93b0 2024-09-10T10:29:07.030 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:07.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.028+0000 7f1acf59e640 1 -- 172.21.15.120:0/3360430997 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ab006bf00 msgr2=0x7f1ab006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.028+0000 7f1acf59e640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ab006bf00 0x7f1ab006e3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ac402a2b0 tx=0x7f1ac402e000 comp rx=0 tx=0).stop 2024-09-10T10:29:07.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.028+0000 7f1acf59e640 1 -- 172.21.15.120:0/3360430997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ac80dce90 msgr2=0x7f1ac80cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.028+0000 7f1acf59e640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ac80dce90 0x7f1ac80cf970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0051790 tx=0x7f1ad0069890 comp rx=0 tx=0).stop 2024-09-10T10:29:07.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.029+0000 7f1acf59e640 1 -- 172.21.15.120:0/3360430997 shutdown_connections 2024-09-10T10:29:07.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.029+0000 7f1acf59e640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f1ab006bf00 0x7f1ab006e3c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.029+0000 7f1acf59e640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac80e6030 0x7f1ac80cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.029+0000 7f1acf59e640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ac80dce90 0x7f1ac80cf970 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.029+0000 7f1acf59e640 1 --2- 172.21.15.120:0/3360430997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ac804aa40 0x7f1ac8049940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.029+0000 7f1acf59e640 1 -- 172.21.15.120:0/3360430997 >> 172.21.15.120:0/3360430997 conn(0x7f1ac804fc50 msgr2=0x7f1ac80d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:07.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.029+0000 7f1acf59e640 1 -- 172.21.15.120:0/3360430997 shutdown_connections 2024-09-10T10:29:07.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.029+0000 7f1acf59e640 1 -- 172.21.15.120:0/3360430997 wait complete. 2024-09-10T10:29:07.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.030+0000 7fd302ffd640 1 -- 172.21.15.120:0/4152550880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd30806be60 msgr2=0x7fd30806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.030+0000 7fd302ffd640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd30806be60 0x7fd30806e320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd310034000 tx=0x7fd31002e000 comp rx=0 tx=0).stop 2024-09-10T10:29:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.030+0000 7fd302ffd640 1 -- 172.21.15.120:0/4152550880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c0c93b0 msgr2=0x7fd31c154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.030+0000 7fd302ffd640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c0c93b0 0x7fd31c154bb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00e8a0 tx=0x7fd30c00ed60 comp rx=0 tx=0).stop 2024-09-10T10:29:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.030+0000 7fd302ffd640 1 -- 172.21.15.120:0/4152550880 shutdown_connections 2024-09-10T10:29:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.030+0000 7fd302ffd640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fd30806be60 0x7fd30806e320 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.030+0000 7fd302ffd640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd31c158800 0x7fd31c0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.030+0000 7fd302ffd640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd31c152a30 0x7fd31c153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.030+0000 7fd302ffd640 1 --2- 172.21.15.120:0/4152550880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd31c0c93b0 0x7fd31c154bb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.030+0000 7fd302ffd640 1 -- 172.21.15.120:0/4152550880 >> 172.21.15.120:0/4152550880 conn(0x7fd31c067950 msgr2=0x7fd31c14d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.031+0000 7fd302ffd640 1 -- 172.21.15.120:0/4152550880 shutdown_connections 2024-09-10T10:29:07.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.031+0000 7fd302ffd640 1 -- 172.21.15.120:0/4152550880 wait complete. 2024-09-10T10:29:07.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.032+0000 7f04013cf640 1 -- 172.21.15.120:0/1465606301 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f03fc0c9b60 con 0x7f03fc150c10 2024-09-10T10:29:07.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.033+0000 7f03f3fff640 1 -- 172.21.15.120:0/1465606301 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f03f407f230 con 0x7f03fc150c10 2024-09-10T10:29:07.034 INFO:teuthology.orchestra.run.smithi120.stdout:55834574881 2024-09-10T10:29:07.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.037+0000 7f04013cf640 1 -- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03dc06be30 msgr2=0x7f03dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.037+0000 7f04013cf640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03dc06be30 0x7f03dc06e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0094a0 tx=0x7f03ec009240 comp rx=0 tx=0).stop 2024-09-10T10:29:07.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.037+0000 7f04013cf640 1 -- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03fc150c10 msgr2=0x7f03fc1deca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.037+0000 7f04013cf640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03fc150c10 0x7f03fc1deca0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f03f4035000 tx=0x7f03f4004a40 comp rx=0 tx=0).stop 2024-09-10T10:29:07.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.0 2024-09-10T10:29:07.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.038+0000 7f04013cf640 1 -- 172.21.15.120:0/1465606301 shutdown_connections 2024-09-10T10:29:07.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.038+0000 7f04013cf640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f03dc06be30 0x7f03dc06e2f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.038+0000 7f04013cf640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f03fc150c10 0x7f03fc1deca0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.038+0000 7f04013cf640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc0c9650 0x7f03fc14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.038+0000 7f04013cf640 1 --2- 172.21.15.120:0/1465606301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc072410 0x7f03fc1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.038+0000 7f04013cf640 1 -- 172.21.15.120:0/1465606301 >> 172.21.15.120:0/1465606301 conn(0x7f03fc06f3c0 msgr2=0x7f03fc0c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:07.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.038+0000 7f04013cf640 1 -- 172.21.15.120:0/1465606301 shutdown_connections 2024-09-10T10:29:07.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.038+0000 7f04013cf640 1 -- 172.21.15.120:0/1465606301 wait complete. 2024-09-10T10:29:07.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.042+0000 7fc571415640 1 -- 172.21.15.120:0/1870047094 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc5640cfe60 con 0x7fc5640d5850 2024-09-10T10:29:07.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.1 2024-09-10T10:29:07.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.2 2024-09-10T10:29:07.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.043+0000 7fc568ff9640 1 -- 172.21.15.120:0/1870047094 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc56c0f65b0 con 0x7fc5640d5850 2024-09-10T10:29:07.046 INFO:teuthology.orchestra.run.smithi120.stdout:55834574881 2024-09-10T10:29:07.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 -- 172.21.15.120:0/1870047094 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc55006be30 msgr2=0x7fc55006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc55006be30 0x7fc55006e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc560005880 tx=0x7fc560009040 comp rx=0 tx=0).stop 2024-09-10T10:29:07.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 -- 172.21.15.120:0/1870047094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5640d5850 msgr2=0x7fc5640d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:07.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5640d5850 0x7fc5640d01b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc56c06a450 tx=0x7fc56c074480 comp rx=0 tx=0).stop 2024-09-10T10:29:07.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 -- 172.21.15.120:0/1870047094 shutdown_connections 2024-09-10T10:29:07.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc55006be30 0x7fc55006e2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5640d5850 0x7fc5640d01b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56404d310 0x7fc5641740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 --2- 172.21.15.120:0/1870047094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc564047a80 0x7fc564173ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:07.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 -- 172.21.15.120:0/1870047094 >> 172.21.15.120:0/1870047094 conn(0x7fc56404d830 msgr2=0x7fc5640050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:07.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 -- 172.21.15.120:0/1870047094 shutdown_connections 2024-09-10T10:29:07.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:07.047+0000 7fc571415640 1 -- 172.21.15.120:0/1870047094 wait complete. 2024-09-10T10:29:07.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.6 2024-09-10T10:29:07.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.4 2024-09-10T10:29:07.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.7 2024-09-10T10:29:07.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.5 2024-09-10T10:29:07.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.3 2024-09-10T10:29:08.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T10:29:08.045 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T10:29:08.046 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T10:29:08.051 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T10:29:08.054 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T10:29:08.054 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T10:29:08.059 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T10:29:08.062 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T10:29:08.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.198+0000 7f33af69b640 1 Processor -- start 2024-09-10T10:29:08.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.199+0000 7f33af69b640 1 -- start start 2024-09-10T10:29:08.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.200+0000 7f33af69b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f33a8158f40 0x7f33a815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.200+0000 7f33af69b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33a8159320 0x7f33a80c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.200+0000 7f33af69b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f33a8162c00 0x7f33a80c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.200+0000 7f33af69b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f33a8162770 con 0x7f33a8162c00 2024-09-10T10:29:08.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.200+0000 7f33af69b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f33a815f430 con 0x7f33a8158f40 2024-09-10T10:29:08.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.200+0000 7f33af69b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f33a815f710 con 0x7f33a8159320 2024-09-10T10:29:08.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.200+0000 7f33adc38640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f33a8162c00 0x7f33a80c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.201+0000 7f33ad437640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f33a8158f40 0x7f33a815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.201+0000 7f33ad437640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f33a8158f40 0x7f33a815ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:54120/0 (socket says 172.21.15.120:54120) 2024-09-10T10:29:08.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.201+0000 7f33ad437640 1 -- 172.21.15.120:0/1884065844 learned_addr learned my addr 172.21.15.120:0/1884065844 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.201+0000 7f33acc36640 1 --2- 172.21.15.120:0/1884065844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33a8159320 0x7f33a80c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.201+0000 7f33ad437640 1 -- 172.21.15.120:0/1884065844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33a8159320 msgr2=0x7f33a80c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.201+0000 7f33ad437640 1 --2- 172.21.15.120:0/1884065844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33a8159320 0x7f33a80c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.201+0000 7f33ad437640 1 -- 172.21.15.120:0/1884065844 >> v2:172.21.15.120:3300/0 conn(0x7f33a8162c00 msgr2=0x7f33a80c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.201+0000 7f33ad437640 1 --2- 172.21.15.120:0/1884065844 >> v2:172.21.15.120:3300/0 conn(0x7f33a8162c00 0x7f33a80c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.201+0000 7f33ad437640 1 -- 172.21.15.120:0/1884065844 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a815f9f0 con 0x7f33a8158f40 2024-09-10T10:29:08.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.203+0000 7f33ad437640 1 --2- 172.21.15.120:0/1884065844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a8158f40 0x7f33a815ee10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f33a80cd270 tx=0x7f33a402a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecb579ba49c322b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.203+0000 7f339ffff640 1 -- 172.21.15.120:0/1884065844 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4037070 con 0x7f33a8158f40 2024-09-10T10:29:08.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.204+0000 7f339ffff640 1 -- 172.21.15.120:0/1884065844 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a4032440 con 0x7f33a8158f40 2024-09-10T10:29:08.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.204+0000 7f339ffff640 1 -- 172.21.15.120:0/1884065844 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a40327f0 con 0x7f33a8158f40 2024-09-10T10:29:08.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.205+0000 7f33af69b640 1 -- 172.21.15.120:0/1884065844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a8158f40 msgr2=0x7f33a815ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.205+0000 7f33af69b640 1 --2- 172.21.15.120:0/1884065844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a8158f40 0x7f33a815ee10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f33a80cd270 tx=0x7f33a402a2a0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.205+0000 7f33af69b640 1 -- 172.21.15.120:0/1884065844 shutdown_connections 2024-09-10T10:29:08.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.205+0000 7f33af69b640 1 --2- 172.21.15.120:0/1884065844 >> v2:172.21.15.120:3300/0 conn(0x7f33a8162c00 0x7f33a80c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.205+0000 7f33af69b640 1 --2- 172.21.15.120:0/1884065844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33a8159320 0x7f33a80c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.205+0000 7f33af69b640 1 --2- 172.21.15.120:0/1884065844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a8158f40 0x7f33a815ee10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.205+0000 7f33af69b640 1 -- 172.21.15.120:0/1884065844 >> 172.21.15.120:0/1884065844 conn(0x7f33a806f3c0 msgr2=0x7f33a806f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.206+0000 7f33af69b640 1 -- 172.21.15.120:0/1884065844 shutdown_connections 2024-09-10T10:29:08.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.206+0000 7f33af69b640 1 -- 172.21.15.120:0/1884065844 wait complete. 2024-09-10T10:29:08.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.208+0000 7f33af69b640 1 Processor -- start 2024-09-10T10:29:08.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.208+0000 7f33af69b640 1 -- start start 2024-09-10T10:29:08.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.209+0000 7f33af69b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33a8158f40 0x7f33a815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.209+0000 7f33af69b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33a8159320 0x7f33a80c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.210+0000 7f33acc36640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33a8159320 0x7f33a80c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.210+0000 7f33acc36640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33a8159320 0x7f33a80c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42254/0 (socket says 172.21.15.120:42254) 2024-09-10T10:29:08.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.210+0000 7f33ad437640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33a8158f40 0x7f33a815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.210+0000 7f33af69b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a8162c00 0x7f33a80c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.210+0000 7f33af69b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8072070 con 0x7f33a8159320 2024-09-10T10:29:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.210+0000 7f33af69b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8162400 con 0x7f33a8162c00 2024-09-10T10:29:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.210+0000 7f33af69b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f33a8071930 con 0x7f33a8158f40 2024-09-10T10:29:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.210+0000 7f33acc36640 1 -- 172.21.15.120:0/3347165246 learned_addr learned my addr 172.21.15.120:0/3347165246 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.211+0000 7f33acc36640 1 -- 172.21.15.120:0/3347165246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33a8158f40 msgr2=0x7f33a815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.211+0000 7f33acc36640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33a8158f40 0x7f33a815cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.211+0000 7f33acc36640 1 -- 172.21.15.120:0/3347165246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a8162c00 msgr2=0x7f33a80c5690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:29:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.211+0000 7f33acc36640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a8162c00 0x7f33a80c5690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.211+0000 7f33acc36640 1 -- 172.21.15.120:0/3347165246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a815d480 con 0x7f33a8159320 2024-09-10T10:29:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.212+0000 7f33acc36640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33a8159320 0x7f33a80c5150 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f33a000e190 tx=0x7f33a000e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.213+0000 7f339dffb640 1 -- 172.21.15.120:0/3347165246 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a001a070 con 0x7f33a8159320 2024-09-10T10:29:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.213+0000 7f33af69b640 1 -- 172.21.15.120:0/3347165246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a8143f60 con 0x7f33a8159320 2024-09-10T10:29:08.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.213+0000 7f33af69b640 1 -- 172.21.15.120:0/3347165246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a815ae10 con 0x7f33a8159320 2024-09-10T10:29:08.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.216+0000 7f339dffb640 1 -- 172.21.15.120:0/3347165246 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a0003b90 con 0x7f33a8159320 2024-09-10T10:29:08.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.216+0000 7f339dffb640 1 -- 172.21.15.120:0/3347165246 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004030 con 0x7f33a8159320 2024-09-10T10:29:08.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.217+0000 7f339dffb640 1 -- 172.21.15.120:0/3347165246 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33a001d070 con 0x7f33a8159320 2024-09-10T10:29:08.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.217+0000 7f339dffb640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f338c06bf30 0x7f338c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.218+0000 7f339dffb640 1 -- 172.21.15.120:0/3347165246 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f33a008b6f0 con 0x7f33a8159320 2024-09-10T10:29:08.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.218+0000 7f33ad437640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f338c06bf30 0x7f338c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.219+0000 7f33af69b640 1 -- 172.21.15.120:0/3347165246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3388005290 con 0x7f33a8159320 2024-09-10T10:29:08.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.223+0000 7f33ad437640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f338c06bf30 0x7f338c06e3f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f33a4034000 tx=0x7f33a402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.228+0000 7f339dffb640 1 -- 172.21.15.120:0/3347165246 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f33a0059c10 con 0x7f33a8159320 2024-09-10T10:29:08.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.255+0000 7f08ab2c9640 1 Processor -- start 2024-09-10T10:29:08.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.255+0000 7f08ab2c9640 1 -- start start 2024-09-10T10:29:08.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f1721504640 1 Processor -- start 2024-09-10T10:29:08.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f08ab2c9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f089c0507a0 0x7f089c0da570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f08ab2c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f089c04aea0 0x7f089c04b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f08ab2c9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f089c006000 con 0x7f089c005a30 2024-09-10T10:29:08.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f08ab2c9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f089c032c40 con 0x7f089c0507a0 2024-09-10T10:29:08.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f08ab2c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f089c0062e0 con 0x7f089c04aea0 2024-09-10T10:29:08.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f1721504640 1 -- start start 2024-09-10T10:29:08.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f08a9065640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f089c0507a0 0x7f089c0da570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f08a9065640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f089c0507a0 0x7f089c0da570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42256/0 (socket says 172.21.15.120:42256) 2024-09-10T10:29:08.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f08a9065640 1 -- 172.21.15.120:0/1821278453 learned_addr learned my addr 172.21.15.120:0/1821278453 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.257+0000 7f08a9866640 1 --2- 172.21.15.120:0/1821278453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f089c04aea0 0x7f089c04b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7ff1cb801640 1 Processor -- start 2024-09-10T10:29:08.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7ff1cb801640 1 -- start start 2024-09-10T10:29:08.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7f08a9065640 1 -- 172.21.15.120:0/1821278453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f089c04aea0 msgr2=0x7f089c04b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7f08a9065640 1 --2- 172.21.15.120:0/1821278453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f089c04aea0 0x7f089c04b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7f08a9065640 1 -- 172.21.15.120:0/1821278453 >> v1:172.21.15.120:6789/0 conn(0x7f089c005a30 legacy=0x7f089c0d43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7f08a9065640 1 -- 172.21.15.120:0/1821278453 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c0065c0 con 0x7f089c0507a0 2024-09-10T10:29:08.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7f1721504640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f17140d62e0 0x7f17140dc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7f1721504640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17140d66c0 0x7f171404bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7f1721504640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17140dcc30 con 0x7f1714044b30 2024-09-10T10:29:08.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7f1721504640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f171404b200 con 0x7f17140d62e0 2024-09-10T10:29:08.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.258+0000 7f1721504640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17140dcf10 con 0x7f17140d66c0 2024-09-10T10:29:08.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7f08a9065640 1 --2- 172.21.15.120:0/1821278453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c0507a0 0x7f089c0da570 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f08940098d0 tx=0x7f089402c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5145ed7646bba1b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7f171b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17140d66c0 0x7f171404bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7ff1cb801640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff1c40bf8b0 0x7ff1c40c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7ff1cb801640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c40c8640 0x7ff1c4073990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7ff1cb801640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c406d680 con 0x7ff1c4155c50 2024-09-10T10:29:08.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7ff1cb801640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c40c78a0 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7ff1cb801640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c406dce0 con 0x7ff1c40c8640 2024-09-10T10:29:08.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7f089bfff640 1 -- 172.21.15.120:0/1821278453 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894038070 con 0x7f089c0507a0 2024-09-10T10:29:08.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7f0fd8fb7640 1 Processor -- start 2024-09-10T10:29:08.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7f171b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17140d66c0 0x7f171404bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:40558/0 (socket says 172.21.15.120:40558) 2024-09-10T10:29:08.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7f171b7fe640 1 -- 172.21.15.120:0/2195986702 learned_addr learned my addr 172.21.15.120:0/2195986702 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7f4a0b020640 1 Processor -- start 2024-09-10T10:29:08.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f089bfff640 1 -- 172.21.15.120:0/1821278453 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0894031710 con 0x7f089c0507a0 2024-09-10T10:29:08.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f089bfff640 1 -- 172.21.15.120:0/1821278453 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894031ae0 con 0x7f089c0507a0 2024-09-10T10:29:08.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f4a0b020640 1 -- start start 2024-09-10T10:29:08.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f171bfff640 1 --2- 172.21.15.120:0/2195986702 >> v2:172.21.15.134:3300/0 conn(0x7f17140d62e0 0x7f17140dc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f171b7fe640 1 -- 172.21.15.120:0/2195986702 >> v2:172.21.15.134:3300/0 conn(0x7f17140d62e0 msgr2=0x7f17140dc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f171b7fe640 1 --2- 172.21.15.120:0/2195986702 >> v2:172.21.15.134:3300/0 conn(0x7f17140d62e0 0x7f17140dc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f171b7fe640 1 -- 172.21.15.120:0/2195986702 >> v1:172.21.15.134:6789/0 conn(0x7f1714044b30 legacy=0x7f1714034ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f7a9657f640 1 Processor -- start 2024-09-10T10:29:08.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f7a9657f640 1 -- start start 2024-09-10T10:29:08.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7ff1c959d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff1c4155c50 0x7ff1c4159000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41526/0 (socket says 172.21.15.120:41526) 2024-09-10T10:29:08.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7ff1c959d640 1 -- 172.21.15.120:0/478245029 learned_addr learned my addr 172.21.15.120:0/478245029 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:08.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.259+0000 7ff1c9d9e640 1 --2- 172.21.15.120:0/478245029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c40c8640 0x7ff1c4073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7ff1c9d9e640 1 -- 172.21.15.120:0/478245029 >> v2:172.21.15.120:3300/0 conn(0x7ff1c40bf8b0 msgr2=0x7ff1c40c8100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:29:08.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7ff1bbfff640 1 -- 172.21.15.120:0/478245029 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1634190636 0 0) 0x7ff1c406d680 con 0x7ff1c4155c50 2024-09-10T10:29:08.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f171b7fe640 1 -- 172.21.15.120:0/2195986702 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17140dd1f0 con 0x7f17140d66c0 2024-09-10T10:29:08.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7f0fd8fb7640 1 -- start start 2024-09-10T10:29:08.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f0fd8fb7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0fd40cc0c0 0x7f0fd415aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f7a9657f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a900d0bf0 0x7f7a900c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f4a0b020640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4a040c8960 0x7f4a040c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f171b7fe640 1 --2- 172.21.15.120:0/2195986702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17140d66c0 0x7f171404bc50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1708009870 tx=0x7f170802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32bdeb915a7243e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f7a9657f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7a900b3790 0x7f7a900ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f4a0a01e640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f4a04150960 0x7f4a0406f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:50092/0 (socket says 172.21.15.120:50092) 2024-09-10T10:29:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0b020640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0406d210 0x7f4a040c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0a01e640 1 -- 172.21.15.120:0/607863027 learned_addr learned my addr 172.21.15.120:0/607863027 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0b020640 1 -- 172.21.15.120:0/607863027 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a04159c40 con 0x7f4a04150960 2024-09-10T10:29:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0b020640 1 -- 172.21.15.120:0/607863027 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4a04159f00 con 0x7f4a040c8960 2024-09-10T10:29:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0b020640 1 -- 172.21.15.120:0/607863027 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0415a1e0 con 0x7f4a0406d210 2024-09-10T10:29:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0981d640 1 --2- 172.21.15.120:0/607863027 >> v2:172.21.15.120:3300/0 conn(0x7f4a040c8960 0x7f4a040c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0a81f640 1 --2- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0406d210 0x7f4a040c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 624751979 0 0) 0x7f4a04159c40 con 0x7f4a04150960 2024-09-10T10:29:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49f0007930 con 0x7f4a04150960 2024-09-10T10:29:08.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f0fd8fb7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0fd40cb2f0 0x7f0fd40cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f0fd8fb7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fd415b1d0 con 0x7f0fd415e9d0 2024-09-10T10:29:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f0fd8fb7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd40cbd00 con 0x7f0fd40cb2f0 2024-09-10T10:29:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f0fd8fb7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd415ba70 con 0x7f0fd40cc0c0 2024-09-10T10:29:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f0fd259c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0fd40cc0c0 0x7f0fd415aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f0fd259c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0fd40cc0c0 0x7f0fd415aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:54138/0 (socket says 172.21.15.120:54138) 2024-09-10T10:29:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.261+0000 7f0fd259c640 1 -- 172.21.15.120:0/2673121857 learned_addr learned my addr 172.21.15.120:0/2673121857 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f0fd259c640 1 -- 172.21.15.120:0/2673121857 >> v2:172.21.15.120:3300/0 conn(0x7f0fd40cb2f0 msgr2=0x7f0fd40cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f0fd259c640 1 --2- 172.21.15.120:0/2673121857 >> v2:172.21.15.120:3300/0 conn(0x7f0fd40cb2f0 0x7f0fd40cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f0fd259c640 1 -- 172.21.15.120:0/2673121857 >> v1:172.21.15.120:6789/0 conn(0x7f0fd415e9d0 legacy=0x7f0fd40cd0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:29:08.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f171bfff640 1 --2- 172.21.15.120:0/2195986702 >> v2:172.21.15.134:3300/0 conn(0x7f17140d62e0 0x7f17140dc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7ff1c8d9c640 1 --2- 172.21.15.120:0/478245029 >> v2:172.21.15.120:3300/0 conn(0x7ff1c40bf8b0 0x7ff1c40c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7ff1c9d9e640 1 --2- 172.21.15.120:0/478245029 >> v2:172.21.15.120:3300/0 conn(0x7ff1c40bf8b0 0x7ff1c40c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7ff1c9d9e640 1 -- 172.21.15.120:0/478245029 >> v1:172.21.15.120:6789/0 conn(0x7ff1c4155c50 legacy=0x7ff1c4159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.260+0000 7ff1c9d9e640 1 -- 172.21.15.120:0/478245029 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c406dfc0 con 0x7ff1c40c8640 2024-09-10T10:29:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7ff1c8d9c640 1 --2- 172.21.15.120:0/478245029 >> v2:172.21.15.120:3300/0 conn(0x7ff1c40bf8b0 0x7ff1c40c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7ff1c9d9e640 1 --2- 172.21.15.120:0/478245029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c40c8640 0x7ff1c4073990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009e10 tx=0x7ff1bc02ea80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f12f4dba6c76b0f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f08ab2c9640 1 -- 172.21.15.120:0/1821278453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c0507a0 msgr2=0x7f089c0da570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f08ab2c9640 1 --2- 172.21.15.120:0/1821278453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c0507a0 0x7f089c0da570 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f08940098d0 tx=0x7f089402c480 comp rx=0 tx=0).stop 2024-09-10T10:29:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f08ab2c9640 1 -- 172.21.15.120:0/1821278453 shutdown_connections 2024-09-10T10:29:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f08ab2c9640 1 --2- 172.21.15.120:0/1821278453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f089c04aea0 0x7f089c04b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f08ab2c9640 1 --2- 172.21.15.120:0/1821278453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c0507a0 0x7f089c0da570 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f08ab2c9640 1 -- 172.21.15.120:0/1821278453 >> 172.21.15.120:0/1821278453 conn(0x7f089c04fc80 msgr2=0x7f089c050090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f7a9657f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a90155430 con 0x7f7a900d0fd0 2024-09-10T10:29:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f7a9657f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7a9015aea0 con 0x7f7a900b3790 2024-09-10T10:29:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f7a9657f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9015b180 con 0x7f7a900d0bf0 2024-09-10T10:29:08.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f7a8f7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f7a900d0fd0 0x7f7a900b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:50098/0 (socket says 172.21.15.120:50098) 2024-09-10T10:29:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f7a8f7fe640 1 -- 172.21.15.120:0/3245962268 learned_addr learned my addr 172.21.15.120:0/3245962268 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f7a8ffff640 1 --2- 172.21.15.120:0/3245962268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a900d0bf0 0x7f7a900c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f7a94b1c640 1 --2- 172.21.15.120:0/3245962268 >> v2:172.21.15.120:3300/0 conn(0x7f7a900b3790 0x7f7a900ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7fad996ca640 1 Processor -- start 2024-09-10T10:29:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f171affd640 1 -- 172.21.15.120:0/2195986702 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708037070 con 0x7f17140d66c0 2024-09-10T10:29:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f171affd640 1 -- 172.21.15.120:0/2195986702 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1708032440 con 0x7f17140d66c0 2024-09-10T10:29:08.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f171affd640 1 -- 172.21.15.120:0/2195986702 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17080327f0 con 0x7f17140d66c0 2024-09-10T10:29:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f08ab2c9640 1 -- 172.21.15.120:0/1821278453 shutdown_connections 2024-09-10T10:29:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7fad996ca640 1 -- start start 2024-09-10T10:29:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3959908988 0 0) 0x7f49f0007930 con 0x7f4a04150960 2024-09-10T10:29:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0406d210 msgr2=0x7f4a040c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 --2- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0406d210 0x7f4a040c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 >> v2:172.21.15.120:3300/0 conn(0x7f4a040c8960 msgr2=0x7f4a040c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 --2- 172.21.15.120:0/607863027 >> v2:172.21.15.120:3300/0 conn(0x7f4a040c8960 0x7f4a040c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0415a4c0 con 0x7f4a04150960 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f4a00001670 con 0x7f4a04150960 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 >> v1:172.21.15.134:6789/0 conn(0x7f4a04150960 legacy=0x7f4a0406f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 --2- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49f0007dc0 0x7f49f00081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 --2- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49f0008c10 0x7f49f0009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 --2- 172.21.15.120:0/607863027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49f0007460 0x7f49f00095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4a04159f00 con 0x7f49f0007460 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0415a1e0 con 0x7f49f0007dc0 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49f0014860 con 0x7f49f0008c10 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7f4a0a01e640 1 --2- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49f0007dc0 0x7f49f00081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7f4a0a01e640 1 -- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49f0008c10 msgr2=0x7f49f0009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:29:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7f4a0a01e640 1 --2- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49f0008c10 0x7f49f0009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7f4a0a01e640 1 -- 172.21.15.120:0/607863027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49f0007460 msgr2=0x7f49f00095d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:29:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7f4a0a01e640 1 --2- 172.21.15.120:0/607863027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49f0007460 0x7f49f00095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7f4a0a01e640 1 -- 172.21.15.120:0/607863027 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f0007930 con 0x7f49f0007dc0 2024-09-10T10:29:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f0fd259c640 1 -- 172.21.15.120:0/2673121857 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd415b4b0 con 0x7f0fd40cc0c0 2024-09-10T10:29:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7ff1bbfff640 1 -- 172.21.15.120:0/478245029 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc032840 con 0x7ff1c40c8640 2024-09-10T10:29:08.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7ff1bbfff640 1 -- 172.21.15.120:0/478245029 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc0329e0 con 0x7ff1c40c8640 2024-09-10T10:29:08.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7ff1bbfff640 1 -- 172.21.15.120:0/478245029 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc032d90 con 0x7ff1c40c8640 2024-09-10T10:29:08.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.262+0000 7f08ab2c9640 1 -- 172.21.15.120:0/1821278453 wait complete. 2024-09-10T10:29:08.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7f7a94b1c640 1 -- 172.21.15.120:0/3245962268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a900d0bf0 msgr2=0x7f7a900c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7f7a94b1c640 1 --2- 172.21.15.120:0/3245962268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a900d0bf0 0x7f7a900c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7ff1cb801640 1 -- 172.21.15.120:0/478245029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c40c8640 msgr2=0x7ff1c4073990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7ff1cb801640 1 --2- 172.21.15.120:0/478245029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c40c8640 0x7ff1c4073990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009e10 tx=0x7ff1bc02ea80 comp rx=0 tx=0).stop 2024-09-10T10:29:08.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7f7a94b1c640 1 -- 172.21.15.120:0/3245962268 >> v1:172.21.15.134:6789/0 conn(0x7f7a900d0fd0 legacy=0x7f7a900b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7f7a94b1c640 1 -- 172.21.15.120:0/3245962268 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9015b460 con 0x7f7a900b3790 2024-09-10T10:29:08.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7ff1cb801640 1 -- 172.21.15.120:0/478245029 shutdown_connections 2024-09-10T10:29:08.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7ff1cb801640 1 --2- 172.21.15.120:0/478245029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c40c8640 0x7ff1c4073990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7ff1cb801640 1 --2- 172.21.15.120:0/478245029 >> v2:172.21.15.120:3300/0 conn(0x7ff1c40bf8b0 0x7ff1c40c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.263+0000 7ff1cb801640 1 -- 172.21.15.120:0/478245029 >> 172.21.15.120:0/478245029 conn(0x7ff1c40cbf90 msgr2=0x7ff1c40cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad996ca640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c04c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad996ca640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fad8c005cc0 0x7fad8c006170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad996ca640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad8c0e33a0 con 0x7fad8c0dcd00 2024-09-10T10:29:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad996ca640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fad8c0066b0 con 0x7fad8c005cc0 2024-09-10T10:29:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad996ca640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad8c0dd0e0 con 0x7fad8c04a7b0 2024-09-10T10:29:08.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad937fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c04c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f1721504640 1 -- 172.21.15.120:0/2195986702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17140d66c0 msgr2=0x7f171404bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f1721504640 1 --2- 172.21.15.120:0/2195986702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17140d66c0 0x7f171404bc50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1708009870 tx=0x7f170802a230 comp rx=0 tx=0).stop 2024-09-10T10:29:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f0fd259c640 1 --2- 172.21.15.120:0/2673121857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd40cc0c0 0x7f0fd415aab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40ce150 tx=0x7f0fcc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81e544bfdeab024 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f0fd159a640 1 -- 172.21.15.120:0/2673121857 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc036070 con 0x7f0fd40cc0c0 2024-09-10T10:29:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f0fd159a640 1 -- 172.21.15.120:0/2673121857 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fcc031600 con 0x7f0fd40cc0c0 2024-09-10T10:29:08.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f0fd159a640 1 -- 172.21.15.120:0/2673121857 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0319d0 con 0x7f0fd40cc0c0 2024-09-10T10:29:08.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f08ab2c9640 1 Processor -- start 2024-09-10T10:29:08.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f4a0a01e640 1 --2- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49f0007dc0 0x7f49f00081e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c1c0 tx=0x7f49f0009c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fb36360f5b6c5cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00007cc0 con 0x7f49f0007dc0 2024-09-10T10:29:08.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a00005af0 con 0x7f49f0007dc0 2024-09-10T10:29:08.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f4a0901c640 1 -- 172.21.15.120:0/607863027 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00007070 con 0x7f49f0007dc0 2024-09-10T10:29:08.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad937fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c04c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:40598/0 (socket says 172.21.15.120:40598) 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad937fe640 1 -- 172.21.15.120:0/3350507104 learned_addr learned my addr 172.21.15.120:0/3350507104 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad9a92c640 1 --2- 172.21.15.120:0/3350507104 >> v2:172.21.15.120:3300/0 conn(0x7fad8c005cc0 0x7fad8c006170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad937fe640 1 -- 172.21.15.120:0/3350507104 >> v2:172.21.15.120:3300/0 conn(0x7fad8c005cc0 msgr2=0x7fad8c006170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad937fe640 1 --2- 172.21.15.120:0/3350507104 >> v2:172.21.15.120:3300/0 conn(0x7fad8c005cc0 0x7fad8c006170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad937fe640 1 -- 172.21.15.120:0/3350507104 >> v1:172.21.15.120:6789/0 conn(0x7fad8c0dcd00 legacy=0x7fad8c04b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7fad937fe640 1 -- 172.21.15.120:0/3350507104 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c0e3640 con 0x7fad8c04a7b0 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f1721504640 1 -- 172.21.15.120:0/2195986702 shutdown_connections 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f1721504640 1 --2- 172.21.15.120:0/2195986702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17140d66c0 0x7f171404bc50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f1721504640 1 --2- 172.21.15.120:0/2195986702 >> v2:172.21.15.134:3300/0 conn(0x7f17140d62e0 0x7f17140dc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f1721504640 1 -- 172.21.15.120:0/2195986702 >> 172.21.15.120:0/2195986702 conn(0x7f171404fc50 msgr2=0x7f1714050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f1721504640 1 -- 172.21.15.120:0/2195986702 shutdown_connections 2024-09-10T10:29:08.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7ff1cb801640 1 -- 172.21.15.120:0/478245029 shutdown_connections 2024-09-10T10:29:08.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.264+0000 7f08ab2c9640 1 -- start start 2024-09-10T10:29:08.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f7a94b1c640 1 --2- 172.21.15.120:0/3245962268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 0x7f7a900ca520 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7a780097a0 tx=0x7f7a7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaf98a551e19d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/2673121857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd40cc0c0 msgr2=0x7f0fd415aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f0fd8fb7640 1 --2- 172.21.15.120:0/2673121857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd40cc0c0 0x7f0fd415aab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40ce150 tx=0x7f0fcc02a6f0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f7a8effd640 1 -- 172.21.15.120:0/3245962268 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78037070 con 0x7f7a900b3790 2024-09-10T10:29:08.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7ff1cb801640 1 -- 172.21.15.120:0/478245029 wait complete. 2024-09-10T10:29:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f7a8effd640 1 -- 172.21.15.120:0/3245962268 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a78032440 con 0x7f7a900b3790 2024-09-10T10:29:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/2673121857 shutdown_connections 2024-09-10T10:29:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f0fd8fb7640 1 --2- 172.21.15.120:0/2673121857 >> v2:172.21.15.120:3300/0 conn(0x7f0fd40cb2f0 0x7f0fd40cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f0fd8fb7640 1 --2- 172.21.15.120:0/2673121857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd40cc0c0 0x7f0fd415aab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/2673121857 >> 172.21.15.120:0/2673121857 conn(0x7f0fd406d560 msgr2=0x7f0fd406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f08ab2c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f089c005a30 0x7f089c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f08ab2c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c04aea0 0x7f089c168ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f4a0b020640 1 -- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49f0007dc0 msgr2=0x7f49f00081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f7a8effd640 1 -- 172.21.15.120:0/3245962268 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78032810 con 0x7f7a900b3790 2024-09-10T10:29:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f08ab2c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f089c0507a0 0x7f089c169410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f08ab2c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f089c006e60 con 0x7f089c04aea0 2024-09-10T10:29:08.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f08ab2c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f089c006b80 con 0x7f089c005a30 2024-09-10T10:29:08.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f08ab2c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f089c04a600 con 0x7f089c0507a0 2024-09-10T10:29:08.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f4a0b020640 1 --2- 172.21.15.120:0/607863027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49f0007dc0 0x7f49f00081e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c1c0 tx=0x7f49f0009c40 comp rx=0 tx=0).stop 2024-09-10T10:29:08.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f4a0a81f640 1 -- 172.21.15.120:0/607863027 reap_dead start 2024-09-10T10:29:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f08a9065640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f089c005a30 0x7f089c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f08a9065640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f089c005a30 0x7f089c0c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:54152/0 (socket says 172.21.15.120:54152) 2024-09-10T10:29:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f08a9065640 1 -- 172.21.15.120:0/4239730027 learned_addr learned my addr 172.21.15.120:0/4239730027 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f08a9866640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f089c0507a0 0x7f089c169410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f4a0b020640 1 -- 172.21.15.120:0/607863027 shutdown_connections 2024-09-10T10:29:08.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f4a0b020640 1 -- 172.21.15.120:0/607863027 >> 172.21.15.120:0/607863027 conn(0x7f4a040cc2c0 msgr2=0x7f4a040cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f08a8864640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c04aea0 0x7f089c168ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f08a9065640 1 -- 172.21.15.120:0/4239730027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f089c0507a0 msgr2=0x7f089c169410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f4a0b020640 1 -- 172.21.15.120:0/607863027 shutdown_connections 2024-09-10T10:29:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f4a0b020640 1 -- 172.21.15.120:0/607863027 wait complete. 2024-09-10T10:29:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f08a9065640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f089c0507a0 0x7f089c169410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f08a9065640 1 -- 172.21.15.120:0/4239730027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c04aea0 msgr2=0x7f089c168ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f08a9065640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c04aea0 0x7f089c168ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f08a9065640 1 -- 172.21.15.120:0/4239730027 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c0c96d0 con 0x7f089c005a30 2024-09-10T10:29:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/2673121857 shutdown_connections 2024-09-10T10:29:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/2673121857 wait complete. 2024-09-10T10:29:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7fad937fe640 1 --2- 172.21.15.120:0/3350507104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c04c2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad80009870 tx=0x7fad8002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d0e9273db291795 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.266+0000 7f08a9065640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f089c005a30 0x7f089c0c4a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f08940098d0 tx=0x7f08940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7f0899ffb640 1 -- 172.21.15.120:0/4239730027 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894042020 con 0x7f089c005a30 2024-09-10T10:29:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7ff1cb801640 1 Processor -- start 2024-09-10T10:29:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7fad92ffd640 1 -- 172.21.15.120:0/3350507104 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80037070 con 0x7fad8c04a7b0 2024-09-10T10:29:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7fad92ffd640 1 -- 172.21.15.120:0/3350507104 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8002ae50 con 0x7fad8c04a7b0 2024-09-10T10:29:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7fad92ffd640 1 -- 172.21.15.120:0/3350507104 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80032860 con 0x7fad8c04a7b0 2024-09-10T10:29:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7ff1cb801640 1 -- start start 2024-09-10T10:29:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.265+0000 7f1721504640 1 -- 172.21.15.120:0/2195986702 wait complete. 2024-09-10T10:29:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7f1721504640 1 Processor -- start 2024-09-10T10:29:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7f7a9657f640 1 -- 172.21.15.120:0/3245962268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 msgr2=0x7f7a900ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7f08ab2c9640 1 -- 172.21.15.120:0/4239730027 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c168420 con 0x7f089c005a30 2024-09-10T10:29:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7f08ab2c9640 1 -- 172.21.15.120:0/4239730027 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c0c9c70 con 0x7f089c005a30 2024-09-10T10:29:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7f1721504640 1 -- start start 2024-09-10T10:29:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7f0899ffb640 1 -- 172.21.15.120:0/4239730027 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0894033830 con 0x7f089c005a30 2024-09-10T10:29:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7f7a9657f640 1 --2- 172.21.15.120:0/3245962268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 0x7f7a900ca520 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7a780097a0 tx=0x7f7a7802a230 comp rx=0 tx=0).stop 2024-09-10T10:29:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f7a9657f640 1 -- 172.21.15.120:0/3245962268 shutdown_connections 2024-09-10T10:29:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f4a0b020640 1 Processor -- start 2024-09-10T10:29:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f7a9657f640 1 --2- 172.21.15.120:0/3245962268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 0x7f7a900ca520 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f7a9657f640 1 --2- 172.21.15.120:0/3245962268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a900d0bf0 0x7f7a900c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7ff1cb801640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c40bf8b0 0x7ff1c41eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f7a9657f640 1 -- 172.21.15.120:0/3245962268 >> 172.21.15.120:0/3245962268 conn(0x7f7a9006c900 msgr2=0x7f7a9006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7f0fd8fb7640 1 Processor -- start 2024-09-10T10:29:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f0899ffb640 1 -- 172.21.15.120:0/4239730027 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894038040 con 0x7f089c005a30 2024-09-10T10:29:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.267+0000 7f0fd8fb7640 1 -- start start 2024-09-10T10:29:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f7a9657f640 1 -- 172.21.15.120:0/3245962268 shutdown_connections 2024-09-10T10:29:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f4a0b020640 1 -- start start 2024-09-10T10:29:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f0fd8fb7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fd40cb2f0 0x7f0fd40c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f0fd8fb7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd40cc0c0 0x7f0fd40c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f0fd8fb7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fd415e9d0 0x7f0fd414a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f0fd8fb7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd415e1d0 con 0x7f0fd415e9d0 2024-09-10T10:29:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f0fd8fb7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd415e540 con 0x7f0fd40cc0c0 2024-09-10T10:29:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f0fd8fb7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd40c5560 con 0x7f0fd40cb2f0 2024-09-10T10:29:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f1721504640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1714044b30 0x7f17140cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7ff1cb801640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1c40c8640 0x7ff1c40c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7ff1cb801640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c4155c50 0x7ff1c40c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7ff1cb801640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4073ed0 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7ff1cb801640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c406d2f0 con 0x7ff1c40c8640 2024-09-10T10:29:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7ff1cb801640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c4074050 con 0x7ff1c4155c50 2024-09-10T10:29:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7ff1c959d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c40bf8b0 0x7ff1c41eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.268+0000 7f7a9657f640 1 -- 172.21.15.120:0/3245962268 wait complete. 2024-09-10T10:29:08.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f1721504640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17140d62e0 0x7f17140481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f0fd2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fd415e9d0 0x7f0fd414a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f0fd2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fd415e9d0 0x7f0fd414a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42342/0 (socket says 172.21.15.120:42342) 2024-09-10T10:29:08.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f0fd2d9d640 1 -- 172.21.15.120:0/3631374894 learned_addr learned my addr 172.21.15.120:0/3631374894 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7ff1c959d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c40bf8b0 0x7ff1c41eccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42332/0 (socket says 172.21.15.120:42332) 2024-09-10T10:29:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7ff1c959d640 1 -- 172.21.15.120:0/3681876537 learned_addr learned my addr 172.21.15.120:0/3681876537 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f0fd259c640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fd40cb2f0 0x7f0fd40c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7fad996ca640 1 -- 172.21.15.120:0/3350507104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 msgr2=0x7fad8c04c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7fad996ca640 1 --2- 172.21.15.120:0/3350507104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c04c2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad80009870 tx=0x7fad8002a300 comp rx=0 tx=0).stop 2024-09-10T10:29:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7ff1c8d9c640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1c40c8640 0x7ff1c40c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f171bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1714044b30 0x7f17140cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f171bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1714044b30 0x7f17140cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:40628/0 (socket says 172.21.15.120:40628) 2024-09-10T10:29:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f171bfff640 1 -- 172.21.15.120:0/454152350 learned_addr learned my addr 172.21.15.120:0/454152350 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f0fd1d9b640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd40cc0c0 0x7f0fd40c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f171b7fe640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17140d62e0 0x7f17140481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0b020640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49f0007dc0 0x7f4a04152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0b020640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04150310 0x7f4a04153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0b020640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a04158b40 0x7f4a04158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f0fd2d9d640 1 -- 172.21.15.120:0/3631374894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fd40cb2f0 msgr2=0x7f0fd40c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f0fd2d9d640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fd40cb2f0 0x7f0fd40c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f0fd2d9d640 1 -- 172.21.15.120:0/3631374894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd40cc0c0 msgr2=0x7f0fd40c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f0fd2d9d640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd40cc0c0 0x7f0fd40c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f0fd2d9d640 1 -- 172.21.15.120:0/3631374894 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd415aaf0 con 0x7f0fd415e9d0 2024-09-10T10:29:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f1721504640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17140486e0 0x7f1714173a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7ff1c9d9e640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c4155c50 0x7ff1c40c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7ff1c959d640 1 -- 172.21.15.120:0/3681876537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c4155c50 msgr2=0x7ff1c40c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7ff1c959d640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c4155c50 0x7ff1c40c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7ff1c959d640 1 -- 172.21.15.120:0/3681876537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1c40c8640 msgr2=0x7ff1c40c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7ff1c959d640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1c40c8640 0x7ff1c40c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7ff1c959d640 1 -- 172.21.15.120:0/3681876537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c4159550 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.269+0000 7f08ab2c9640 1 -- 172.21.15.120:0/4239730027 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0884005290 con 0x7f089c005a30 2024-09-10T10:29:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0b020640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4a040c74f0 con 0x7f4a04158b40 2024-09-10T10:29:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0b020640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0415aa80 con 0x7f4a04150310 2024-09-10T10:29:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0b020640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4a04150d40 con 0x7f49f0007dc0 2024-09-10T10:29:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0a01e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49f0007dc0 0x7f4a04152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0a01e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49f0007dc0 0x7f4a04152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:40632/0 (socket says 172.21.15.120:40632) 2024-09-10T10:29:08.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0a01e640 1 -- 172.21.15.120:0/3627313531 learned_addr learned my addr 172.21.15.120:0/3627313531 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f1720d03640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17140486e0 0x7f1714173a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17140dc930 con 0x7f17140486e0 2024-09-10T10:29:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f171404b730 con 0x7f17140d62e0 2024-09-10T10:29:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17140355e0 con 0x7f1714044b30 2024-09-10T10:29:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7fad996ca640 1 -- 172.21.15.120:0/3350507104 shutdown_connections 2024-09-10T10:29:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7fad996ca640 1 --2- 172.21.15.120:0/3350507104 >> v2:172.21.15.120:3300/0 conn(0x7fad8c005cc0 0x7fad8c006170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7fad996ca640 1 --2- 172.21.15.120:0/3350507104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c04c2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7fad996ca640 1 -- 172.21.15.120:0/3350507104 >> 172.21.15.120:0/3350507104 conn(0x7fad8c04fe30 msgr2=0x7fad8c050240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7fad996ca640 1 -- 172.21.15.120:0/3350507104 shutdown_connections 2024-09-10T10:29:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7fad996ca640 1 -- 172.21.15.120:0/3350507104 wait complete. 2024-09-10T10:29:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f0fd2d9d640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fd415e9d0 0x7f0fd414a820 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c440 tx=0x7f0fc400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f1720d03640 1 -- 172.21.15.120:0/454152350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1714044b30 msgr2=0x7f17140cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f1720d03640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1714044b30 0x7f17140cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f1720d03640 1 -- 172.21.15.120:0/454152350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17140d62e0 msgr2=0x7f17140481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f1720d03640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17140d62e0 0x7f17140481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f1720d03640 1 -- 172.21.15.120:0/454152350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17140d0150 con 0x7f17140486e0 2024-09-10T10:29:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f171bfff640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1714044b30 0x7f17140cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:29:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f7a9657f640 1 Processor -- start 2024-09-10T10:29:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f7a9657f640 1 -- start start 2024-09-10T10:29:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0a81f640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a04158b40 0x7f4a04158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.270+0000 7f4a0981d640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04150310 0x7f4a04153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f1720d03640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17140486e0 0x7f1714173a70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f171c051790 tx=0x7f171c069890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7ff1c959d640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c40bf8b0 0x7ff1c41eccd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c420 tx=0x7ff1b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f0899ffb640 1 -- 172.21.15.120:0/4239730027 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0894045020 con 0x7f089c005a30 2024-09-10T10:29:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f4a0a81f640 1 -- 172.21.15.120:0/3627313531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49f0007dc0 msgr2=0x7f4a04152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f4a0a81f640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49f0007dc0 0x7f4a04152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f4a0a81f640 1 -- 172.21.15.120:0/3627313531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04150310 msgr2=0x7f4a04153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f4a0a81f640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04150310 0x7f4a04153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f4a0a81f640 1 -- 172.21.15.120:0/3627313531 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a04151080 con 0x7f4a04158b40 2024-09-10T10:29:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f0fc2ffd640 1 -- 172.21.15.120:0/3631374894 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4011790 con 0x7f0fd415e9d0 2024-09-10T10:29:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f0fc2ffd640 1 -- 172.21.15.120:0/3631374894 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc4017070 con 0x7f0fd415e9d0 2024-09-10T10:29:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f0fc2ffd640 1 -- 172.21.15.120:0/3631374894 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4011b70 con 0x7f0fd415e9d0 2024-09-10T10:29:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f7a9657f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 0x7f7a901f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f0899ffb640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f088806be80 0x7f088806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f7a9657f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a900d0bf0 0x7f7a901f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f7a9657f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a900d0fd0 0x7f7a901f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a9657f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9015edc0 con 0x7f7a900b3790 2024-09-10T10:29:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a9657f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9015ba20 con 0x7f7a900d0fd0 2024-09-10T10:29:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a9657f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9015ef40 con 0x7f7a900d0bf0 2024-09-10T10:29:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 0x7f7a901f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 0x7f7a901f39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42372/0 (socket says 172.21.15.120:42372) 2024-09-10T10:29:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a8ffff640 1 -- 172.21.15.120:0/872006456 learned_addr learned my addr 172.21.15.120:0/872006456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a94b1c640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a900d0fd0 0x7f7a901f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a8f7fe640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a900d0bf0 0x7f7a901f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/3631374894 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd40c9ff0 con 0x7f0fd415e9d0 2024-09-10T10:29:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.271+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/3631374894 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd40ca5b0 con 0x7f0fd415e9d0 2024-09-10T10:29:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f08a8864640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f088806be80 0x7f088806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a8ffff640 1 -- 172.21.15.120:0/872006456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a900d0bf0 msgr2=0x7f7a901f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a8ffff640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a900d0bf0 0x7f7a901f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a8ffff640 1 -- 172.21.15.120:0/872006456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a900d0fd0 msgr2=0x7f7a901f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a8ffff640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a900d0fd0 0x7f7a901f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a8ffff640 1 -- 172.21.15.120:0/872006456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a900c8610 con 0x7f7a900b3790 2024-09-10T10:29:08.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f0899ffb640 1 -- 172.21.15.120:0/4239730027 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f08940abf30 con 0x7f089c005a30 2024-09-10T10:29:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f7a8ffff640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 0x7f7a901f39c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7a88007f20 tx=0x7f7a8800e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f1718ff9640 1 -- 172.21.15.120:0/454152350 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c091070 con 0x7f17140486e0 2024-09-10T10:29:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f1718ff9640 1 -- 172.21.15.120:0/454152350 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f171c071840 con 0x7f17140486e0 2024-09-10T10:29:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f1718ff9640 1 -- 172.21.15.120:0/454152350 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c071cb0 con 0x7f17140486e0 2024-09-10T10:29:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7ff1b9ffb640 1 -- 172.21.15.120:0/3681876537 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40074e0 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7f4a0a81f640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a04158b40 0x7f4a04158ff0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4a04070c60 tx=0x7f49fc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.273+0000 7ff1cb801640 1 -- 172.21.15.120:0/3681876537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c415a2d0 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.273+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/3631374894 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd4151200 con 0x7f0fd415e9d0 2024-09-10T10:29:08.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.273+0000 7ff1cb801640 1 -- 172.21.15.120:0/3681876537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c415a890 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.273+0000 7ff1b9ffb640 1 -- 172.21.15.120:0/3681876537 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b40047e0 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.273+0000 7f08a8864640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f088806be80 0x7f088806e340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f08a4051790 tx=0x7f08a406e040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7fad996ca640 1 Processor -- start 2024-09-10T10:29:08.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.272+0000 7fad996ca640 1 -- start start 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.273+0000 7fad996ca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad8c005cc0 0x7fad8c0d4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7fad996ca640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c17bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7fad996ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad8c0dcd00 0x7fad8c0d3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7fad937fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c17bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7fad937fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c17bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:40636/0 (socket says 172.21.15.120:40636) 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7fad937fe640 1 -- 172.21.15.120:0/1219452316 learned_addr learned my addr 172.21.15.120:0/1219452316 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7fad93fff640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad8c005cc0 0x7fad8c0d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c045b60 con 0x7fad8c0dcd00 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c045880 con 0x7fad8c005cc0 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad8c045e40 con 0x7fad8c04a7b0 2024-09-10T10:29:08.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17140d1f10 con 0x7f17140486e0 2024-09-10T10:29:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1714047100 con 0x7f17140486e0 2024-09-10T10:29:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.274+0000 7ff1b9ffb640 1 -- 172.21.15.120:0/3681876537 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4004c10 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.273+0000 7f7a8cff9640 1 -- 172.21.15.120:0/872006456 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88014620 con 0x7f7a900b3790 2024-09-10T10:29:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.273+0000 7f7a8cff9640 1 -- 172.21.15.120:0/872006456 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a88019070 con 0x7f7a900b3790 2024-09-10T10:29:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.273+0000 7f7a8cff9640 1 -- 172.21.15.120:0/872006456 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88014a00 con 0x7f7a900b3790 2024-09-10T10:29:08.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7f7a9657f640 1 -- 172.21.15.120:0/872006456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a901571f0 con 0x7f7a900b3790 2024-09-10T10:29:08.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7f0fc2ffd640 1 -- 172.21.15.120:0/3631374894 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fc4011d10 con 0x7f0fd415e9d0 2024-09-10T10:29:08.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7fad937fe640 1 -- 172.21.15.120:0/1219452316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad8c005cc0 msgr2=0x7fad8c0d4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7fad937fe640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad8c005cc0 0x7fad8c0d4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7fad937fe640 1 -- 172.21.15.120:0/1219452316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad8c0dcd00 msgr2=0x7fad8c0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7fad937fe640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad8c0dcd00 0x7fad8c0d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7fad937fe640 1 -- 172.21.15.120:0/1219452316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c0d5d60 con 0x7fad8c04a7b0 2024-09-10T10:29:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7f49faffd640 1 -- 172.21.15.120:0/3627313531 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc00bc20 con 0x7f4a04158b40 2024-09-10T10:29:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7f4a0b020640 1 -- 172.21.15.120:0/3627313531 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a041efde0 con 0x7f4a04158b40 2024-09-10T10:29:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7ff1b9ffb640 1 -- 172.21.15.120:0/3681876537 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1b4011070 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7f7a9657f640 1 -- 172.21.15.120:0/872006456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a901f1780 con 0x7f7a900b3790 2024-09-10T10:29:08.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7f0fc2ffd640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fb406c080 0x7f0fb406e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7f0fc2ffd640 1 -- 172.21.15.120:0/3631374894 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f0fc4011930 con 0x7f0fd415e9d0 2024-09-10T10:29:08.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.275+0000 7f0fd259c640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fb406c080 0x7f0fb406e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7fad937fe640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c17bc70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad80000c00 tx=0x7fad800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7f4a0b020640 1 -- 172.21.15.120:0/3627313531 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a04151600 con 0x7f4a04158b40 2024-09-10T10:29:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7fad90ff9640 1 -- 172.21.15.120:0/1219452316 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80041070 con 0x7fad8c04a7b0 2024-09-10T10:29:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171404b020 con 0x7f17140486e0 2024-09-10T10:29:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7f7a9657f640 1 -- 172.21.15.120:0/872006456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a900b3080 con 0x7f7a900b3790 2024-09-10T10:29:08.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7fad90ff9640 1 -- 172.21.15.120:0/1219452316 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8003c870 con 0x7fad8c04a7b0 2024-09-10T10:29:08.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7fad90ff9640 1 -- 172.21.15.120:0/1219452316 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80037040 con 0x7fad8c04a7b0 2024-09-10T10:29:08.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7f0899ffb640 1 -- 172.21.15.120:0/4239730027 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f089404c030 con 0x7f089c005a30 2024-09-10T10:29:08.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7f0fd259c640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fb406c080 0x7f0fb406e540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc033000 tx=0x7f0fcc009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8c16bd30 con 0x7fad8c04a7b0 2024-09-10T10:29:08.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7ff1cb801640 1 -- 172.21.15.120:0/3681876537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a4005290 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.276+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8c0d62e0 con 0x7fad8c04a7b0 2024-09-10T10:29:08.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.277+0000 7f4a0b020640 1 -- 172.21.15.120:0/3627313531 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a040cc8f0 con 0x7f4a04158b40 2024-09-10T10:29:08.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.277+0000 7ff1b9ffb640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1a806be30 0x7ff1a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.278+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8c00dec0 con 0x7fad8c04a7b0 2024-09-10T10:29:08.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.278+0000 7ff1b9ffb640 1 -- 172.21.15.120:0/3681876537 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7ff1b408af00 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.278+0000 7ff1c8d9c640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1a806be30 0x7ff1a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.278+0000 7ff1c8d9c640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1a806be30 0x7ff1a806e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40c4660 tx=0x7ff1c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.279+0000 7fad90ff9640 1 -- 172.21.15.120:0/1219452316 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad80044070 con 0x7fad8c04a7b0 2024-09-10T10:29:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.279+0000 7fad90ff9640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fad7806be80 0x7fad7806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.280+0000 7fad90ff9640 1 -- 172.21.15.120:0/1219452316 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fad800ac050 con 0x7fad8c04a7b0 2024-09-10T10:29:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.280+0000 7fad93fff640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fad7806be80 0x7fad7806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.280+0000 7fad93fff640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fad7806be80 0x7fad7806e340 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad88000f60 tx=0x7fad88009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.282+0000 7f49faffd640 1 -- 172.21.15.120:0/3627313531 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49fc003990 con 0x7f4a04158b40 2024-09-10T10:29:08.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.282+0000 7f0fc2ffd640 1 -- 172.21.15.120:0/3631374894 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0fc4021020 con 0x7f0fd415e9d0 2024-09-10T10:29:08.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.282+0000 7f49faffd640 1 -- 172.21.15.120:0/3627313531 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc003d40 con 0x7f4a04158b40 2024-09-10T10:29:08.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.283+0000 7fad90ff9640 1 -- 172.21.15.120:0/1219452316 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad8c00dec0 con 0x7fad8c04a7b0 2024-09-10T10:29:08.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.285+0000 7f7a8cff9640 1 -- 172.21.15.120:0/872006456 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a88014ba0 con 0x7f7a900b3790 2024-09-10T10:29:08.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.286+0000 7f7a8cff9640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7a7006be80 0x7f7a7006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.286+0000 7f7a8f7fe640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7a7006be80 0x7f7a7006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.287+0000 7f7a8f7fe640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7a7006be80 0x7f7a7006e340 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a9006cf70 tx=0x7f7a80007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.293+0000 7ff1b9ffb640 1 -- 172.21.15.120:0/3681876537 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff1b4058c30 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.295+0000 7f1718ff9640 1 -- 172.21.15.120:0/454152350 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f171c094070 con 0x7f17140486e0 2024-09-10T10:29:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.296+0000 7f1718ff9640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f170006be60 0x7f170006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.296+0000 7f1718ff9640 1 -- 172.21.15.120:0/454152350 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f171c107610 con 0x7f17140486e0 2024-09-10T10:29:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.296+0000 7f171bfff640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f170006be60 0x7f170006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.296+0000 7f7a8cff9640 1 -- 172.21.15.120:0/872006456 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f7a88058930 con 0x7f7a900b3790 2024-09-10T10:29:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.297+0000 7f171bfff640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f170006be60 0x7f170006e320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f171000f240 tx=0x7f1710008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.297+0000 7f49faffd640 1 -- 172.21.15.120:0/3627313531 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49fc004030 con 0x7f4a04158b40 2024-09-10T10:29:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.298+0000 7f49faffd640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f49e806be30 0x7f49e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.298+0000 7f49faffd640 1 -- 172.21.15.120:0/3627313531 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f49fc08b630 con 0x7f4a04158b40 2024-09-10T10:29:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.298+0000 7f4a0a01e640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f49e806be30 0x7f49e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.299+0000 7f4a0a01e640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f49e806be30 0x7f49e806e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c1c0 tx=0x7f4a00009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.305+0000 7f7a8cff9640 1 -- 172.21.15.120:0/872006456 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7a900c8610 con 0x7f7a900b3790 2024-09-10T10:29:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.314+0000 7f1718ff9640 1 -- 172.21.15.120:0/454152350 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f171404b020 con 0x7f17140486e0 2024-09-10T10:29:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.323+0000 7f49faffd640 1 -- 172.21.15.120:0/3627313531 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f49fc01b060 con 0x7f4a04158b40 2024-09-10T10:29:08.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.620+0000 7f33af69b640 1 -- 172.21.15.120:0/3347165246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3388005580 con 0x7f33a8159320 2024-09-10T10:29:08.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.621+0000 7f339dffb640 1 -- 172.21.15.120:0/3347165246 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f33a0059510 con 0x7f33a8159320 2024-09-10T10:29:08.622 INFO:teuthology.orchestra.run.smithi120.stdout:55834574883 2024-09-10T10:29:08.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.624+0000 7f33937fe640 1 -- 172.21.15.120:0/3347165246 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f338c06bf30 msgr2=0x7f338c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.624+0000 7f33937fe640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f338c06bf30 0x7f338c06e3f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f33a4034000 tx=0x7f33a402e000 comp rx=0 tx=0).stop 2024-09-10T10:29:08.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.625+0000 7f33937fe640 1 -- 172.21.15.120:0/3347165246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33a8159320 msgr2=0x7f33a80c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.625+0000 7f33937fe640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33a8159320 0x7f33a80c5150 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f33a000e190 tx=0x7f33a000e650 comp rx=0 tx=0).stop 2024-09-10T10:29:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.625+0000 7f33937fe640 1 -- 172.21.15.120:0/3347165246 shutdown_connections 2024-09-10T10:29:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.625+0000 7f33937fe640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f338c06bf30 0x7f338c06e3f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.625+0000 7f33937fe640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a8162c00 0x7f33a80c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.625+0000 7f33937fe640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33a8159320 0x7f33a80c5150 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.625+0000 7f33937fe640 1 --2- 172.21.15.120:0/3347165246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33a8158f40 0x7f33a815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.626+0000 7f33937fe640 1 -- 172.21.15.120:0/3347165246 >> 172.21.15.120:0/3347165246 conn(0x7f33a806f3c0 msgr2=0x7f33a8156760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.626+0000 7f33937fe640 1 -- 172.21.15.120:0/3347165246 shutdown_connections 2024-09-10T10:29:08.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.626+0000 7f33937fe640 1 -- 172.21.15.120:0/3347165246 wait complete. 2024-09-10T10:29:08.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574883 for osd.0 2024-09-10T10:29:08.643 DEBUG:teuthology.parallel:result is None 2024-09-10T10:29:08.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.653+0000 7f08ab2c9640 1 -- 172.21.15.120:0/4239730027 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0884005580 con 0x7f089c005a30 2024-09-10T10:29:08.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.653+0000 7f0899ffb640 1 -- 172.21.15.120:0/4239730027 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f089407a340 con 0x7f089c005a30 2024-09-10T10:29:08.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.654+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/3631374894 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0fd40cbea0 con 0x7f0fd415e9d0 2024-09-10T10:29:08.655 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:08.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.654+0000 7f0fc2ffd640 1 -- 172.21.15.120:0/3631374894 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0fc4059460 con 0x7f0fd415e9d0 2024-09-10T10:29:08.658 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.657+0000 7f08ab2c9640 1 -- 172.21.15.120:0/4239730027 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f088806be80 msgr2=0x7f088806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.657+0000 7f08ab2c9640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f088806be80 0x7f088806e340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f08a4051790 tx=0x7f08a406e040 comp rx=0 tx=0).stop 2024-09-10T10:29:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.657+0000 7f08ab2c9640 1 -- 172.21.15.120:0/4239730027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f089c005a30 msgr2=0x7f089c0c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.657+0000 7f08ab2c9640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f089c005a30 0x7f089c0c4a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f08940098d0 tx=0x7f08940046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/3631374894 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fb406c080 msgr2=0x7f0fb406e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f0fd8fb7640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fb406c080 0x7f0fb406e540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc033000 tx=0x7f0fcc009000 comp rx=0 tx=0).stop 2024-09-10T10:29:08.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f08ab2c9640 1 -- 172.21.15.120:0/4239730027 shutdown_connections 2024-09-10T10:29:08.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f08ab2c9640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f088806be80 0x7f088806e340 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f08ab2c9640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f089c0507a0 0x7f089c169410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f08ab2c9640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c04aea0 0x7f089c168ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f08ab2c9640 1 --2- 172.21.15.120:0/4239730027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f089c005a30 0x7f089c0c4a00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f08ab2c9640 1 -- 172.21.15.120:0/4239730027 >> 172.21.15.120:0/4239730027 conn(0x7f089c04fc80 msgr2=0x7f089c0d83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f08ab2c9640 1 -- 172.21.15.120:0/4239730027 shutdown_connections 2024-09-10T10:29:08.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f08ab2c9640 1 -- 172.21.15.120:0/4239730027 wait complete. 2024-09-10T10:29:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/3631374894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fd415e9d0 msgr2=0x7f0fd414a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f0fd8fb7640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fd415e9d0 0x7f0fd414a820 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c440 tx=0x7f0fc400c900 comp rx=0 tx=0).stop 2024-09-10T10:29:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/3631374894 shutdown_connections 2024-09-10T10:29:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.658+0000 7f0fd8fb7640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0fb406c080 0x7f0fb406e540 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.659+0000 7f0fd8fb7640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fd415e9d0 0x7f0fd414a820 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.659+0000 7f0fd8fb7640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd40cc0c0 0x7f0fd40c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.659+0000 7f0fd8fb7640 1 --2- 172.21.15.120:0/3631374894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fd40cb2f0 0x7f0fd40c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.659+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/3631374894 >> 172.21.15.120:0/3631374894 conn(0x7f0fd406d560 msgr2=0x7f0fd40d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.659+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/3631374894 shutdown_connections 2024-09-10T10:29:08.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.659+0000 7f0fd8fb7640 1 -- 172.21.15.120:0/3631374894 wait complete. 2024-09-10T10:29:08.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.661+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fad8c0d3b40 con 0x7fad8c04a7b0 2024-09-10T10:29:08.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.662+0000 7fad90ff9640 1 -- 172.21.15.120:0/1219452316 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fad8007a020 con 0x7fad8c04a7b0 2024-09-10T10:29:08.664 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.664+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fad7806be80 msgr2=0x7fad7806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.664+0000 7fad996ca640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fad7806be80 0x7fad7806e340 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad88000f60 tx=0x7fad88009040 comp rx=0 tx=0).stop 2024-09-10T10:29:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.664+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 msgr2=0x7fad8c17bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.664+0000 7fad996ca640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c17bc70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad80000c00 tx=0x7fad800046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.665+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 shutdown_connections 2024-09-10T10:29:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.665+0000 7fad996ca640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fad7806be80 0x7fad7806e340 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.665+0000 7fad996ca640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad8c0dcd00 0x7fad8c0d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.665+0000 7fad996ca640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad8c04a7b0 0x7fad8c17bc70 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad80000c00 tx=0x7fad800046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.665+0000 7fad996ca640 1 --2- 172.21.15.120:0/1219452316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad8c005cc0 0x7fad8c0d4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.665+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 >> 172.21.15.120:0/1219452316 conn(0x7fad8c04fe30 msgr2=0x7fad8c04e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.665+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 shutdown_connections 2024-09-10T10:29:08.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.665+0000 7fad996ca640 1 -- 172.21.15.120:0/1219452316 wait complete. 2024-09-10T10:29:08.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.5 2024-09-10T10:29:08.678 DEBUG:teuthology.parallel:result is None 2024-09-10T10:29:08.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.6 2024-09-10T10:29:08.680 DEBUG:teuthology.parallel:result is None 2024-09-10T10:29:08.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.2 2024-09-10T10:29:08.680 DEBUG:teuthology.parallel:result is None 2024-09-10T10:29:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.684+0000 7ff1cb801640 1 -- 172.21.15.120:0/3681876537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff1a4005580 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.684+0000 7ff1b9ffb640 1 -- 172.21.15.120:0/3681876537 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff1b405cf30 con 0x7ff1c40bf8b0 2024-09-10T10:29:08.685 INFO:teuthology.orchestra.run.smithi120.stdout:55834574883 2024-09-10T10:29:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.686+0000 7ff1cb801640 1 -- 172.21.15.120:0/3681876537 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1a806be30 msgr2=0x7ff1a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.686+0000 7ff1cb801640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1a806be30 0x7ff1a806e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40c4660 tx=0x7ff1c0008040 comp rx=0 tx=0).stop 2024-09-10T10:29:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.687+0000 7ff1cb801640 1 -- 172.21.15.120:0/3681876537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c40bf8b0 msgr2=0x7ff1c41eccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.687+0000 7ff1cb801640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c40bf8b0 0x7ff1c41eccd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c420 tx=0x7ff1b400c8e0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.687+0000 7ff1cb801640 1 -- 172.21.15.120:0/3681876537 shutdown_connections 2024-09-10T10:29:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.687+0000 7ff1cb801640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff1a806be30 0x7ff1a806e2f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.687+0000 7ff1cb801640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1c4155c50 0x7ff1c40c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.687+0000 7ff1cb801640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1c40c8640 0x7ff1c40c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.687+0000 7ff1cb801640 1 --2- 172.21.15.120:0/3681876537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c40bf8b0 0x7ff1c41eccd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.687+0000 7ff1cb801640 1 -- 172.21.15.120:0/3681876537 >> 172.21.15.120:0/3681876537 conn(0x7ff1c40cbf90 msgr2=0x7ff1c40c9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.687+0000 7ff1cb801640 1 -- 172.21.15.120:0/3681876537 shutdown_connections 2024-09-10T10:29:08.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.687+0000 7ff1cb801640 1 -- 172.21.15.120:0/3681876537 wait complete. 2024-09-10T10:29:08.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.692+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f17140da030 con 0x7f17140486e0 2024-09-10T10:29:08.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.692+0000 7f1718ff9640 1 -- 172.21.15.120:0/454152350 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f17140da030 con 0x7f17140486e0 2024-09-10T10:29:08.694 INFO:teuthology.orchestra.run.smithi120.stdout:55834574883 2024-09-10T10:29:08.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.693+0000 7f7a9657f640 1 -- 172.21.15.120:0/872006456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7a900cc840 con 0x7f7a900b3790 2024-09-10T10:29:08.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.693+0000 7f4a0b020640 1 -- 172.21.15.120:0/3627313531 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4a04071f20 con 0x7f4a04158b40 2024-09-10T10:29:08.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.694+0000 7f7a8cff9640 1 -- 172.21.15.120:0/872006456 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7a8805a890 con 0x7f7a900b3790 2024-09-10T10:29:08.695 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:08.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.694+0000 7f49faffd640 1 -- 172.21.15.120:0/3627313531 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f49fc059360 con 0x7f4a04158b40 2024-09-10T10:29:08.695 INFO:teuthology.orchestra.run.smithi120.stdout:55834574882 2024-09-10T10:29:08.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f170006be60 msgr2=0x7f170006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f170006be60 0x7f170006e320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f171000f240 tx=0x7f1710008040 comp rx=0 tx=0).stop 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17140486e0 msgr2=0x7f1714173a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17140486e0 0x7f1714173a70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f171c051790 tx=0x7f171c069890 comp rx=0 tx=0).stop 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 shutdown_connections 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f170006be60 0x7f170006e320 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17140486e0 0x7f1714173a70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17140d62e0 0x7f17140481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 --2- 172.21.15.120:0/454152350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1714044b30 0x7f17140cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 >> 172.21.15.120:0/454152350 conn(0x7f171404fc50 msgr2=0x7f17140d54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 shutdown_connections 2024-09-10T10:29:08.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.695+0000 7f1721504640 1 -- 172.21.15.120:0/454152350 wait complete. 2024-09-10T10:29:08.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 -- 172.21.15.120:0/872006456 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7a7006be80 msgr2=0x7f7a7006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7a7006be80 0x7f7a7006e340 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a9006cf70 tx=0x7f7a80007e80 comp rx=0 tx=0).stop 2024-09-10T10:29:08.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 -- 172.21.15.120:0/872006456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 msgr2=0x7f7a901f39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 0x7f7a901f39c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7a88007f20 tx=0x7f7a8800e570 comp rx=0 tx=0).stop 2024-09-10T10:29:08.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 -- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f49e806be30 msgr2=0x7f49e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f49e806be30 0x7f49e806e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c1c0 tx=0x7f4a00009030 comp rx=0 tx=0).stop 2024-09-10T10:29:08.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 -- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a04158b40 msgr2=0x7f4a04158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a04158b40 0x7f4a04158ff0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4a04070c60 tx=0x7f49fc00b410 comp rx=0 tx=0).stop 2024-09-10T10:29:08.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 -- 172.21.15.120:0/872006456 shutdown_connections 2024-09-10T10:29:08.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f7a7006be80 0x7f7a7006e340 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a900d0fd0 0x7f7a901f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a900d0bf0 0x7f7a901f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 --2- 172.21.15.120:0/872006456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a900b3790 0x7f7a901f39c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 -- 172.21.15.120:0/872006456 >> 172.21.15.120:0/872006456 conn(0x7f7a9006c900 msgr2=0x7f7a900c8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 -- 172.21.15.120:0/3627313531 shutdown_connections 2024-09-10T10:29:08.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f49e806be30 0x7f49e806e2f0 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c1c0 tx=0x7f4a00009030 comp rx=0 tx=0).stop 2024-09-10T10:29:08.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a04158b40 0x7f4a04158ff0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04150310 0x7f4a04153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 --2- 172.21.15.120:0/3627313531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49f0007dc0 0x7f4a04152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 -- 172.21.15.120:0/3627313531 >> 172.21.15.120:0/3627313531 conn(0x7f4a040cc2c0 msgr2=0x7f4a040cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 -- 172.21.15.120:0/872006456 shutdown_connections 2024-09-10T10:29:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 -- 172.21.15.120:0/3627313531 shutdown_connections 2024-09-10T10:29:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f7a9657f640 1 -- 172.21.15.120:0/872006456 wait complete. 2024-09-10T10:29:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.696+0000 7f4a0b020640 1 -- 172.21.15.120:0/3627313531 wait complete. 2024-09-10T10:29:08.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574883 for osd.7 2024-09-10T10:29:08.702 DEBUG:teuthology.parallel:result is None 2024-09-10T10:29:08.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.3 2024-09-10T10:29:08.712 DEBUG:teuthology.parallel:result is None 2024-09-10T10:29:08.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.1 2024-09-10T10:29:08.713 DEBUG:teuthology.parallel:result is None 2024-09-10T10:29:08.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574883 for osd.4 2024-09-10T10:29:08.713 DEBUG:teuthology.parallel:result is None 2024-09-10T10:29:08.713 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-10T10:29:08.714 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:29:08.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.850+0000 7f20d5e8b640 1 Processor -- start 2024-09-10T10:29:08.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.850+0000 7f20d5e8b640 1 -- start start 2024-09-10T10:29:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20d5e8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d0152ae0 0x7f20d0159170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20d5e8b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f20d00c5140 0x7f20d0163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20d5e8b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d015cc60 con 0x7f20d00c9580 2024-09-10T10:29:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20d5e8b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f20d01596b0 con 0x7f20d00c5140 2024-09-10T10:29:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20d5e8b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f20d0152ec0 con 0x7f20d0152ae0 2024-09-10T10:29:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20cf7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d0152ae0 0x7f20d0159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20cf7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d0152ae0 0x7f20d0159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:40642/0 (socket says 172.21.15.120:40642) 2024-09-10T10:29:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20cffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f20d00c5140 0x7f20d0163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20cf7fe640 1 -- 172.21.15.120:0/1350549611 learned_addr learned my addr 172.21.15.120:0/1350549611 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20ceffd640 1 --1- 172.21.15.120:0/1350549611 >> v1:172.21.15.134:6789/0 conn(0x7f20d00c9580 0x7f20d00c3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:50102/0 (socket says 172.21.15.120:50102) 2024-09-10T10:29:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.852+0000 7f20cf7fe640 1 -- 172.21.15.120:0/1350549611 >> v2:172.21.15.134:3300/0 conn(0x7f20d00c5140 msgr2=0x7f20d0163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.851+0000 7f20ce7fc640 1 -- 172.21.15.120:0/1350549611 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740966150 0 0) 0x7f20d015cc60 con 0x7f20d00c9580 2024-09-10T10:29:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.852+0000 7f20cf7fe640 1 --2- 172.21.15.120:0/1350549611 >> v2:172.21.15.134:3300/0 conn(0x7f20d00c5140 0x7f20d0163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.852+0000 7f20cf7fe640 1 -- 172.21.15.120:0/1350549611 >> v1:172.21.15.134:6789/0 conn(0x7f20d00c9580 legacy=0x7f20d00c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.852+0000 7f20cf7fe640 1 -- 172.21.15.120:0/1350549611 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d00c9a70 con 0x7f20d0152ae0 2024-09-10T10:29:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.852+0000 7f20cf7fe640 1 --2- 172.21.15.120:0/1350549611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d0152ae0 0x7f20d0159170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009870 tx=0x7f20c002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d22c85f5898824c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.852+0000 7f20ce7fc640 1 -- 172.21.15.120:0/1350549611 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0037070 con 0x7f20d0152ae0 2024-09-10T10:29:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.852+0000 7f20ce7fc640 1 -- 172.21.15.120:0/1350549611 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c0032440 con 0x7f20d0152ae0 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.852+0000 7f20ce7fc640 1 -- 172.21.15.120:0/1350549611 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0032810 con 0x7f20d0152ae0 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.853+0000 7f20d5e8b640 1 -- 172.21.15.120:0/1350549611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d0152ae0 msgr2=0x7f20d0159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.853+0000 7f20d5e8b640 1 --2- 172.21.15.120:0/1350549611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d0152ae0 0x7f20d0159170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009870 tx=0x7f20c002a270 comp rx=0 tx=0).stop 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.853+0000 7f20d5e8b640 1 -- 172.21.15.120:0/1350549611 shutdown_connections 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.853+0000 7f20d5e8b640 1 --2- 172.21.15.120:0/1350549611 >> v2:172.21.15.134:3300/0 conn(0x7f20d00c5140 0x7f20d0163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.853+0000 7f20d5e8b640 1 --2- 172.21.15.120:0/1350549611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d0152ae0 0x7f20d0159170 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.853+0000 7f20d5e8b640 1 -- 172.21.15.120:0/1350549611 >> 172.21.15.120:0/1350549611 conn(0x7f20d006bc10 msgr2=0x7f20d006fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.853+0000 7f20d5e8b640 1 -- 172.21.15.120:0/1350549611 shutdown_connections 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.853+0000 7f20d5e8b640 1 -- 172.21.15.120:0/1350549611 wait complete. 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.854+0000 7f20d5e8b640 1 Processor -- start 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.854+0000 7f20d5e8b640 1 -- start start 2024-09-10T10:29:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20d5e8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20d00c5140 0x7f20d014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20d5e8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d00c9580 0x7f20d014e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20d5e8b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d0152ae0 0x7f20d014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20d5e8b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0158970 con 0x7f20d0152ae0 2024-09-10T10:29:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20d5e8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20d00ca030 con 0x7f20d00c5140 2024-09-10T10:29:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20d5e8b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f20d0158c50 con 0x7f20d00c9580 2024-09-10T10:29:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d0152ae0 0x7f20d014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d0152ae0 0x7f20d014eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42382/0 (socket says 172.21.15.120:42382) 2024-09-10T10:29:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20cffff640 1 -- 172.21.15.120:0/2442038104 learned_addr learned my addr 172.21.15.120:0/2442038104 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20cffff640 1 -- 172.21.15.120:0/2442038104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d00c9580 msgr2=0x7f20d014e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20cffff640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d00c9580 0x7f20d014e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20cffff640 1 -- 172.21.15.120:0/2442038104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20d00c5140 msgr2=0x7f20d014db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20cffff640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20d00c5140 0x7f20d014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.855+0000 7f20cffff640 1 -- 172.21.15.120:0/2442038104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d014cab0 con 0x7f20d0152ae0 2024-09-10T10:29:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.856+0000 7f20cffff640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d0152ae0 0x7f20d014eef0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f20c800ec10 tx=0x7f20c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.856+0000 7f20b7fff640 1 -- 172.21.15.120:0/2442038104 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8018020 con 0x7f20d0152ae0 2024-09-10T10:29:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.856+0000 7f20b7fff640 1 -- 172.21.15.120:0/2442038104 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c800cd20 con 0x7f20d0152ae0 2024-09-10T10:29:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.856+0000 7f20d5e8b640 1 -- 172.21.15.120:0/2442038104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d014cd90 con 0x7f20d0152ae0 2024-09-10T10:29:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.856+0000 7f20b7fff640 1 -- 172.21.15.120:0/2442038104 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c80146c0 con 0x7f20d0152ae0 2024-09-10T10:29:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.856+0000 7f20d5e8b640 1 -- 172.21.15.120:0/2442038104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d014d350 con 0x7f20d0152ae0 2024-09-10T10:29:08.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.857+0000 7f20d5e8b640 1 -- 172.21.15.120:0/2442038104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d00c6ee0 con 0x7f20d0152ae0 2024-09-10T10:29:08.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.857+0000 7f20b7fff640 1 -- 172.21.15.120:0/2442038104 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20c8005d10 con 0x7f20d0152ae0 2024-09-10T10:29:08.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.858+0000 7f20b7fff640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f20bc06be30 0x7f20bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:08.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.858+0000 7f20b7fff640 1 -- 172.21.15.120:0/2442038104 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f20c808c320 con 0x7f20d0152ae0 2024-09-10T10:29:08.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.858+0000 7f20cf7fe640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f20bc06be30 0x7f20bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:08.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.858+0000 7f20cf7fe640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f20bc06be30 0x7f20bc06e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20c0034000 tx=0x7f20c002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:08.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:08.862+0000 7f20b7fff640 1 -- 172.21.15.120:0/2442038104 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f20d00c6ee0 con 0x7f20d0152ae0 2024-09-10T10:29:09.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.025+0000 7f20d5e8b640 1 -- 172.21.15.120:0/2442038104 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f20d0156a90 con 0x7f20bc06be30 2024-09-10T10:29:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.033+0000 7f20b7fff640 1 -- 172.21.15.120:0/2442038104 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332945 (secure 0 0 0) 0x7f20d0156a90 con 0x7f20bc06be30 2024-09-10T10:29:09.038 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:29:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 -- 172.21.15.120:0/2442038104 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f20bc06be30 msgr2=0x7f20bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f20bc06be30 0x7f20bc06e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20c0034000 tx=0x7f20c002c000 comp rx=0 tx=0).stop 2024-09-10T10:29:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 -- 172.21.15.120:0/2442038104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d0152ae0 msgr2=0x7f20d014eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d0152ae0 0x7f20d014eef0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f20c800ec10 tx=0x7f20c800c490 comp rx=0 tx=0).stop 2024-09-10T10:29:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 -- 172.21.15.120:0/2442038104 shutdown_connections 2024-09-10T10:29:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f20bc06be30 0x7f20bc06e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20d0152ae0 0x7f20d014eef0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20d00c9580 0x7f20d014e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 --2- 172.21.15.120:0/2442038104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20d00c5140 0x7f20d014db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 -- 172.21.15.120:0/2442038104 >> 172.21.15.120:0/2442038104 conn(0x7f20d006bc10 msgr2=0x7f20d00ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 -- 172.21.15.120:0/2442038104 shutdown_connections 2024-09-10T10:29:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.036+0000 7f20d5e8b640 1 -- 172.21.15.120:0/2442038104 wait complete. 2024-09-10T10:29:09.048 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-10T10:29:08.120056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634472,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":119,"num_read_kb":103,"num_write":408,"num_write_kb":2539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227948,"kb_used_data":7544,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754746772,"statfs":{"total":773094113280,"available":772860694528,"internally_reserved":0,"allocated":7725056,"data_stored":5078956,"data_compressed":68176,"data_compressed_allocated":98304,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.662305"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893394+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:29:05.893394+0000","last_peered":"2024-09-10T10:29:05.893394+0000","last_clean":"2024-09-10T10:29:05.893394+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:29:05.893394+0000","last_undegraded":"2024-09-10T10:29:05.893394+0000","last_fullsized":"2024-09-10T10:29:05.893394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831510+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:29:05.831510+0000","last_peered":"2024-09-10T10:29:05.831510+0000","last_clean":"2024-09-10T10:29:05.831510+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:29:05.831510+0000","last_undegraded":"2024-09-10T10:29:05.831510+0000","last_fullsized":"2024-09-10T10:29:05.831510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831742+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:29:05.831742+0000","last_peered":"2024-09-10T10:29:05.831742+0000","last_clean":"2024-09-10T10:29:05.831742+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:29:05.831742+0000","last_undegraded":"2024-09-10T10:29:05.831742+0000","last_fullsized":"2024-09-10T10:29:05.831742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832208+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:29:05.832208+0000","last_peered":"2024-09-10T10:29:05.832208+0000","last_clean":"2024-09-10T10:29:05.832208+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:29:05.832208+0000","last_undegraded":"2024-09-10T10:29:05.832208+0000","last_fullsized":"2024-09-10T10:29:05.832208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892600+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:29:05.892600+0000","last_peered":"2024-09-10T10:29:05.892600+0000","last_clean":"2024-09-10T10:29:05.892600+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:29:05.892600+0000","last_undegraded":"2024-09-10T10:29:05.892600+0000","last_fullsized":"2024-09-10T10:29:05.892600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955501+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:29:05.955501+0000","last_peered":"2024-09-10T10:29:05.955501+0000","last_clean":"2024-09-10T10:29:05.955501+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:29:05.955501+0000","last_undegraded":"2024-09-10T10:29:05.955501+0000","last_fullsized":"2024-09-10T10:29:05.955501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"60'13","reported_seq":158,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521178+0000","last_change":"2024-09-10T10:27:51.060242+0000","last_active":"2024-09-10T10:29:05.521178+0000","last_peered":"2024-09-10T10:29:05.521178+0000","last_clean":"2024-09-10T10:29:05.521178+0000","last_became_active":"2024-09-10T10:27:51.049148+0000","last_became_peered":"2024-09-10T10:27:51.049148+0000","last_unstale":"2024-09-10T10:29:05.521178+0000","last_undegraded":"2024-09-10T10:29:05.521178+0000","last_fullsized":"2024-09-10T10:29:05.521178+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.604144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486386+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:29:05.486386+0000","last_peered":"2024-09-10T10:29:05.486386+0000","last_clean":"2024-09-10T10:29:05.486386+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:29:05.486386+0000","last_undegraded":"2024-09-10T10:29:05.486386+0000","last_fullsized":"2024-09-10T10:29:05.486386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955770+0000","last_change":"2024-09-10T10:28:24.244697+0000","last_active":"2024-09-10T10:29:05.955770+0000","last_peered":"2024-09-10T10:29:05.955770+0000","last_clean":"2024-09-10T10:29:05.955770+0000","last_became_active":"2024-09-10T10:28:24.244262+0000","last_became_peered":"2024-09-10T10:28:24.244262+0000","last_unstale":"2024-09-10T10:29:05.955770+0000","last_undegraded":"2024-09-10T10:29:05.955770+0000","last_fullsized":"2024-09-10T10:29:05.955770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:50.683186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6,2],"acting":[4,0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520262+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:29:05.520262+0000","last_peered":"2024-09-10T10:29:05.520262+0000","last_clean":"2024-09-10T10:29:05.520262+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:29:05.520262+0000","last_undegraded":"2024-09-10T10:29:05.520262+0000","last_fullsized":"2024-09-10T10:29:05.520262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":110,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482709+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:29:05.482709+0000","last_peered":"2024-09-10T10:29:05.482709+0000","last_clean":"2024-09-10T10:29:05.482709+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:29:05.482709+0000","last_undegraded":"2024-09-10T10:29:05.482709+0000","last_fullsized":"2024-09-10T10:29:05.482709+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955722+0000","last_change":"2024-09-10T10:28:59.443607+0000","last_active":"2024-09-10T10:29:05.955722+0000","last_peered":"2024-09-10T10:29:05.955722+0000","last_clean":"2024-09-10T10:29:05.955722+0000","last_became_active":"2024-09-10T10:28:59.443172+0000","last_became_peered":"2024-09-10T10:28:59.443172+0000","last_unstale":"2024-09-10T10:29:05.955722+0000","last_undegraded":"2024-09-10T10:29:05.955722+0000","last_fullsized":"2024-09-10T10:29:05.955722+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:36.379163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485088+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:29:05.485088+0000","last_peered":"2024-09-10T10:29:05.485088+0000","last_clean":"2024-09-10T10:29:05.485088+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:29:05.485088+0000","last_undegraded":"2024-09-10T10:29:05.485088+0000","last_fullsized":"2024-09-10T10:29:05.485088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"60'5","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520220+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:29:05.520220+0000","last_peered":"2024-09-10T10:29:05.520220+0000","last_clean":"2024-09-10T10:29:05.520220+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:29:05.520220+0000","last_undegraded":"2024-09-10T10:29:05.520220+0000","last_fullsized":"2024-09-10T10:29:05.520220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956119+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:29:05.956119+0000","last_peered":"2024-09-10T10:29:05.956119+0000","last_clean":"2024-09-10T10:29:05.956119+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:29:05.956119+0000","last_undegraded":"2024-09-10T10:29:05.956119+0000","last_fullsized":"2024-09-10T10:29:05.956119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831889+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:29:05.831889+0000","last_peered":"2024-09-10T10:29:05.831889+0000","last_clean":"2024-09-10T10:29:05.831889+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:29:05.831889+0000","last_undegraded":"2024-09-10T10:29:05.831889+0000","last_fullsized":"2024-09-10T10:29:05.831889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688805+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:29:05.688805+0000","last_peered":"2024-09-10T10:29:05.688805+0000","last_clean":"2024-09-10T10:29:05.688805+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:29:05.688805+0000","last_undegraded":"2024-09-10T10:29:05.688805+0000","last_fullsized":"2024-09-10T10:29:05.688805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955843+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:29:05.955843+0000","last_peered":"2024-09-10T10:29:05.955843+0000","last_clean":"2024-09-10T10:29:05.955843+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:29:05.955843+0000","last_undegraded":"2024-09-10T10:29:05.955843+0000","last_fullsized":"2024-09-10T10:29:05.955843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831383+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:29:05.831383+0000","last_peered":"2024-09-10T10:29:05.831383+0000","last_clean":"2024-09-10T10:29:05.831383+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:29:05.831383+0000","last_undegraded":"2024-09-10T10:29:05.831383+0000","last_fullsized":"2024-09-10T10:29:05.831383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688223+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:29:05.688223+0000","last_peered":"2024-09-10T10:29:05.688223+0000","last_clean":"2024-09-10T10:29:05.688223+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:29:05.688223+0000","last_undegraded":"2024-09-10T10:29:05.688223+0000","last_fullsized":"2024-09-10T10:29:05.688223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688656+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:29:05.688656+0000","last_peered":"2024-09-10T10:29:05.688656+0000","last_clean":"2024-09-10T10:29:05.688656+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:29:05.688656+0000","last_undegraded":"2024-09-10T10:29:05.688656+0000","last_fullsized":"2024-09-10T10:29:05.688656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955038+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:29:05.955038+0000","last_peered":"2024-09-10T10:29:05.955038+0000","last_clean":"2024-09-10T10:29:05.955038+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:29:05.955038+0000","last_undegraded":"2024-09-10T10:29:05.955038+0000","last_fullsized":"2024-09-10T10:29:05.955038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520435+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:29:05.520435+0000","last_peered":"2024-09-10T10:29:05.520435+0000","last_clean":"2024-09-10T10:29:05.520435+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:29:05.520435+0000","last_undegraded":"2024-09-10T10:29:05.520435+0000","last_fullsized":"2024-09-10T10:29:05.520435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956141+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:29:05.956141+0000","last_peered":"2024-09-10T10:29:05.956141+0000","last_clean":"2024-09-10T10:29:05.956141+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:29:05.956141+0000","last_undegraded":"2024-09-10T10:29:05.956141+0000","last_fullsized":"2024-09-10T10:29:05.956141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892494+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:29:05.892494+0000","last_peered":"2024-09-10T10:29:05.892494+0000","last_clean":"2024-09-10T10:29:05.892494+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:29:05.892494+0000","last_undegraded":"2024-09-10T10:29:05.892494+0000","last_fullsized":"2024-09-10T10:29:05.892494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'8","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485743+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:29:05.485743+0000","last_peered":"2024-09-10T10:29:05.485743+0000","last_clean":"2024-09-10T10:29:05.485743+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:29:05.485743+0000","last_undegraded":"2024-09-10T10:29:05.485743+0000","last_fullsized":"2024-09-10T10:29:05.485743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955720+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:29:05.955720+0000","last_peered":"2024-09-10T10:29:05.955720+0000","last_clean":"2024-09-10T10:29:05.955720+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:29:05.955720+0000","last_undegraded":"2024-09-10T10:29:05.955720+0000","last_fullsized":"2024-09-10T10:29:05.955720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485344+0000","last_change":"2024-09-10T10:28:24.374205+0000","last_active":"2024-09-10T10:29:05.485344+0000","last_peered":"2024-09-10T10:29:05.485344+0000","last_clean":"2024-09-10T10:29:05.485344+0000","last_became_active":"2024-09-10T10:28:24.373921+0000","last_became_peered":"2024-09-10T10:28:24.373921+0000","last_unstale":"2024-09-10T10:29:05.485344+0000","last_undegraded":"2024-09-10T10:29:05.485344+0000","last_fullsized":"2024-09-10T10:29:05.485344+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:36.923293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"64'15","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688969+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:29:05.688969+0000","last_peered":"2024-09-10T10:29:05.688969+0000","last_clean":"2024-09-10T10:29:05.688969+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:29:05.688969+0000","last_undegraded":"2024-09-10T10:29:05.688969+0000","last_fullsized":"2024-09-10T10:29:05.688969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831546+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:29:05.831546+0000","last_peered":"2024-09-10T10:29:05.831546+0000","last_clean":"2024-09-10T10:29:05.831546+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:29:05.831546+0000","last_undegraded":"2024-09-10T10:29:05.831546+0000","last_fullsized":"2024-09-10T10:29:05.831546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688907+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:29:05.688907+0000","last_peered":"2024-09-10T10:29:05.688907+0000","last_clean":"2024-09-10T10:29:05.688907+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:29:05.688907+0000","last_undegraded":"2024-09-10T10:29:05.688907+0000","last_fullsized":"2024-09-10T10:29:05.688907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521226+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:29:05.521226+0000","last_peered":"2024-09-10T10:29:05.521226+0000","last_clean":"2024-09-10T10:29:05.521226+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:29:05.521226+0000","last_undegraded":"2024-09-10T10:29:05.521226+0000","last_fullsized":"2024-09-10T10:29:05.521226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893084+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:29:05.893084+0000","last_peered":"2024-09-10T10:29:05.893084+0000","last_clean":"2024-09-10T10:29:05.893084+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:29:05.893084+0000","last_undegraded":"2024-09-10T10:29:05.893084+0000","last_fullsized":"2024-09-10T10:29:05.893084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486566+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:29:05.486566+0000","last_peered":"2024-09-10T10:29:05.486566+0000","last_clean":"2024-09-10T10:29:05.486566+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:29:05.486566+0000","last_undegraded":"2024-09-10T10:29:05.486566+0000","last_fullsized":"2024-09-10T10:29:05.486566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688951+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:29:05.688951+0000","last_peered":"2024-09-10T10:29:05.688951+0000","last_clean":"2024-09-10T10:29:05.688951+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.688951+0000","last_undegraded":"2024-09-10T10:29:05.688951+0000","last_fullsized":"2024-09-10T10:29:05.688951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688451+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:29:05.688451+0000","last_peered":"2024-09-10T10:29:05.688451+0000","last_clean":"2024-09-10T10:29:05.688451+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:29:05.688451+0000","last_undegraded":"2024-09-10T10:29:05.688451+0000","last_fullsized":"2024-09-10T10:29:05.688451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956082+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:29:05.956082+0000","last_peered":"2024-09-10T10:29:05.956082+0000","last_clean":"2024-09-10T10:29:05.956082+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:29:05.956082+0000","last_undegraded":"2024-09-10T10:29:05.956082+0000","last_fullsized":"2024-09-10T10:29:05.956082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486359+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:29:05.486359+0000","last_peered":"2024-09-10T10:29:05.486359+0000","last_clean":"2024-09-10T10:29:05.486359+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:29:05.486359+0000","last_undegraded":"2024-09-10T10:29:05.486359+0000","last_fullsized":"2024-09-10T10:29:05.486359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486083+0000","last_change":"2024-09-10T10:28:24.244988+0000","last_active":"2024-09-10T10:29:05.486083+0000","last_peered":"2024-09-10T10:29:05.486083+0000","last_clean":"2024-09-10T10:29:05.486083+0000","last_became_active":"2024-09-10T10:28:24.244556+0000","last_became_peered":"2024-09-10T10:28:24.244556+0000","last_unstale":"2024-09-10T10:29:05.486083+0000","last_undegraded":"2024-09-10T10:29:05.486083+0000","last_fullsized":"2024-09-10T10:29:05.486083+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:08.127492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688685+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:29:05.688685+0000","last_peered":"2024-09-10T10:29:05.688685+0000","last_clean":"2024-09-10T10:29:05.688685+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:29:05.688685+0000","last_undegraded":"2024-09-10T10:29:05.688685+0000","last_fullsized":"2024-09-10T10:29:05.688685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'9","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482578+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:29:05.482578+0000","last_peered":"2024-09-10T10:29:05.482578+0000","last_clean":"2024-09-10T10:29:05.482578+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.482578+0000","last_undegraded":"2024-09-10T10:29:05.482578+0000","last_fullsized":"2024-09-10T10:29:05.482578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688718+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:29:05.688718+0000","last_peered":"2024-09-10T10:29:05.688718+0000","last_clean":"2024-09-10T10:29:05.688718+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:29:05.688718+0000","last_undegraded":"2024-09-10T10:29:05.688718+0000","last_fullsized":"2024-09-10T10:29:05.688718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892835+0000","last_change":"2024-09-10T10:28:59.446889+0000","last_active":"2024-09-10T10:29:05.892835+0000","last_peered":"2024-09-10T10:29:05.892835+0000","last_clean":"2024-09-10T10:29:05.892835+0000","last_became_active":"2024-09-10T10:28:59.446544+0000","last_became_peered":"2024-09-10T10:28:59.446544+0000","last_unstale":"2024-09-10T10:29:05.892835+0000","last_undegraded":"2024-09-10T10:29:05.892835+0000","last_fullsized":"2024-09-10T10:29:05.892835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:32.352466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520235+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:29:05.520235+0000","last_peered":"2024-09-10T10:29:05.520235+0000","last_clean":"2024-09-10T10:29:05.520235+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:29:05.520235+0000","last_undegraded":"2024-09-10T10:29:05.520235+0000","last_fullsized":"2024-09-10T10:29:05.520235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893277+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:29:05.893277+0000","last_peered":"2024-09-10T10:29:05.893277+0000","last_clean":"2024-09-10T10:29:05.893277+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:29:05.893277+0000","last_undegraded":"2024-09-10T10:29:05.893277+0000","last_fullsized":"2024-09-10T10:29:05.893277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831398+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:29:05.831398+0000","last_peered":"2024-09-10T10:29:05.831398+0000","last_clean":"2024-09-10T10:29:05.831398+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:29:05.831398+0000","last_undegraded":"2024-09-10T10:29:05.831398+0000","last_fullsized":"2024-09-10T10:29:05.831398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486160+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:29:05.486160+0000","last_peered":"2024-09-10T10:29:05.486160+0000","last_clean":"2024-09-10T10:29:05.486160+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:29:05.486160+0000","last_undegraded":"2024-09-10T10:29:05.486160+0000","last_fullsized":"2024-09-10T10:29:05.486160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":281,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485314+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:29:05.485314+0000","last_peered":"2024-09-10T10:29:05.485314+0000","last_clean":"2024-09-10T10:29:05.485314+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:29:05.485314+0000","last_undegraded":"2024-09-10T10:29:05.485314+0000","last_fullsized":"2024-09-10T10:29:05.485314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521019+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:29:05.521019+0000","last_peered":"2024-09-10T10:29:05.521019+0000","last_clean":"2024-09-10T10:29:05.521019+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:29:05.521019+0000","last_undegraded":"2024-09-10T10:29:05.521019+0000","last_fullsized":"2024-09-10T10:29:05.521019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482817+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:29:05.482817+0000","last_peered":"2024-09-10T10:29:05.482817+0000","last_clean":"2024-09-10T10:29:05.482817+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:29:05.482817+0000","last_undegraded":"2024-09-10T10:29:05.482817+0000","last_fullsized":"2024-09-10T10:29:05.482817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484865+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:29:05.484865+0000","last_peered":"2024-09-10T10:29:05.484865+0000","last_clean":"2024-09-10T10:29:05.484865+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:29:05.484865+0000","last_undegraded":"2024-09-10T10:29:05.484865+0000","last_fullsized":"2024-09-10T10:29:05.484865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486504+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:29:05.486504+0000","last_peered":"2024-09-10T10:29:05.486504+0000","last_clean":"2024-09-10T10:29:05.486504+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:29:05.486504+0000","last_undegraded":"2024-09-10T10:29:05.486504+0000","last_fullsized":"2024-09-10T10:29:05.486504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688628+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:29:05.688628+0000","last_peered":"2024-09-10T10:29:05.688628+0000","last_clean":"2024-09-10T10:29:05.688628+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:29:05.688628+0000","last_undegraded":"2024-09-10T10:29:05.688628+0000","last_fullsized":"2024-09-10T10:29:05.688628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893128+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:29:05.893128+0000","last_peered":"2024-09-10T10:29:05.893128+0000","last_clean":"2024-09-10T10:29:05.893128+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:29:05.893128+0000","last_undegraded":"2024-09-10T10:29:05.893128+0000","last_fullsized":"2024-09-10T10:29:05.893128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521367+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:29:05.521367+0000","last_peered":"2024-09-10T10:29:05.521367+0000","last_clean":"2024-09-10T10:29:05.521367+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:29:05.521367+0000","last_undegraded":"2024-09-10T10:29:05.521367+0000","last_fullsized":"2024-09-10T10:29:05.521367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483041+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:29:05.483041+0000","last_peered":"2024-09-10T10:29:05.483041+0000","last_clean":"2024-09-10T10:29:05.483041+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:29:05.483041+0000","last_undegraded":"2024-09-10T10:29:05.483041+0000","last_fullsized":"2024-09-10T10:29:05.483041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485530+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:29:05.485530+0000","last_peered":"2024-09-10T10:29:05.485530+0000","last_clean":"2024-09-10T10:29:05.485530+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:29:05.485530+0000","last_undegraded":"2024-09-10T10:29:05.485530+0000","last_fullsized":"2024-09-10T10:29:05.485530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521232+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:29:05.521232+0000","last_peered":"2024-09-10T10:29:05.521232+0000","last_clean":"2024-09-10T10:29:05.521232+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:29:05.521232+0000","last_undegraded":"2024-09-10T10:29:05.521232+0000","last_fullsized":"2024-09-10T10:29:05.521232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956264+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:29:05.956264+0000","last_peered":"2024-09-10T10:29:05.956264+0000","last_clean":"2024-09-10T10:29:05.956264+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:29:05.956264+0000","last_undegraded":"2024-09-10T10:29:05.956264+0000","last_fullsized":"2024-09-10T10:29:05.956264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955704+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:29:05.955704+0000","last_peered":"2024-09-10T10:29:05.955704+0000","last_clean":"2024-09-10T10:29:05.955704+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:29:05.955704+0000","last_undegraded":"2024-09-10T10:29:05.955704+0000","last_fullsized":"2024-09-10T10:29:05.955704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'18","reported_seq":147,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482470+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:29:05.482470+0000","last_peered":"2024-09-10T10:29:05.482470+0000","last_clean":"2024-09-10T10:29:05.482470+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:29:05.482470+0000","last_undegraded":"2024-09-10T10:29:05.482470+0000","last_fullsized":"2024-09-10T10:29:05.482470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893225+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:29:05.893225+0000","last_peered":"2024-09-10T10:29:05.893225+0000","last_clean":"2024-09-10T10:29:05.893225+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:29:05.893225+0000","last_undegraded":"2024-09-10T10:29:05.893225+0000","last_fullsized":"2024-09-10T10:29:05.893225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":160,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:28:59.453146+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:28:59.446453+0000","last_became_peered":"2024-09-10T10:28:59.446453+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:31.347510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1176,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832175+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:29:05.832175+0000","last_peered":"2024-09-10T10:29:05.832175+0000","last_clean":"2024-09-10T10:29:05.832175+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:29:05.832175+0000","last_undegraded":"2024-09-10T10:29:05.832175+0000","last_fullsized":"2024-09-10T10:29:05.832175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520795+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:29:05.520795+0000","last_peered":"2024-09-10T10:29:05.520795+0000","last_clean":"2024-09-10T10:29:05.520795+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:29:05.520795+0000","last_undegraded":"2024-09-10T10:29:05.520795+0000","last_fullsized":"2024-09-10T10:29:05.520795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"60'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486347+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:29:05.486347+0000","last_peered":"2024-09-10T10:29:05.486347+0000","last_clean":"2024-09-10T10:29:05.486347+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:29:05.486347+0000","last_undegraded":"2024-09-10T10:29:05.486347+0000","last_fullsized":"2024-09-10T10:29:05.486347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832184+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:29:05.832184+0000","last_peered":"2024-09-10T10:29:05.832184+0000","last_clean":"2024-09-10T10:29:05.832184+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:29:05.832184+0000","last_undegraded":"2024-09-10T10:29:05.832184+0000","last_fullsized":"2024-09-10T10:29:05.832184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892463+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:29:05.892463+0000","last_peered":"2024-09-10T10:29:05.892463+0000","last_clean":"2024-09-10T10:29:05.892463+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:29:05.892463+0000","last_undegraded":"2024-09-10T10:29:05.892463+0000","last_fullsized":"2024-09-10T10:29:05.892463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893348+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:29:05.893348+0000","last_peered":"2024-09-10T10:29:05.893348+0000","last_clean":"2024-09-10T10:29:05.893348+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:29:05.893348+0000","last_undegraded":"2024-09-10T10:29:05.893348+0000","last_fullsized":"2024-09-10T10:29:05.893348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486563+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:29:05.486563+0000","last_peered":"2024-09-10T10:29:05.486563+0000","last_clean":"2024-09-10T10:29:05.486563+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:29:05.486563+0000","last_undegraded":"2024-09-10T10:29:05.486563+0000","last_fullsized":"2024-09-10T10:29:05.486563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520992+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:29:05.520992+0000","last_peered":"2024-09-10T10:29:05.520992+0000","last_clean":"2024-09-10T10:29:05.520992+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:29:05.520992+0000","last_undegraded":"2024-09-10T10:29:05.520992+0000","last_fullsized":"2024-09-10T10:29:05.520992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"69'27","reported_seq":156,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485568+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:29:05.485568+0000","last_peered":"2024-09-10T10:29:05.485568+0000","last_clean":"2024-09-10T10:29:05.485568+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:29:05.485568+0000","last_undegraded":"2024-09-10T10:29:05.485568+0000","last_fullsized":"2024-09-10T10:29:05.485568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37014,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"60'3","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688649+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:29:05.688649+0000","last_peered":"2024-09-10T10:29:05.688649+0000","last_clean":"2024-09-10T10:29:05.688649+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:29:05.688649+0000","last_undegraded":"2024-09-10T10:29:05.688649+0000","last_fullsized":"2024-09-10T10:29:05.688649+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955123+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:29:05.955123+0000","last_peered":"2024-09-10T10:29:05.955123+0000","last_clean":"2024-09-10T10:29:05.955123+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:29:05.955123+0000","last_undegraded":"2024-09-10T10:29:05.955123+0000","last_fullsized":"2024-09-10T10:29:05.955123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'8","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688658+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:29:05.688658+0000","last_peered":"2024-09-10T10:29:05.688658+0000","last_clean":"2024-09-10T10:29:05.688658+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:29:05.688658+0000","last_undegraded":"2024-09-10T10:29:05.688658+0000","last_fullsized":"2024-09-10T10:29:05.688658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955890+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:29:05.955890+0000","last_peered":"2024-09-10T10:29:05.955890+0000","last_clean":"2024-09-10T10:29:05.955890+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:29:05.955890+0000","last_undegraded":"2024-09-10T10:29:05.955890+0000","last_fullsized":"2024-09-10T10:29:05.955890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892512+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:29:05.892512+0000","last_peered":"2024-09-10T10:29:05.892512+0000","last_clean":"2024-09-10T10:29:05.892512+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:29:05.892512+0000","last_undegraded":"2024-09-10T10:29:05.892512+0000","last_fullsized":"2024-09-10T10:29:05.892512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486525+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:29:05.486525+0000","last_peered":"2024-09-10T10:29:05.486525+0000","last_clean":"2024-09-10T10:29:05.486525+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:29:05.486525+0000","last_undegraded":"2024-09-10T10:29:05.486525+0000","last_fullsized":"2024-09-10T10:29:05.486525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831619+0000","last_change":"2024-09-10T10:28:24.248827+0000","last_active":"2024-09-10T10:29:05.831619+0000","last_peered":"2024-09-10T10:29:05.831619+0000","last_clean":"2024-09-10T10:29:05.831619+0000","last_became_active":"2024-09-10T10:28:24.248395+0000","last_became_peered":"2024-09-10T10:28:24.248395+0000","last_unstale":"2024-09-10T10:29:05.831619+0000","last_undegraded":"2024-09-10T10:29:05.831619+0000","last_fullsized":"2024-09-10T10:29:05.831619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:05.306669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":106,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688561+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:29:05.688561+0000","last_peered":"2024-09-10T10:29:05.688561+0000","last_clean":"2024-09-10T10:29:05.688561+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:29:05.688561+0000","last_undegraded":"2024-09-10T10:29:05.688561+0000","last_fullsized":"2024-09-10T10:29:05.688561+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"60'6","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485563+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:29:05.485563+0000","last_peered":"2024-09-10T10:29:05.485563+0000","last_clean":"2024-09-10T10:29:05.485563+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:29:05.485563+0000","last_undegraded":"2024-09-10T10:29:05.485563+0000","last_fullsized":"2024-09-10T10:29:05.485563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831826+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:29:05.831826+0000","last_peered":"2024-09-10T10:29:05.831826+0000","last_clean":"2024-09-10T10:29:05.831826+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:29:05.831826+0000","last_undegraded":"2024-09-10T10:29:05.831826+0000","last_fullsized":"2024-09-10T10:29:05.831826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893171+0000","last_change":"2024-09-10T10:28:24.376473+0000","last_active":"2024-09-10T10:29:05.893171+0000","last_peered":"2024-09-10T10:29:05.893171+0000","last_clean":"2024-09-10T10:29:05.893171+0000","last_became_active":"2024-09-10T10:28:24.376079+0000","last_became_peered":"2024-09-10T10:28:24.376079+0000","last_unstale":"2024-09-10T10:29:05.893171+0000","last_undegraded":"2024-09-10T10:29:05.893171+0000","last_fullsized":"2024-09-10T10:29:05.893171+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:53.600117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,1],"acting":[0,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688345+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:29:05.688345+0000","last_peered":"2024-09-10T10:29:05.688345+0000","last_clean":"2024-09-10T10:29:05.688345+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:29:05.688345+0000","last_undegraded":"2024-09-10T10:29:05.688345+0000","last_fullsized":"2024-09-10T10:29:05.688345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485919+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:29:05.485919+0000","last_peered":"2024-09-10T10:29:05.485919+0000","last_clean":"2024-09-10T10:29:05.485919+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:29:05.485919+0000","last_undegraded":"2024-09-10T10:29:05.485919+0000","last_fullsized":"2024-09-10T10:29:05.485919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485377+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:29:05.485377+0000","last_peered":"2024-09-10T10:29:05.485377+0000","last_clean":"2024-09-10T10:29:05.485377+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:29:05.485377+0000","last_undegraded":"2024-09-10T10:29:05.485377+0000","last_fullsized":"2024-09-10T10:29:05.485377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831665+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:29:05.831665+0000","last_peered":"2024-09-10T10:29:05.831665+0000","last_clean":"2024-09-10T10:29:05.831665+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:29:05.831665+0000","last_undegraded":"2024-09-10T10:29:05.831665+0000","last_fullsized":"2024-09-10T10:29:05.831665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832255+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:29:05.832255+0000","last_peered":"2024-09-10T10:29:05.832255+0000","last_clean":"2024-09-10T10:29:05.832255+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:29:05.832255+0000","last_undegraded":"2024-09-10T10:29:05.832255+0000","last_fullsized":"2024-09-10T10:29:05.832255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485185+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:29:05.485185+0000","last_peered":"2024-09-10T10:29:05.485185+0000","last_clean":"2024-09-10T10:29:05.485185+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:29:05.485185+0000","last_undegraded":"2024-09-10T10:29:05.485185+0000","last_fullsized":"2024-09-10T10:29:05.485185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893219+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:29:05.893219+0000","last_peered":"2024-09-10T10:29:05.893219+0000","last_clean":"2024-09-10T10:29:05.893219+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:29:05.893219+0000","last_undegraded":"2024-09-10T10:29:05.893219+0000","last_fullsized":"2024-09-10T10:29:05.893219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482644+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:29:05.482644+0000","last_peered":"2024-09-10T10:29:05.482644+0000","last_clean":"2024-09-10T10:29:05.482644+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:29:05.482644+0000","last_undegraded":"2024-09-10T10:29:05.482644+0000","last_fullsized":"2024-09-10T10:29:05.482644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'6","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831961+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:29:05.831961+0000","last_peered":"2024-09-10T10:29:05.831961+0000","last_clean":"2024-09-10T10:29:05.831961+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:29:05.831961+0000","last_undegraded":"2024-09-10T10:29:05.831961+0000","last_fullsized":"2024-09-10T10:29:05.831961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955040+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:29:05.955040+0000","last_peered":"2024-09-10T10:29:05.955040+0000","last_clean":"2024-09-10T10:29:05.955040+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:29:05.955040+0000","last_undegraded":"2024-09-10T10:29:05.955040+0000","last_fullsized":"2024-09-10T10:29:05.955040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832152+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:29:05.832152+0000","last_peered":"2024-09-10T10:29:05.832152+0000","last_clean":"2024-09-10T10:29:05.832152+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:29:05.832152+0000","last_undegraded":"2024-09-10T10:29:05.832152+0000","last_fullsized":"2024-09-10T10:29:05.832152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484806+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:29:05.484806+0000","last_peered":"2024-09-10T10:29:05.484806+0000","last_clean":"2024-09-10T10:29:05.484806+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:29:05.484806+0000","last_undegraded":"2024-09-10T10:29:05.484806+0000","last_fullsized":"2024-09-10T10:29:05.484806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482528+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:29:05.482528+0000","last_peered":"2024-09-10T10:29:05.482528+0000","last_clean":"2024-09-10T10:29:05.482528+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:29:05.482528+0000","last_undegraded":"2024-09-10T10:29:05.482528+0000","last_fullsized":"2024-09-10T10:29:05.482528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486220+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:29:05.486220+0000","last_peered":"2024-09-10T10:29:05.486220+0000","last_clean":"2024-09-10T10:29:05.486220+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:29:05.486220+0000","last_undegraded":"2024-09-10T10:29:05.486220+0000","last_fullsized":"2024-09-10T10:29:05.486220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893191+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:29:05.893191+0000","last_peered":"2024-09-10T10:29:05.893191+0000","last_clean":"2024-09-10T10:29:05.893191+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:29:05.893191+0000","last_undegraded":"2024-09-10T10:29:05.893191+0000","last_fullsized":"2024-09-10T10:29:05.893191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832375+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:29:05.832375+0000","last_peered":"2024-09-10T10:29:05.832375+0000","last_clean":"2024-09-10T10:29:05.832375+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:29:05.832375+0000","last_undegraded":"2024-09-10T10:29:05.832375+0000","last_fullsized":"2024-09-10T10:29:05.832375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688489+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:29:05.688489+0000","last_peered":"2024-09-10T10:29:05.688489+0000","last_clean":"2024-09-10T10:29:05.688489+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:29:05.688489+0000","last_undegraded":"2024-09-10T10:29:05.688489+0000","last_fullsized":"2024-09-10T10:29:05.688489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485637+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:29:05.485637+0000","last_peered":"2024-09-10T10:29:05.485637+0000","last_clean":"2024-09-10T10:29:05.485637+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:29:05.485637+0000","last_undegraded":"2024-09-10T10:29:05.485637+0000","last_fullsized":"2024-09-10T10:29:05.485637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832269+0000","last_change":"2024-09-10T10:28:59.446917+0000","last_active":"2024-09-10T10:29:05.832269+0000","last_peered":"2024-09-10T10:29:05.832269+0000","last_clean":"2024-09-10T10:29:05.832269+0000","last_became_active":"2024-09-10T10:28:59.446546+0000","last_became_peered":"2024-09-10T10:28:59.446546+0000","last_unstale":"2024-09-10T10:29:05.832269+0000","last_undegraded":"2024-09-10T10:29:05.832269+0000","last_fullsized":"2024-09-10T10:29:05.832269+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:30.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485795+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:29:05.485795+0000","last_peered":"2024-09-10T10:29:05.485795+0000","last_clean":"2024-09-10T10:29:05.485795+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:29:05.485795+0000","last_undegraded":"2024-09-10T10:29:05.485795+0000","last_fullsized":"2024-09-10T10:29:05.485795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831886+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:29:05.831886+0000","last_peered":"2024-09-10T10:29:05.831886+0000","last_clean":"2024-09-10T10:29:05.831886+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:29:05.831886+0000","last_undegraded":"2024-09-10T10:29:05.831886+0000","last_fullsized":"2024-09-10T10:29:05.831886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486218+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:29:05.486218+0000","last_peered":"2024-09-10T10:29:05.486218+0000","last_clean":"2024-09-10T10:29:05.486218+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:29:05.486218+0000","last_undegraded":"2024-09-10T10:29:05.486218+0000","last_fullsized":"2024-09-10T10:29:05.486218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892472+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:29:05.892472+0000","last_peered":"2024-09-10T10:29:05.892472+0000","last_clean":"2024-09-10T10:29:05.892472+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:29:05.892472+0000","last_undegraded":"2024-09-10T10:29:05.892472+0000","last_fullsized":"2024-09-10T10:29:05.892472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688898+0000","last_change":"2024-09-10T10:28:23.249392+0000","last_active":"2024-09-10T10:29:05.688898+0000","last_peered":"2024-09-10T10:29:05.688898+0000","last_clean":"2024-09-10T10:29:05.688898+0000","last_became_active":"2024-09-10T10:28:23.249039+0000","last_became_peered":"2024-09-10T10:28:23.249039+0000","last_unstale":"2024-09-10T10:29:05.688898+0000","last_undegraded":"2024-09-10T10:29:05.688898+0000","last_fullsized":"2024-09-10T10:29:05.688898+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:46.656488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485234+0000","last_change":"2024-09-10T10:28:24.244885+0000","last_active":"2024-09-10T10:29:05.485234+0000","last_peered":"2024-09-10T10:29:05.485234+0000","last_clean":"2024-09-10T10:29:05.485234+0000","last_became_active":"2024-09-10T10:28:24.244560+0000","last_became_peered":"2024-09-10T10:28:24.244560+0000","last_unstale":"2024-09-10T10:29:05.485234+0000","last_undegraded":"2024-09-10T10:29:05.485234+0000","last_fullsized":"2024-09-10T10:29:05.485234+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:36.642112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483017+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:29:05.483017+0000","last_peered":"2024-09-10T10:29:05.483017+0000","last_clean":"2024-09-10T10:29:05.483017+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:29:05.483017+0000","last_undegraded":"2024-09-10T10:29:05.483017+0000","last_fullsized":"2024-09-10T10:29:05.483017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486768+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:29:05.486768+0000","last_peered":"2024-09-10T10:29:05.486768+0000","last_clean":"2024-09-10T10:29:05.486768+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:29:05.486768+0000","last_undegraded":"2024-09-10T10:29:05.486768+0000","last_fullsized":"2024-09-10T10:29:05.486768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485724+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:29:05.485724+0000","last_peered":"2024-09-10T10:29:05.485724+0000","last_clean":"2024-09-10T10:29:05.485724+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:29:05.485724+0000","last_undegraded":"2024-09-10T10:29:05.485724+0000","last_fullsized":"2024-09-10T10:29:05.485724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688424+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:29:05.688424+0000","last_peered":"2024-09-10T10:29:05.688424+0000","last_clean":"2024-09-10T10:29:05.688424+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:29:05.688424+0000","last_undegraded":"2024-09-10T10:29:05.688424+0000","last_fullsized":"2024-09-10T10:29:05.688424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"60'1","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892613+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:29:05.892613+0000","last_peered":"2024-09-10T10:29:05.892613+0000","last_clean":"2024-09-10T10:29:05.892613+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:29:05.892613+0000","last_undegraded":"2024-09-10T10:29:05.892613+0000","last_fullsized":"2024-09-10T10:29:05.892613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482510+0000","last_change":"2024-09-10T10:28:24.243479+0000","last_active":"2024-09-10T10:29:05.482510+0000","last_peered":"2024-09-10T10:29:05.482510+0000","last_clean":"2024-09-10T10:29:05.482510+0000","last_became_active":"2024-09-10T10:28:24.243088+0000","last_became_peered":"2024-09-10T10:28:24.243088+0000","last_unstale":"2024-09-10T10:29:05.482510+0000","last_undegraded":"2024-09-10T10:29:05.482510+0000","last_fullsized":"2024-09-10T10:29:05.482510+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:39.465491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":86,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831975+0000","last_change":"2024-09-10T10:27:58.327792+0000","last_active":"2024-09-10T10:29:05.831975+0000","last_peered":"2024-09-10T10:29:05.831975+0000","last_clean":"2024-09-10T10:29:05.831975+0000","last_became_active":"2024-09-10T10:27:58.327502+0000","last_became_peered":"2024-09-10T10:27:58.327502+0000","last_unstale":"2024-09-10T10:29:05.831975+0000","last_undegraded":"2024-09-10T10:29:05.831975+0000","last_fullsized":"2024-09-10T10:29:05.831975+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485354+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:29:05.485354+0000","last_peered":"2024-09-10T10:29:05.485354+0000","last_clean":"2024-09-10T10:29:05.485354+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:29:05.485354+0000","last_undegraded":"2024-09-10T10:29:05.485354+0000","last_fullsized":"2024-09-10T10:29:05.485354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485527+0000","last_change":"2024-09-10T10:28:40.345087+0000","last_active":"2024-09-10T10:29:05.485527+0000","last_peered":"2024-09-10T10:29:05.485527+0000","last_clean":"2024-09-10T10:29:05.485527+0000","last_became_active":"2024-09-10T10:28:40.344766+0000","last_became_peered":"2024-09-10T10:28:40.344766+0000","last_unstale":"2024-09-10T10:29:05.485527+0000","last_undegraded":"2024-09-10T10:29:05.485527+0000","last_fullsized":"2024-09-10T10:29:05.485527+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521337+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:29:05.521337+0000","last_peered":"2024-09-10T10:29:05.521337+0000","last_clean":"2024-09-10T10:29:05.521337+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:29:05.521337+0000","last_undegraded":"2024-09-10T10:29:05.521337+0000","last_fullsized":"2024-09-10T10:29:05.521337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831395+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:29:05.831395+0000","last_peered":"2024-09-10T10:29:05.831395+0000","last_clean":"2024-09-10T10:29:05.831395+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:29:05.831395+0000","last_undegraded":"2024-09-10T10:29:05.831395+0000","last_fullsized":"2024-09-10T10:29:05.831395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520601+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:29:05.520601+0000","last_peered":"2024-09-10T10:29:05.520601+0000","last_clean":"2024-09-10T10:29:05.520601+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:29:05.520601+0000","last_undegraded":"2024-09-10T10:29:05.520601+0000","last_fullsized":"2024-09-10T10:29:05.520601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486096+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:29:05.486096+0000","last_peered":"2024-09-10T10:29:05.486096+0000","last_clean":"2024-09-10T10:29:05.486096+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:29:05.486096+0000","last_undegraded":"2024-09-10T10:29:05.486096+0000","last_fullsized":"2024-09-10T10:29:05.486096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520639+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:29:05.520639+0000","last_peered":"2024-09-10T10:29:05.520639+0000","last_clean":"2024-09-10T10:29:05.520639+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:29:05.520639+0000","last_undegraded":"2024-09-10T10:29:05.520639+0000","last_fullsized":"2024-09-10T10:29:05.520639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486354+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:29:05.486354+0000","last_peered":"2024-09-10T10:29:05.486354+0000","last_clean":"2024-09-10T10:29:05.486354+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:29:05.486354+0000","last_undegraded":"2024-09-10T10:29:05.486354+0000","last_fullsized":"2024-09-10T10:29:05.486354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892681+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:29:05.892681+0000","last_peered":"2024-09-10T10:29:05.892681+0000","last_clean":"2024-09-10T10:29:05.892681+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:29:05.892681+0000","last_undegraded":"2024-09-10T10:29:05.892681+0000","last_fullsized":"2024-09-10T10:29:05.892681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688961+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:29:05.688961+0000","last_peered":"2024-09-10T10:29:05.688961+0000","last_clean":"2024-09-10T10:29:05.688961+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:29:05.688961+0000","last_undegraded":"2024-09-10T10:29:05.688961+0000","last_fullsized":"2024-09-10T10:29:05.688961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521230+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:29:05.521230+0000","last_peered":"2024-09-10T10:29:05.521230+0000","last_clean":"2024-09-10T10:29:05.521230+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:29:05.521230+0000","last_undegraded":"2024-09-10T10:29:05.521230+0000","last_fullsized":"2024-09-10T10:29:05.521230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44104,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":247,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":88208,"data_compressed":3324,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":995328,"data_stored":1059983,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":974848,"data_stored":519451,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":966656,"data_stored":485579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":475783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":512959,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":954368,"data_stored":483521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":933888,"data_stored":475985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":1065695,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":38506,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":37804,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:29:09.057 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:29:09.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.185+0000 7fb5a2e27640 1 Processor -- start 2024-09-10T10:29:09.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.185+0000 7fb5a2e27640 1 -- start start 2024-09-10T10:29:09.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.186+0000 7fb5a2e27640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb59c06ce90 0x7fb59c0c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.186+0000 7fb5a2e27640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb59c0ca380 0x7fb59c158da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.186+0000 7fb5a2e27640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59c0c8db0 con 0x7fb59c06d270 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.186+0000 7fb5a2e27640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb59c15cc10 con 0x7fb59c0ca380 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.186+0000 7fb5a2e27640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb59c0c9050 con 0x7fb59c06ce90 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.186+0000 7fb593fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb59c0ca380 0x7fb59c158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.186+0000 7fb593fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb59c0ca380 0x7fb59c158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42396/0 (socket says 172.21.15.120:42396) 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.186+0000 7fb593fff640 1 -- 172.21.15.120:0/1066102936 learned_addr learned my addr 172.21.15.120:0/1066102936 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.187+0000 7fb5a0bc3640 1 --2- 172.21.15.120:0/1066102936 >> v2:172.21.15.134:3300/0 conn(0x7fb59c06ce90 0x7fb59c0c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.187+0000 7fb593fff640 1 -- 172.21.15.120:0/1066102936 >> v2:172.21.15.134:3300/0 conn(0x7fb59c06ce90 msgr2=0x7fb59c0c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.187+0000 7fb593fff640 1 --2- 172.21.15.120:0/1066102936 >> v2:172.21.15.134:3300/0 conn(0x7fb59c06ce90 0x7fb59c0c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.187+0000 7fb593fff640 1 -- 172.21.15.120:0/1066102936 >> v1:172.21.15.134:6789/0 conn(0x7fb59c06d270 legacy=0x7fb59c0c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.187+0000 7fb593fff640 1 -- 172.21.15.120:0/1066102936 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59c0c9330 con 0x7fb59c0ca380 2024-09-10T10:29:09.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.187+0000 7fb593fff640 1 --2- 172.21.15.120:0/1066102936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c0ca380 0x7fb59c158da0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb59c071bc0 tx=0x7fb58c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c671d9339f2cefd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:09.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5937fe640 1 -- 172.21.15.120:0/1066102936 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c037070 con 0x7fb59c0ca380 2024-09-10T10:29:09.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5937fe640 1 -- 172.21.15.120:0/1066102936 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c02ae50 con 0x7fb59c0ca380 2024-09-10T10:29:09.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5937fe640 1 -- 172.21.15.120:0/1066102936 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c032860 con 0x7fb59c0ca380 2024-09-10T10:29:09.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1066102936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c0ca380 msgr2=0x7fb59c158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5a2e27640 1 --2- 172.21.15.120:0/1066102936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c0ca380 0x7fb59c158da0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb59c071bc0 tx=0x7fb58c02a300 comp rx=0 tx=0).stop 2024-09-10T10:29:09.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1066102936 shutdown_connections 2024-09-10T10:29:09.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5a2e27640 1 --2- 172.21.15.120:0/1066102936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c0ca380 0x7fb59c158da0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5a2e27640 1 --2- 172.21.15.120:0/1066102936 >> v2:172.21.15.134:3300/0 conn(0x7fb59c06ce90 0x7fb59c0c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1066102936 >> 172.21.15.120:0/1066102936 conn(0x7fb59c0c96b0 msgr2=0x7fb59c0c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:09.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1066102936 shutdown_connections 2024-09-10T10:29:09.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.188+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1066102936 wait complete. 2024-09-10T10:29:09.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.189+0000 7fb5a2e27640 1 Processor -- start 2024-09-10T10:29:09.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.189+0000 7fb5a2e27640 1 -- start start 2024-09-10T10:29:09.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.190+0000 7fb5a2e27640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb59c06ce90 0x7fb59c158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.190+0000 7fb5a2e27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c06d270 0x7fb59c1f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.190+0000 7fb5a2e27640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb59c0ca380 0x7fb59c0c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.190+0000 7fb5a2e27640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c0c6fc0 con 0x7fb59c06d270 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.190+0000 7fb5a2e27640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c0c3c80 con 0x7fb59c06ce90 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.190+0000 7fb5a2e27640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb59c067950 con 0x7fb59c0ca380 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.190+0000 7fb593fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c06d270 0x7fb59c1f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.190+0000 7fb593fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c06d270 0x7fb59c1f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42404/0 (socket says 172.21.15.120:42404) 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.190+0000 7fb593fff640 1 -- 172.21.15.120:0/1870192794 learned_addr learned my addr 172.21.15.120:0/1870192794 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb5a0bc3640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb59c06ce90 0x7fb59c158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb5a13c4640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb59c0ca380 0x7fb59c0c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb593fff640 1 -- 172.21.15.120:0/1870192794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb59c0ca380 msgr2=0x7fb59c0c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb593fff640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb59c0ca380 0x7fb59c0c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb593fff640 1 -- 172.21.15.120:0/1870192794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb59c06ce90 msgr2=0x7fb59c158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb593fff640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb59c06ce90 0x7fb59c158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb593fff640 1 -- 172.21.15.120:0/1870192794 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59c156060 con 0x7fb59c06d270 2024-09-10T10:29:09.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb5a13c4640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb59c0ca380 0x7fb59c0c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:09.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb593fff640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c06d270 0x7fb59c1f1df0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb58c02a340 tx=0x7fb58c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:09.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb5917fa640 1 -- 172.21.15.120:0/1870192794 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c041070 con 0x7fb59c06d270 2024-09-10T10:29:09.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb5917fa640 1 -- 172.21.15.120:0/1870192794 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c03c870 con 0x7fb59c06d270 2024-09-10T10:29:09.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1870192794 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59c15bb90 con 0x7fb59c06d270 2024-09-10T10:29:09.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb5917fa640 1 -- 172.21.15.120:0/1870192794 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c037040 con 0x7fb59c06d270 2024-09-10T10:29:09.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.191+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1870192794 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59c1f28d0 con 0x7fb59c06d270 2024-09-10T10:29:09.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.192+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1870192794 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb59c0c4260 con 0x7fb59c06d270 2024-09-10T10:29:09.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.192+0000 7fb5917fa640 1 -- 172.21.15.120:0/1870192794 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb58c044070 con 0x7fb59c06d270 2024-09-10T10:29:09.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.193+0000 7fb5917fa640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb58806be60 0x7fb58806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.193+0000 7fb5917fa640 1 -- 172.21.15.120:0/1870192794 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fb58c0abdc0 con 0x7fb59c06d270 2024-09-10T10:29:09.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.193+0000 7fb5a0bc3640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb58806be60 0x7fb58806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.193+0000 7fb5a0bc3640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb58806be60 0x7fb58806e320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb598006fd0 tx=0x7fb598008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:09.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.197+0000 7fb5917fa640 1 -- 172.21.15.120:0/1870192794 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb59c156060 con 0x7fb59c06d270 2024-09-10T10:29:09.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.357+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1870192794 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb59c0c1e40 con 0x7fb58806be60 2024-09-10T10:29:09.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.365+0000 7fb5917fa640 1 -- 172.21.15.120:0/1870192794 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332945 (secure 0 0 0) 0x7fb59c0c1e40 con 0x7fb58806be60 2024-09-10T10:29:09.370 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:09.371 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:29:09.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1870192794 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb58806be60 msgr2=0x7fb58806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb58806be60 0x7fb58806e320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb598006fd0 tx=0x7fb598008040 comp rx=0 tx=0).stop 2024-09-10T10:29:09.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1870192794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c06d270 msgr2=0x7fb59c1f1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c06d270 0x7fb59c1f1df0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb58c02a340 tx=0x7fb58c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1870192794 shutdown_connections 2024-09-10T10:29:09.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb58806be60 0x7fb58806e320 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb59c0ca380 0x7fb59c0c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb59c06d270 0x7fb59c1f1df0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 --2- 172.21.15.120:0/1870192794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb59c06ce90 0x7fb59c158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1870192794 >> 172.21.15.120:0/1870192794 conn(0x7fb59c0c96b0 msgr2=0x7fb59c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:09.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1870192794 shutdown_connections 2024-09-10T10:29:09.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.368+0000 7fb5a2e27640 1 -- 172.21.15.120:0/1870192794 wait complete. 2024-09-10T10:29:09.381 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-10T10:29:08.120056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634472,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":119,"num_read_kb":103,"num_write":408,"num_write_kb":2539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227948,"kb_used_data":7544,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754746772,"statfs":{"total":773094113280,"available":772860694528,"internally_reserved":0,"allocated":7725056,"data_stored":5078956,"data_compressed":68176,"data_compressed_allocated":98304,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.662305"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893394+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:29:05.893394+0000","last_peered":"2024-09-10T10:29:05.893394+0000","last_clean":"2024-09-10T10:29:05.893394+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:29:05.893394+0000","last_undegraded":"2024-09-10T10:29:05.893394+0000","last_fullsized":"2024-09-10T10:29:05.893394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831510+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:29:05.831510+0000","last_peered":"2024-09-10T10:29:05.831510+0000","last_clean":"2024-09-10T10:29:05.831510+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:29:05.831510+0000","last_undegraded":"2024-09-10T10:29:05.831510+0000","last_fullsized":"2024-09-10T10:29:05.831510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831742+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:29:05.831742+0000","last_peered":"2024-09-10T10:29:05.831742+0000","last_clean":"2024-09-10T10:29:05.831742+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:29:05.831742+0000","last_undegraded":"2024-09-10T10:29:05.831742+0000","last_fullsized":"2024-09-10T10:29:05.831742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832208+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:29:05.832208+0000","last_peered":"2024-09-10T10:29:05.832208+0000","last_clean":"2024-09-10T10:29:05.832208+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:29:05.832208+0000","last_undegraded":"2024-09-10T10:29:05.832208+0000","last_fullsized":"2024-09-10T10:29:05.832208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892600+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:29:05.892600+0000","last_peered":"2024-09-10T10:29:05.892600+0000","last_clean":"2024-09-10T10:29:05.892600+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:29:05.892600+0000","last_undegraded":"2024-09-10T10:29:05.892600+0000","last_fullsized":"2024-09-10T10:29:05.892600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955501+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:29:05.955501+0000","last_peered":"2024-09-10T10:29:05.955501+0000","last_clean":"2024-09-10T10:29:05.955501+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:29:05.955501+0000","last_undegraded":"2024-09-10T10:29:05.955501+0000","last_fullsized":"2024-09-10T10:29:05.955501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"60'13","reported_seq":158,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521178+0000","last_change":"2024-09-10T10:27:51.060242+0000","last_active":"2024-09-10T10:29:05.521178+0000","last_peered":"2024-09-10T10:29:05.521178+0000","last_clean":"2024-09-10T10:29:05.521178+0000","last_became_active":"2024-09-10T10:27:51.049148+0000","last_became_peered":"2024-09-10T10:27:51.049148+0000","last_unstale":"2024-09-10T10:29:05.521178+0000","last_undegraded":"2024-09-10T10:29:05.521178+0000","last_fullsized":"2024-09-10T10:29:05.521178+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.604144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486386+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:29:05.486386+0000","last_peered":"2024-09-10T10:29:05.486386+0000","last_clean":"2024-09-10T10:29:05.486386+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:29:05.486386+0000","last_undegraded":"2024-09-10T10:29:05.486386+0000","last_fullsized":"2024-09-10T10:29:05.486386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955770+0000","last_change":"2024-09-10T10:28:24.244697+0000","last_active":"2024-09-10T10:29:05.955770+0000","last_peered":"2024-09-10T10:29:05.955770+0000","last_clean":"2024-09-10T10:29:05.955770+0000","last_became_active":"2024-09-10T10:28:24.244262+0000","last_became_peered":"2024-09-10T10:28:24.244262+0000","last_unstale":"2024-09-10T10:29:05.955770+0000","last_undegraded":"2024-09-10T10:29:05.955770+0000","last_fullsized":"2024-09-10T10:29:05.955770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:50.683186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6,2],"acting":[4,0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520262+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:29:05.520262+0000","last_peered":"2024-09-10T10:29:05.520262+0000","last_clean":"2024-09-10T10:29:05.520262+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:29:05.520262+0000","last_undegraded":"2024-09-10T10:29:05.520262+0000","last_fullsized":"2024-09-10T10:29:05.520262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":110,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482709+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:29:05.482709+0000","last_peered":"2024-09-10T10:29:05.482709+0000","last_clean":"2024-09-10T10:29:05.482709+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:29:05.482709+0000","last_undegraded":"2024-09-10T10:29:05.482709+0000","last_fullsized":"2024-09-10T10:29:05.482709+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955722+0000","last_change":"2024-09-10T10:28:59.443607+0000","last_active":"2024-09-10T10:29:05.955722+0000","last_peered":"2024-09-10T10:29:05.955722+0000","last_clean":"2024-09-10T10:29:05.955722+0000","last_became_active":"2024-09-10T10:28:59.443172+0000","last_became_peered":"2024-09-10T10:28:59.443172+0000","last_unstale":"2024-09-10T10:29:05.955722+0000","last_undegraded":"2024-09-10T10:29:05.955722+0000","last_fullsized":"2024-09-10T10:29:05.955722+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:36.379163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485088+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:29:05.485088+0000","last_peered":"2024-09-10T10:29:05.485088+0000","last_clean":"2024-09-10T10:29:05.485088+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:29:05.485088+0000","last_undegraded":"2024-09-10T10:29:05.485088+0000","last_fullsized":"2024-09-10T10:29:05.485088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"60'5","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520220+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:29:05.520220+0000","last_peered":"2024-09-10T10:29:05.520220+0000","last_clean":"2024-09-10T10:29:05.520220+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:29:05.520220+0000","last_undegraded":"2024-09-10T10:29:05.520220+0000","last_fullsized":"2024-09-10T10:29:05.520220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956119+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:29:05.956119+0000","last_peered":"2024-09-10T10:29:05.956119+0000","last_clean":"2024-09-10T10:29:05.956119+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:29:05.956119+0000","last_undegraded":"2024-09-10T10:29:05.956119+0000","last_fullsized":"2024-09-10T10:29:05.956119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831889+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:29:05.831889+0000","last_peered":"2024-09-10T10:29:05.831889+0000","last_clean":"2024-09-10T10:29:05.831889+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:29:05.831889+0000","last_undegraded":"2024-09-10T10:29:05.831889+0000","last_fullsized":"2024-09-10T10:29:05.831889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688805+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:29:05.688805+0000","last_peered":"2024-09-10T10:29:05.688805+0000","last_clean":"2024-09-10T10:29:05.688805+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:29:05.688805+0000","last_undegraded":"2024-09-10T10:29:05.688805+0000","last_fullsized":"2024-09-10T10:29:05.688805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955843+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:29:05.955843+0000","last_peered":"2024-09-10T10:29:05.955843+0000","last_clean":"2024-09-10T10:29:05.955843+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:29:05.955843+0000","last_undegraded":"2024-09-10T10:29:05.955843+0000","last_fullsized":"2024-09-10T10:29:05.955843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831383+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:29:05.831383+0000","last_peered":"2024-09-10T10:29:05.831383+0000","last_clean":"2024-09-10T10:29:05.831383+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:29:05.831383+0000","last_undegraded":"2024-09-10T10:29:05.831383+0000","last_fullsized":"2024-09-10T10:29:05.831383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688223+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:29:05.688223+0000","last_peered":"2024-09-10T10:29:05.688223+0000","last_clean":"2024-09-10T10:29:05.688223+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:29:05.688223+0000","last_undegraded":"2024-09-10T10:29:05.688223+0000","last_fullsized":"2024-09-10T10:29:05.688223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688656+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:29:05.688656+0000","last_peered":"2024-09-10T10:29:05.688656+0000","last_clean":"2024-09-10T10:29:05.688656+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:29:05.688656+0000","last_undegraded":"2024-09-10T10:29:05.688656+0000","last_fullsized":"2024-09-10T10:29:05.688656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955038+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:29:05.955038+0000","last_peered":"2024-09-10T10:29:05.955038+0000","last_clean":"2024-09-10T10:29:05.955038+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:29:05.955038+0000","last_undegraded":"2024-09-10T10:29:05.955038+0000","last_fullsized":"2024-09-10T10:29:05.955038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520435+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:29:05.520435+0000","last_peered":"2024-09-10T10:29:05.520435+0000","last_clean":"2024-09-10T10:29:05.520435+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:29:05.520435+0000","last_undegraded":"2024-09-10T10:29:05.520435+0000","last_fullsized":"2024-09-10T10:29:05.520435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956141+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:29:05.956141+0000","last_peered":"2024-09-10T10:29:05.956141+0000","last_clean":"2024-09-10T10:29:05.956141+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:29:05.956141+0000","last_undegraded":"2024-09-10T10:29:05.956141+0000","last_fullsized":"2024-09-10T10:29:05.956141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892494+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:29:05.892494+0000","last_peered":"2024-09-10T10:29:05.892494+0000","last_clean":"2024-09-10T10:29:05.892494+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:29:05.892494+0000","last_undegraded":"2024-09-10T10:29:05.892494+0000","last_fullsized":"2024-09-10T10:29:05.892494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'8","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485743+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:29:05.485743+0000","last_peered":"2024-09-10T10:29:05.485743+0000","last_clean":"2024-09-10T10:29:05.485743+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:29:05.485743+0000","last_undegraded":"2024-09-10T10:29:05.485743+0000","last_fullsized":"2024-09-10T10:29:05.485743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955720+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:29:05.955720+0000","last_peered":"2024-09-10T10:29:05.955720+0000","last_clean":"2024-09-10T10:29:05.955720+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:29:05.955720+0000","last_undegraded":"2024-09-10T10:29:05.955720+0000","last_fullsized":"2024-09-10T10:29:05.955720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485344+0000","last_change":"2024-09-10T10:28:24.374205+0000","last_active":"2024-09-10T10:29:05.485344+0000","last_peered":"2024-09-10T10:29:05.485344+0000","last_clean":"2024-09-10T10:29:05.485344+0000","last_became_active":"2024-09-10T10:28:24.373921+0000","last_became_peered":"2024-09-10T10:28:24.373921+0000","last_unstale":"2024-09-10T10:29:05.485344+0000","last_undegraded":"2024-09-10T10:29:05.485344+0000","last_fullsized":"2024-09-10T10:29:05.485344+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:36.923293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"64'15","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688969+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:29:05.688969+0000","last_peered":"2024-09-10T10:29:05.688969+0000","last_clean":"2024-09-10T10:29:05.688969+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:29:05.688969+0000","last_undegraded":"2024-09-10T10:29:05.688969+0000","last_fullsized":"2024-09-10T10:29:05.688969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831546+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:29:05.831546+0000","last_peered":"2024-09-10T10:29:05.831546+0000","last_clean":"2024-09-10T10:29:05.831546+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:29:05.831546+0000","last_undegraded":"2024-09-10T10:29:05.831546+0000","last_fullsized":"2024-09-10T10:29:05.831546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688907+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:29:05.688907+0000","last_peered":"2024-09-10T10:29:05.688907+0000","last_clean":"2024-09-10T10:29:05.688907+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:29:05.688907+0000","last_undegraded":"2024-09-10T10:29:05.688907+0000","last_fullsized":"2024-09-10T10:29:05.688907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521226+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:29:05.521226+0000","last_peered":"2024-09-10T10:29:05.521226+0000","last_clean":"2024-09-10T10:29:05.521226+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:29:05.521226+0000","last_undegraded":"2024-09-10T10:29:05.521226+0000","last_fullsized":"2024-09-10T10:29:05.521226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893084+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:29:05.893084+0000","last_peered":"2024-09-10T10:29:05.893084+0000","last_clean":"2024-09-10T10:29:05.893084+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:29:05.893084+0000","last_undegraded":"2024-09-10T10:29:05.893084+0000","last_fullsized":"2024-09-10T10:29:05.893084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486566+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:29:05.486566+0000","last_peered":"2024-09-10T10:29:05.486566+0000","last_clean":"2024-09-10T10:29:05.486566+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:29:05.486566+0000","last_undegraded":"2024-09-10T10:29:05.486566+0000","last_fullsized":"2024-09-10T10:29:05.486566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688951+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:29:05.688951+0000","last_peered":"2024-09-10T10:29:05.688951+0000","last_clean":"2024-09-10T10:29:05.688951+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.688951+0000","last_undegraded":"2024-09-10T10:29:05.688951+0000","last_fullsized":"2024-09-10T10:29:05.688951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688451+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:29:05.688451+0000","last_peered":"2024-09-10T10:29:05.688451+0000","last_clean":"2024-09-10T10:29:05.688451+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:29:05.688451+0000","last_undegraded":"2024-09-10T10:29:05.688451+0000","last_fullsized":"2024-09-10T10:29:05.688451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956082+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:29:05.956082+0000","last_peered":"2024-09-10T10:29:05.956082+0000","last_clean":"2024-09-10T10:29:05.956082+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:29:05.956082+0000","last_undegraded":"2024-09-10T10:29:05.956082+0000","last_fullsized":"2024-09-10T10:29:05.956082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486359+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:29:05.486359+0000","last_peered":"2024-09-10T10:29:05.486359+0000","last_clean":"2024-09-10T10:29:05.486359+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:29:05.486359+0000","last_undegraded":"2024-09-10T10:29:05.486359+0000","last_fullsized":"2024-09-10T10:29:05.486359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486083+0000","last_change":"2024-09-10T10:28:24.244988+0000","last_active":"2024-09-10T10:29:05.486083+0000","last_peered":"2024-09-10T10:29:05.486083+0000","last_clean":"2024-09-10T10:29:05.486083+0000","last_became_active":"2024-09-10T10:28:24.244556+0000","last_became_peered":"2024-09-10T10:28:24.244556+0000","last_unstale":"2024-09-10T10:29:05.486083+0000","last_undegraded":"2024-09-10T10:29:05.486083+0000","last_fullsized":"2024-09-10T10:29:05.486083+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:08.127492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688685+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:29:05.688685+0000","last_peered":"2024-09-10T10:29:05.688685+0000","last_clean":"2024-09-10T10:29:05.688685+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:29:05.688685+0000","last_undegraded":"2024-09-10T10:29:05.688685+0000","last_fullsized":"2024-09-10T10:29:05.688685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'9","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482578+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:29:05.482578+0000","last_peered":"2024-09-10T10:29:05.482578+0000","last_clean":"2024-09-10T10:29:05.482578+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.482578+0000","last_undegraded":"2024-09-10T10:29:05.482578+0000","last_fullsized":"2024-09-10T10:29:05.482578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688718+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:29:05.688718+0000","last_peered":"2024-09-10T10:29:05.688718+0000","last_clean":"2024-09-10T10:29:05.688718+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:29:05.688718+0000","last_undegraded":"2024-09-10T10:29:05.688718+0000","last_fullsized":"2024-09-10T10:29:05.688718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892835+0000","last_change":"2024-09-10T10:28:59.446889+0000","last_active":"2024-09-10T10:29:05.892835+0000","last_peered":"2024-09-10T10:29:05.892835+0000","last_clean":"2024-09-10T10:29:05.892835+0000","last_became_active":"2024-09-10T10:28:59.446544+0000","last_became_peered":"2024-09-10T10:28:59.446544+0000","last_unstale":"2024-09-10T10:29:05.892835+0000","last_undegraded":"2024-09-10T10:29:05.892835+0000","last_fullsized":"2024-09-10T10:29:05.892835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:32.352466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520235+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:29:05.520235+0000","last_peered":"2024-09-10T10:29:05.520235+0000","last_clean":"2024-09-10T10:29:05.520235+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:29:05.520235+0000","last_undegraded":"2024-09-10T10:29:05.520235+0000","last_fullsized":"2024-09-10T10:29:05.520235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893277+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:29:05.893277+0000","last_peered":"2024-09-10T10:29:05.893277+0000","last_clean":"2024-09-10T10:29:05.893277+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:29:05.893277+0000","last_undegraded":"2024-09-10T10:29:05.893277+0000","last_fullsized":"2024-09-10T10:29:05.893277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831398+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:29:05.831398+0000","last_peered":"2024-09-10T10:29:05.831398+0000","last_clean":"2024-09-10T10:29:05.831398+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:29:05.831398+0000","last_undegraded":"2024-09-10T10:29:05.831398+0000","last_fullsized":"2024-09-10T10:29:05.831398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486160+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:29:05.486160+0000","last_peered":"2024-09-10T10:29:05.486160+0000","last_clean":"2024-09-10T10:29:05.486160+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:29:05.486160+0000","last_undegraded":"2024-09-10T10:29:05.486160+0000","last_fullsized":"2024-09-10T10:29:05.486160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":281,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485314+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:29:05.485314+0000","last_peered":"2024-09-10T10:29:05.485314+0000","last_clean":"2024-09-10T10:29:05.485314+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:29:05.485314+0000","last_undegraded":"2024-09-10T10:29:05.485314+0000","last_fullsized":"2024-09-10T10:29:05.485314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521019+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:29:05.521019+0000","last_peered":"2024-09-10T10:29:05.521019+0000","last_clean":"2024-09-10T10:29:05.521019+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:29:05.521019+0000","last_undegraded":"2024-09-10T10:29:05.521019+0000","last_fullsized":"2024-09-10T10:29:05.521019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482817+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:29:05.482817+0000","last_peered":"2024-09-10T10:29:05.482817+0000","last_clean":"2024-09-10T10:29:05.482817+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:29:05.482817+0000","last_undegraded":"2024-09-10T10:29:05.482817+0000","last_fullsized":"2024-09-10T10:29:05.482817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484865+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:29:05.484865+0000","last_peered":"2024-09-10T10:29:05.484865+0000","last_clean":"2024-09-10T10:29:05.484865+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:29:05.484865+0000","last_undegraded":"2024-09-10T10:29:05.484865+0000","last_fullsized":"2024-09-10T10:29:05.484865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486504+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:29:05.486504+0000","last_peered":"2024-09-10T10:29:05.486504+0000","last_clean":"2024-09-10T10:29:05.486504+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:29:05.486504+0000","last_undegraded":"2024-09-10T10:29:05.486504+0000","last_fullsized":"2024-09-10T10:29:05.486504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688628+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:29:05.688628+0000","last_peered":"2024-09-10T10:29:05.688628+0000","last_clean":"2024-09-10T10:29:05.688628+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:29:05.688628+0000","last_undegraded":"2024-09-10T10:29:05.688628+0000","last_fullsized":"2024-09-10T10:29:05.688628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893128+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:29:05.893128+0000","last_peered":"2024-09-10T10:29:05.893128+0000","last_clean":"2024-09-10T10:29:05.893128+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:29:05.893128+0000","last_undegraded":"2024-09-10T10:29:05.893128+0000","last_fullsized":"2024-09-10T10:29:05.893128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521367+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:29:05.521367+0000","last_peered":"2024-09-10T10:29:05.521367+0000","last_clean":"2024-09-10T10:29:05.521367+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:29:05.521367+0000","last_undegraded":"2024-09-10T10:29:05.521367+0000","last_fullsized":"2024-09-10T10:29:05.521367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483041+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:29:05.483041+0000","last_peered":"2024-09-10T10:29:05.483041+0000","last_clean":"2024-09-10T10:29:05.483041+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:29:05.483041+0000","last_undegraded":"2024-09-10T10:29:05.483041+0000","last_fullsized":"2024-09-10T10:29:05.483041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485530+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:29:05.485530+0000","last_peered":"2024-09-10T10:29:05.485530+0000","last_clean":"2024-09-10T10:29:05.485530+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:29:05.485530+0000","last_undegraded":"2024-09-10T10:29:05.485530+0000","last_fullsized":"2024-09-10T10:29:05.485530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521232+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:29:05.521232+0000","last_peered":"2024-09-10T10:29:05.521232+0000","last_clean":"2024-09-10T10:29:05.521232+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:29:05.521232+0000","last_undegraded":"2024-09-10T10:29:05.521232+0000","last_fullsized":"2024-09-10T10:29:05.521232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956264+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:29:05.956264+0000","last_peered":"2024-09-10T10:29:05.956264+0000","last_clean":"2024-09-10T10:29:05.956264+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:29:05.956264+0000","last_undegraded":"2024-09-10T10:29:05.956264+0000","last_fullsized":"2024-09-10T10:29:05.956264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955704+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:29:05.955704+0000","last_peered":"2024-09-10T10:29:05.955704+0000","last_clean":"2024-09-10T10:29:05.955704+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:29:05.955704+0000","last_undegraded":"2024-09-10T10:29:05.955704+0000","last_fullsized":"2024-09-10T10:29:05.955704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'18","reported_seq":147,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482470+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:29:05.482470+0000","last_peered":"2024-09-10T10:29:05.482470+0000","last_clean":"2024-09-10T10:29:05.482470+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:29:05.482470+0000","last_undegraded":"2024-09-10T10:29:05.482470+0000","last_fullsized":"2024-09-10T10:29:05.482470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893225+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:29:05.893225+0000","last_peered":"2024-09-10T10:29:05.893225+0000","last_clean":"2024-09-10T10:29:05.893225+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:29:05.893225+0000","last_undegraded":"2024-09-10T10:29:05.893225+0000","last_fullsized":"2024-09-10T10:29:05.893225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":160,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:28:59.453146+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:28:59.446453+0000","last_became_peered":"2024-09-10T10:28:59.446453+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:31.347510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1176,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832175+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:29:05.832175+0000","last_peered":"2024-09-10T10:29:05.832175+0000","last_clean":"2024-09-10T10:29:05.832175+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:29:05.832175+0000","last_undegraded":"2024-09-10T10:29:05.832175+0000","last_fullsized":"2024-09-10T10:29:05.832175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520795+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:29:05.520795+0000","last_peered":"2024-09-10T10:29:05.520795+0000","last_clean":"2024-09-10T10:29:05.520795+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:29:05.520795+0000","last_undegraded":"2024-09-10T10:29:05.520795+0000","last_fullsized":"2024-09-10T10:29:05.520795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"60'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486347+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:29:05.486347+0000","last_peered":"2024-09-10T10:29:05.486347+0000","last_clean":"2024-09-10T10:29:05.486347+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:29:05.486347+0000","last_undegraded":"2024-09-10T10:29:05.486347+0000","last_fullsized":"2024-09-10T10:29:05.486347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832184+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:29:05.832184+0000","last_peered":"2024-09-10T10:29:05.832184+0000","last_clean":"2024-09-10T10:29:05.832184+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:29:05.832184+0000","last_undegraded":"2024-09-10T10:29:05.832184+0000","last_fullsized":"2024-09-10T10:29:05.832184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892463+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:29:05.892463+0000","last_peered":"2024-09-10T10:29:05.892463+0000","last_clean":"2024-09-10T10:29:05.892463+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:29:05.892463+0000","last_undegraded":"2024-09-10T10:29:05.892463+0000","last_fullsized":"2024-09-10T10:29:05.892463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893348+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:29:05.893348+0000","last_peered":"2024-09-10T10:29:05.893348+0000","last_clean":"2024-09-10T10:29:05.893348+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:29:05.893348+0000","last_undegraded":"2024-09-10T10:29:05.893348+0000","last_fullsized":"2024-09-10T10:29:05.893348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486563+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:29:05.486563+0000","last_peered":"2024-09-10T10:29:05.486563+0000","last_clean":"2024-09-10T10:29:05.486563+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:29:05.486563+0000","last_undegraded":"2024-09-10T10:29:05.486563+0000","last_fullsized":"2024-09-10T10:29:05.486563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520992+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:29:05.520992+0000","last_peered":"2024-09-10T10:29:05.520992+0000","last_clean":"2024-09-10T10:29:05.520992+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:29:05.520992+0000","last_undegraded":"2024-09-10T10:29:05.520992+0000","last_fullsized":"2024-09-10T10:29:05.520992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"69'27","reported_seq":156,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485568+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:29:05.485568+0000","last_peered":"2024-09-10T10:29:05.485568+0000","last_clean":"2024-09-10T10:29:05.485568+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:29:05.485568+0000","last_undegraded":"2024-09-10T10:29:05.485568+0000","last_fullsized":"2024-09-10T10:29:05.485568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37014,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"60'3","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688649+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:29:05.688649+0000","last_peered":"2024-09-10T10:29:05.688649+0000","last_clean":"2024-09-10T10:29:05.688649+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:29:05.688649+0000","last_undegraded":"2024-09-10T10:29:05.688649+0000","last_fullsized":"2024-09-10T10:29:05.688649+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955123+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:29:05.955123+0000","last_peered":"2024-09-10T10:29:05.955123+0000","last_clean":"2024-09-10T10:29:05.955123+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:29:05.955123+0000","last_undegraded":"2024-09-10T10:29:05.955123+0000","last_fullsized":"2024-09-10T10:29:05.955123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'8","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688658+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:29:05.688658+0000","last_peered":"2024-09-10T10:29:05.688658+0000","last_clean":"2024-09-10T10:29:05.688658+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:29:05.688658+0000","last_undegraded":"2024-09-10T10:29:05.688658+0000","last_fullsized":"2024-09-10T10:29:05.688658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955890+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:29:05.955890+0000","last_peered":"2024-09-10T10:29:05.955890+0000","last_clean":"2024-09-10T10:29:05.955890+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:29:05.955890+0000","last_undegraded":"2024-09-10T10:29:05.955890+0000","last_fullsized":"2024-09-10T10:29:05.955890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892512+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:29:05.892512+0000","last_peered":"2024-09-10T10:29:05.892512+0000","last_clean":"2024-09-10T10:29:05.892512+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:29:05.892512+0000","last_undegraded":"2024-09-10T10:29:05.892512+0000","last_fullsized":"2024-09-10T10:29:05.892512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486525+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:29:05.486525+0000","last_peered":"2024-09-10T10:29:05.486525+0000","last_clean":"2024-09-10T10:29:05.486525+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:29:05.486525+0000","last_undegraded":"2024-09-10T10:29:05.486525+0000","last_fullsized":"2024-09-10T10:29:05.486525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831619+0000","last_change":"2024-09-10T10:28:24.248827+0000","last_active":"2024-09-10T10:29:05.831619+0000","last_peered":"2024-09-10T10:29:05.831619+0000","last_clean":"2024-09-10T10:29:05.831619+0000","last_became_active":"2024-09-10T10:28:24.248395+0000","last_became_peered":"2024-09-10T10:28:24.248395+0000","last_unstale":"2024-09-10T10:29:05.831619+0000","last_undegraded":"2024-09-10T10:29:05.831619+0000","last_fullsized":"2024-09-10T10:29:05.831619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:05.306669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":106,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688561+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:29:05.688561+0000","last_peered":"2024-09-10T10:29:05.688561+0000","last_clean":"2024-09-10T10:29:05.688561+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:29:05.688561+0000","last_undegraded":"2024-09-10T10:29:05.688561+0000","last_fullsized":"2024-09-10T10:29:05.688561+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"60'6","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485563+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:29:05.485563+0000","last_peered":"2024-09-10T10:29:05.485563+0000","last_clean":"2024-09-10T10:29:05.485563+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:29:05.485563+0000","last_undegraded":"2024-09-10T10:29:05.485563+0000","last_fullsized":"2024-09-10T10:29:05.485563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831826+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:29:05.831826+0000","last_peered":"2024-09-10T10:29:05.831826+0000","last_clean":"2024-09-10T10:29:05.831826+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:29:05.831826+0000","last_undegraded":"2024-09-10T10:29:05.831826+0000","last_fullsized":"2024-09-10T10:29:05.831826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893171+0000","last_change":"2024-09-10T10:28:24.376473+0000","last_active":"2024-09-10T10:29:05.893171+0000","last_peered":"2024-09-10T10:29:05.893171+0000","last_clean":"2024-09-10T10:29:05.893171+0000","last_became_active":"2024-09-10T10:28:24.376079+0000","last_became_peered":"2024-09-10T10:28:24.376079+0000","last_unstale":"2024-09-10T10:29:05.893171+0000","last_undegraded":"2024-09-10T10:29:05.893171+0000","last_fullsized":"2024-09-10T10:29:05.893171+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:53.600117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,1],"acting":[0,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688345+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:29:05.688345+0000","last_peered":"2024-09-10T10:29:05.688345+0000","last_clean":"2024-09-10T10:29:05.688345+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:29:05.688345+0000","last_undegraded":"2024-09-10T10:29:05.688345+0000","last_fullsized":"2024-09-10T10:29:05.688345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485919+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:29:05.485919+0000","last_peered":"2024-09-10T10:29:05.485919+0000","last_clean":"2024-09-10T10:29:05.485919+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:29:05.485919+0000","last_undegraded":"2024-09-10T10:29:05.485919+0000","last_fullsized":"2024-09-10T10:29:05.485919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485377+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:29:05.485377+0000","last_peered":"2024-09-10T10:29:05.485377+0000","last_clean":"2024-09-10T10:29:05.485377+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:29:05.485377+0000","last_undegraded":"2024-09-10T10:29:05.485377+0000","last_fullsized":"2024-09-10T10:29:05.485377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831665+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:29:05.831665+0000","last_peered":"2024-09-10T10:29:05.831665+0000","last_clean":"2024-09-10T10:29:05.831665+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:29:05.831665+0000","last_undegraded":"2024-09-10T10:29:05.831665+0000","last_fullsized":"2024-09-10T10:29:05.831665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832255+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:29:05.832255+0000","last_peered":"2024-09-10T10:29:05.832255+0000","last_clean":"2024-09-10T10:29:05.832255+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:29:05.832255+0000","last_undegraded":"2024-09-10T10:29:05.832255+0000","last_fullsized":"2024-09-10T10:29:05.832255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485185+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:29:05.485185+0000","last_peered":"2024-09-10T10:29:05.485185+0000","last_clean":"2024-09-10T10:29:05.485185+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:29:05.485185+0000","last_undegraded":"2024-09-10T10:29:05.485185+0000","last_fullsized":"2024-09-10T10:29:05.485185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893219+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:29:05.893219+0000","last_peered":"2024-09-10T10:29:05.893219+0000","last_clean":"2024-09-10T10:29:05.893219+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:29:05.893219+0000","last_undegraded":"2024-09-10T10:29:05.893219+0000","last_fullsized":"2024-09-10T10:29:05.893219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482644+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:29:05.482644+0000","last_peered":"2024-09-10T10:29:05.482644+0000","last_clean":"2024-09-10T10:29:05.482644+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:29:05.482644+0000","last_undegraded":"2024-09-10T10:29:05.482644+0000","last_fullsized":"2024-09-10T10:29:05.482644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'6","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831961+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:29:05.831961+0000","last_peered":"2024-09-10T10:29:05.831961+0000","last_clean":"2024-09-10T10:29:05.831961+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:29:05.831961+0000","last_undegraded":"2024-09-10T10:29:05.831961+0000","last_fullsized":"2024-09-10T10:29:05.831961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955040+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:29:05.955040+0000","last_peered":"2024-09-10T10:29:05.955040+0000","last_clean":"2024-09-10T10:29:05.955040+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:29:05.955040+0000","last_undegraded":"2024-09-10T10:29:05.955040+0000","last_fullsized":"2024-09-10T10:29:05.955040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832152+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:29:05.832152+0000","last_peered":"2024-09-10T10:29:05.832152+0000","last_clean":"2024-09-10T10:29:05.832152+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:29:05.832152+0000","last_undegraded":"2024-09-10T10:29:05.832152+0000","last_fullsized":"2024-09-10T10:29:05.832152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484806+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:29:05.484806+0000","last_peered":"2024-09-10T10:29:05.484806+0000","last_clean":"2024-09-10T10:29:05.484806+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:29:05.484806+0000","last_undegraded":"2024-09-10T10:29:05.484806+0000","last_fullsized":"2024-09-10T10:29:05.484806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482528+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:29:05.482528+0000","last_peered":"2024-09-10T10:29:05.482528+0000","last_clean":"2024-09-10T10:29:05.482528+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:29:05.482528+0000","last_undegraded":"2024-09-10T10:29:05.482528+0000","last_fullsized":"2024-09-10T10:29:05.482528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486220+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:29:05.486220+0000","last_peered":"2024-09-10T10:29:05.486220+0000","last_clean":"2024-09-10T10:29:05.486220+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:29:05.486220+0000","last_undegraded":"2024-09-10T10:29:05.486220+0000","last_fullsized":"2024-09-10T10:29:05.486220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893191+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:29:05.893191+0000","last_peered":"2024-09-10T10:29:05.893191+0000","last_clean":"2024-09-10T10:29:05.893191+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:29:05.893191+0000","last_undegraded":"2024-09-10T10:29:05.893191+0000","last_fullsized":"2024-09-10T10:29:05.893191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832375+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:29:05.832375+0000","last_peered":"2024-09-10T10:29:05.832375+0000","last_clean":"2024-09-10T10:29:05.832375+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:29:05.832375+0000","last_undegraded":"2024-09-10T10:29:05.832375+0000","last_fullsized":"2024-09-10T10:29:05.832375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688489+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:29:05.688489+0000","last_peered":"2024-09-10T10:29:05.688489+0000","last_clean":"2024-09-10T10:29:05.688489+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:29:05.688489+0000","last_undegraded":"2024-09-10T10:29:05.688489+0000","last_fullsized":"2024-09-10T10:29:05.688489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485637+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:29:05.485637+0000","last_peered":"2024-09-10T10:29:05.485637+0000","last_clean":"2024-09-10T10:29:05.485637+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:29:05.485637+0000","last_undegraded":"2024-09-10T10:29:05.485637+0000","last_fullsized":"2024-09-10T10:29:05.485637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832269+0000","last_change":"2024-09-10T10:28:59.446917+0000","last_active":"2024-09-10T10:29:05.832269+0000","last_peered":"2024-09-10T10:29:05.832269+0000","last_clean":"2024-09-10T10:29:05.832269+0000","last_became_active":"2024-09-10T10:28:59.446546+0000","last_became_peered":"2024-09-10T10:28:59.446546+0000","last_unstale":"2024-09-10T10:29:05.832269+0000","last_undegraded":"2024-09-10T10:29:05.832269+0000","last_fullsized":"2024-09-10T10:29:05.832269+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:30.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485795+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:29:05.485795+0000","last_peered":"2024-09-10T10:29:05.485795+0000","last_clean":"2024-09-10T10:29:05.485795+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:29:05.485795+0000","last_undegraded":"2024-09-10T10:29:05.485795+0000","last_fullsized":"2024-09-10T10:29:05.485795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831886+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:29:05.831886+0000","last_peered":"2024-09-10T10:29:05.831886+0000","last_clean":"2024-09-10T10:29:05.831886+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:29:05.831886+0000","last_undegraded":"2024-09-10T10:29:05.831886+0000","last_fullsized":"2024-09-10T10:29:05.831886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486218+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:29:05.486218+0000","last_peered":"2024-09-10T10:29:05.486218+0000","last_clean":"2024-09-10T10:29:05.486218+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:29:05.486218+0000","last_undegraded":"2024-09-10T10:29:05.486218+0000","last_fullsized":"2024-09-10T10:29:05.486218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892472+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:29:05.892472+0000","last_peered":"2024-09-10T10:29:05.892472+0000","last_clean":"2024-09-10T10:29:05.892472+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:29:05.892472+0000","last_undegraded":"2024-09-10T10:29:05.892472+0000","last_fullsized":"2024-09-10T10:29:05.892472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688898+0000","last_change":"2024-09-10T10:28:23.249392+0000","last_active":"2024-09-10T10:29:05.688898+0000","last_peered":"2024-09-10T10:29:05.688898+0000","last_clean":"2024-09-10T10:29:05.688898+0000","last_became_active":"2024-09-10T10:28:23.249039+0000","last_became_peered":"2024-09-10T10:28:23.249039+0000","last_unstale":"2024-09-10T10:29:05.688898+0000","last_undegraded":"2024-09-10T10:29:05.688898+0000","last_fullsized":"2024-09-10T10:29:05.688898+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:46.656488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485234+0000","last_change":"2024-09-10T10:28:24.244885+0000","last_active":"2024-09-10T10:29:05.485234+0000","last_peered":"2024-09-10T10:29:05.485234+0000","last_clean":"2024-09-10T10:29:05.485234+0000","last_became_active":"2024-09-10T10:28:24.244560+0000","last_became_peered":"2024-09-10T10:28:24.244560+0000","last_unstale":"2024-09-10T10:29:05.485234+0000","last_undegraded":"2024-09-10T10:29:05.485234+0000","last_fullsized":"2024-09-10T10:29:05.485234+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:36.642112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483017+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:29:05.483017+0000","last_peered":"2024-09-10T10:29:05.483017+0000","last_clean":"2024-09-10T10:29:05.483017+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:29:05.483017+0000","last_undegraded":"2024-09-10T10:29:05.483017+0000","last_fullsized":"2024-09-10T10:29:05.483017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486768+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:29:05.486768+0000","last_peered":"2024-09-10T10:29:05.486768+0000","last_clean":"2024-09-10T10:29:05.486768+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:29:05.486768+0000","last_undegraded":"2024-09-10T10:29:05.486768+0000","last_fullsized":"2024-09-10T10:29:05.486768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485724+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:29:05.485724+0000","last_peered":"2024-09-10T10:29:05.485724+0000","last_clean":"2024-09-10T10:29:05.485724+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:29:05.485724+0000","last_undegraded":"2024-09-10T10:29:05.485724+0000","last_fullsized":"2024-09-10T10:29:05.485724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688424+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:29:05.688424+0000","last_peered":"2024-09-10T10:29:05.688424+0000","last_clean":"2024-09-10T10:29:05.688424+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:29:05.688424+0000","last_undegraded":"2024-09-10T10:29:05.688424+0000","last_fullsized":"2024-09-10T10:29:05.688424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"60'1","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892613+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:29:05.892613+0000","last_peered":"2024-09-10T10:29:05.892613+0000","last_clean":"2024-09-10T10:29:05.892613+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:29:05.892613+0000","last_undegraded":"2024-09-10T10:29:05.892613+0000","last_fullsized":"2024-09-10T10:29:05.892613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482510+0000","last_change":"2024-09-10T10:28:24.243479+0000","last_active":"2024-09-10T10:29:05.482510+0000","last_peered":"2024-09-10T10:29:05.482510+0000","last_clean":"2024-09-10T10:29:05.482510+0000","last_became_active":"2024-09-10T10:28:24.243088+0000","last_became_peered":"2024-09-10T10:28:24.243088+0000","last_unstale":"2024-09-10T10:29:05.482510+0000","last_undegraded":"2024-09-10T10:29:05.482510+0000","last_fullsized":"2024-09-10T10:29:05.482510+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:39.465491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":86,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831975+0000","last_change":"2024-09-10T10:27:58.327792+0000","last_active":"2024-09-10T10:29:05.831975+0000","last_peered":"2024-09-10T10:29:05.831975+0000","last_clean":"2024-09-10T10:29:05.831975+0000","last_became_active":"2024-09-10T10:27:58.327502+0000","last_became_peered":"2024-09-10T10:27:58.327502+0000","last_unstale":"2024-09-10T10:29:05.831975+0000","last_undegraded":"2024-09-10T10:29:05.831975+0000","last_fullsized":"2024-09-10T10:29:05.831975+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485354+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:29:05.485354+0000","last_peered":"2024-09-10T10:29:05.485354+0000","last_clean":"2024-09-10T10:29:05.485354+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:29:05.485354+0000","last_undegraded":"2024-09-10T10:29:05.485354+0000","last_fullsized":"2024-09-10T10:29:05.485354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485527+0000","last_change":"2024-09-10T10:28:40.345087+0000","last_active":"2024-09-10T10:29:05.485527+0000","last_peered":"2024-09-10T10:29:05.485527+0000","last_clean":"2024-09-10T10:29:05.485527+0000","last_became_active":"2024-09-10T10:28:40.344766+0000","last_became_peered":"2024-09-10T10:28:40.344766+0000","last_unstale":"2024-09-10T10:29:05.485527+0000","last_undegraded":"2024-09-10T10:29:05.485527+0000","last_fullsized":"2024-09-10T10:29:05.485527+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521337+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:29:05.521337+0000","last_peered":"2024-09-10T10:29:05.521337+0000","last_clean":"2024-09-10T10:29:05.521337+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:29:05.521337+0000","last_undegraded":"2024-09-10T10:29:05.521337+0000","last_fullsized":"2024-09-10T10:29:05.521337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831395+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:29:05.831395+0000","last_peered":"2024-09-10T10:29:05.831395+0000","last_clean":"2024-09-10T10:29:05.831395+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:29:05.831395+0000","last_undegraded":"2024-09-10T10:29:05.831395+0000","last_fullsized":"2024-09-10T10:29:05.831395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520601+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:29:05.520601+0000","last_peered":"2024-09-10T10:29:05.520601+0000","last_clean":"2024-09-10T10:29:05.520601+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:29:05.520601+0000","last_undegraded":"2024-09-10T10:29:05.520601+0000","last_fullsized":"2024-09-10T10:29:05.520601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486096+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:29:05.486096+0000","last_peered":"2024-09-10T10:29:05.486096+0000","last_clean":"2024-09-10T10:29:05.486096+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:29:05.486096+0000","last_undegraded":"2024-09-10T10:29:05.486096+0000","last_fullsized":"2024-09-10T10:29:05.486096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520639+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:29:05.520639+0000","last_peered":"2024-09-10T10:29:05.520639+0000","last_clean":"2024-09-10T10:29:05.520639+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:29:05.520639+0000","last_undegraded":"2024-09-10T10:29:05.520639+0000","last_fullsized":"2024-09-10T10:29:05.520639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486354+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:29:05.486354+0000","last_peered":"2024-09-10T10:29:05.486354+0000","last_clean":"2024-09-10T10:29:05.486354+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:29:05.486354+0000","last_undegraded":"2024-09-10T10:29:05.486354+0000","last_fullsized":"2024-09-10T10:29:05.486354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892681+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:29:05.892681+0000","last_peered":"2024-09-10T10:29:05.892681+0000","last_clean":"2024-09-10T10:29:05.892681+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:29:05.892681+0000","last_undegraded":"2024-09-10T10:29:05.892681+0000","last_fullsized":"2024-09-10T10:29:05.892681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688961+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:29:05.688961+0000","last_peered":"2024-09-10T10:29:05.688961+0000","last_clean":"2024-09-10T10:29:05.688961+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:29:05.688961+0000","last_undegraded":"2024-09-10T10:29:05.688961+0000","last_fullsized":"2024-09-10T10:29:05.688961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521230+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:29:05.521230+0000","last_peered":"2024-09-10T10:29:05.521230+0000","last_clean":"2024-09-10T10:29:05.521230+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:29:05.521230+0000","last_undegraded":"2024-09-10T10:29:05.521230+0000","last_fullsized":"2024-09-10T10:29:05.521230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44104,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":247,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":88208,"data_compressed":3324,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":995328,"data_stored":1059983,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":974848,"data_stored":519451,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":966656,"data_stored":485579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":475783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":512959,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":954368,"data_stored":483521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":933888,"data_stored":475985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":1065695,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":38506,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":37804,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:29:09.386 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-10T10:29:09.386 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-10T10:29:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.521+0000 7f0357931640 1 Processor -- start 2024-09-10T10:29:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.521+0000 7f0357931640 1 -- start start 2024-09-10T10:29:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.522+0000 7f0357931640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0350154be0 0x7f035015b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.522+0000 7f0357931640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f035006fdb0 0x7f03500c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.522+0000 7f0357931640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03500c99b0 con 0x7f03500c7320 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.522+0000 7f0357931640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f03500c92b0 con 0x7f035006fdb0 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.522+0000 7f0357931640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f035015aec0 con 0x7f0350154be0 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.522+0000 7f0354ecc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f035006fdb0 0x7f03500c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.522+0000 7f0354ecc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f035006fdb0 0x7f03500c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42418/0 (socket says 172.21.15.120:42418) 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.522+0000 7f0354ecc640 1 -- 172.21.15.120:0/814021875 learned_addr learned my addr 172.21.15.120:0/814021875 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.523+0000 7f03556cd640 1 --2- 172.21.15.120:0/814021875 >> v2:172.21.15.134:3300/0 conn(0x7f0350154be0 0x7f035015b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.523+0000 7f0354ecc640 1 -- 172.21.15.120:0/814021875 >> v2:172.21.15.134:3300/0 conn(0x7f0350154be0 msgr2=0x7f035015b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.523+0000 7f0354ecc640 1 --2- 172.21.15.120:0/814021875 >> v2:172.21.15.134:3300/0 conn(0x7f0350154be0 0x7f035015b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.523+0000 7f0354ecc640 1 -- 172.21.15.120:0/814021875 >> v1:172.21.15.134:6789/0 conn(0x7f03500c7320 legacy=0x7f035015df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.523+0000 7f0354ecc640 1 -- 172.21.15.120:0/814021875 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03500c8cb0 con 0x7f035006fdb0 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.523+0000 7f03556cd640 1 --2- 172.21.15.120:0/814021875 >> v2:172.21.15.134:3300/0 conn(0x7f0350154be0 0x7f035015b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.523+0000 7f0354ecc640 1 --2- 172.21.15.120:0/814021875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f035006fdb0 0x7f03500c9c30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f035015b040 tx=0x7f034002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfbd38ef40db2b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:09.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.523+0000 7f0347fff640 1 -- 172.21.15.120:0/814021875 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340037070 con 0x7f035006fdb0 2024-09-10T10:29:09.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.524+0000 7f0347fff640 1 -- 172.21.15.120:0/814021875 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0340032440 con 0x7f035006fdb0 2024-09-10T10:29:09.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.524+0000 7f0347fff640 1 -- 172.21.15.120:0/814021875 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340032810 con 0x7f035006fdb0 2024-09-10T10:29:09.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.524+0000 7f0357931640 1 -- 172.21.15.120:0/814021875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f035006fdb0 msgr2=0x7f03500c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.524+0000 7f0357931640 1 --2- 172.21.15.120:0/814021875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f035006fdb0 0x7f03500c9c30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f035015b040 tx=0x7f034002a2c0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.524+0000 7f0357931640 1 -- 172.21.15.120:0/814021875 shutdown_connections 2024-09-10T10:29:09.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.524+0000 7f0357931640 1 --2- 172.21.15.120:0/814021875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f035006fdb0 0x7f03500c9c30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.524+0000 7f0357931640 1 --2- 172.21.15.120:0/814021875 >> v2:172.21.15.134:3300/0 conn(0x7f0350154be0 0x7f035015b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.524+0000 7f0357931640 1 -- 172.21.15.120:0/814021875 >> 172.21.15.120:0/814021875 conn(0x7f03500680e0 msgr2=0x7f0350070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.524+0000 7f0357931640 1 -- 172.21.15.120:0/814021875 shutdown_connections 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.524+0000 7f0357931640 1 -- 172.21.15.120:0/814021875 wait complete. 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.525+0000 7f0357931640 1 Processor -- start 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.525+0000 7f0357931640 1 -- start start 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0357931640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f035006fdb0 0x7f03501507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0357931640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03500c7320 0x7f035014ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0357931640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350154be0 0x7f035015c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0357931640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f035015b890 con 0x7f0350154be0 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0357931640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f035015ab50 con 0x7f03500c7320 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0357931640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0350154fc0 con 0x7f035006fdb0 2024-09-10T10:29:09.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0355ece640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350154be0 0x7f035015c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0355ece640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350154be0 0x7f035015c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42422/0 (socket says 172.21.15.120:42422) 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0355ece640 1 -- 172.21.15.120:0/1287519819 learned_addr learned my addr 172.21.15.120:0/1287519819 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f03556cd640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f035006fdb0 0x7f03501507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0355ece640 1 -- 172.21.15.120:0/1287519819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f035006fdb0 msgr2=0x7f03501507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0355ece640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f035006fdb0 0x7f03501507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0355ece640 1 -- 172.21.15.120:0/1287519819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03500c7320 msgr2=0x7f035014ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0355ece640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03500c7320 0x7f035014ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f0355ece640 1 -- 172.21.15.120:0/1287519819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035014fa40 con 0x7f0350154be0 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.526+0000 7f03556cd640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f035006fdb0 0x7f03501507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.527+0000 7f0355ece640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350154be0 0x7f035015c520 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f034800c410 tx=0x7f034800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.527+0000 7f0345ffb640 1 -- 172.21.15.120:0/1287519819 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348011790 con 0x7f0350154be0 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.527+0000 7f0345ffb640 1 -- 172.21.15.120:0/1287519819 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0348017070 con 0x7f0350154be0 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.527+0000 7f0345ffb640 1 -- 172.21.15.120:0/1287519819 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348011bb0 con 0x7f0350154be0 2024-09-10T10:29:09.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.527+0000 7f0357931640 1 -- 172.21.15.120:0/1287519819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035014f480 con 0x7f0350154be0 2024-09-10T10:29:09.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.527+0000 7f0357931640 1 -- 172.21.15.120:0/1287519819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035014fcf0 con 0x7f0350154be0 2024-09-10T10:29:09.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.528+0000 7f0357931640 1 -- 172.21.15.120:0/1287519819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035006c5b0 con 0x7f0350154be0 2024-09-10T10:29:09.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.528+0000 7f0345ffb640 1 -- 172.21.15.120:0/1287519819 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f034801c460 con 0x7f0350154be0 2024-09-10T10:29:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.529+0000 7f0345ffb640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f033406be60 0x7f033406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.529+0000 7f0345ffb640 1 -- 172.21.15.120:0/1287519819 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f034808aef0 con 0x7f0350154be0 2024-09-10T10:29:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.529+0000 7f03556cd640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f033406be60 0x7f033406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.529+0000 7f03556cd640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f033406be60 0x7f033406e320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f034c00e3d0 tx=0x7f034c007a10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:09.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.533+0000 7f0345ffb640 1 -- 172.21.15.120:0/1287519819 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0348021020 con 0x7f0350154be0 2024-09-10T10:29:09.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.699+0000 7f0357931640 1 -- 172.21.15.120:0/1287519819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} v 0) -- 0x7f03500cba60 con 0x7f0350154be0 2024-09-10T10:29:09.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.702+0000 7f0345ffb640 1 -- 172.21.15.120:0/1287519819 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]=0 v11) ==== 118+0+0 (secure 0 0 0) 0x7f03480593a0 con 0x7f0350154be0 2024-09-10T10:29:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 -- 172.21.15.120:0/1287519819 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f033406be60 msgr2=0x7f033406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f033406be60 0x7f033406e320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f034c00e3d0 tx=0x7f034c007a10 comp rx=0 tx=0).stop 2024-09-10T10:29:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 -- 172.21.15.120:0/1287519819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350154be0 msgr2=0x7f035015c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350154be0 0x7f035015c520 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f034800c410 tx=0x7f034800c8d0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 -- 172.21.15.120:0/1287519819 shutdown_connections 2024-09-10T10:29:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f033406be60 0x7f033406e320 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350154be0 0x7f035015c520 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03500c7320 0x7f035014ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 --2- 172.21.15.120:0/1287519819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f035006fdb0 0x7f03501507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 -- 172.21.15.120:0/1287519819 >> 172.21.15.120:0/1287519819 conn(0x7f03500680e0 msgr2=0x7f03500cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:09.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 -- 172.21.15.120:0/1287519819 shutdown_connections 2024-09-10T10:29:09.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.705+0000 7f0357931640 1 -- 172.21.15.120:0/1287519819 wait complete. 2024-09-10T10:29:09.717 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T10:29:09.727 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T10:29:09.727 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:29:09.727 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:29:09.743 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T10:29:09.743 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:29:09.798 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:29:09.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.937+0000 7fa46a94b640 1 Processor -- start 2024-09-10T10:29:09.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.937+0000 7fa46a94b640 1 -- start start 2024-09-10T10:29:09.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa46a94b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa464154f50 0x7fa4640cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa46a94b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cb500 0x7fa46415e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa46a94b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4640c9db0 con 0x7fa464154b70 2024-09-10T10:29:09.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa46a94b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa46406d4e0 con 0x7fa464154f50 2024-09-10T10:29:09.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa46a94b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa4640ca5d0 con 0x7fa4640cb500 2024-09-10T10:29:09.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa463fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fa464154b70 0x7fa4640c96a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41552/0 (socket says 172.21.15.120:41552) 2024-09-10T10:29:09.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa463fff640 1 -- 172.21.15.120:0/783279335 learned_addr learned my addr 172.21.15.120:0/783279335 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:09.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa4637fe640 1 --2- 172.21.15.120:0/783279335 >> v2:172.21.15.134:3300/0 conn(0x7fa464154f50 0x7fa4640cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa468ee8640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cb500 0x7fa46415e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099584632 0 0) 0x7fa4640c9db0 con 0x7fa464154b70 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa44c003520 con 0x7fa464154b70 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2521881994 0 0) 0x7fa44c003520 con 0x7fa464154b70 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cb500 msgr2=0x7fa46415e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cb500 0x7fa46415e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 >> v2:172.21.15.134:3300/0 conn(0x7fa464154f50 msgr2=0x7fa4640cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 --2- 172.21.15.120:0/783279335 >> v2:172.21.15.134:3300/0 conn(0x7fa464154f50 0x7fa4640cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4640ca890 con 0x7fa464154b70 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7fa454001670 con 0x7fa464154b70 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 >> v1:172.21.15.120:6789/0 conn(0x7fa464154b70 legacy=0x7fa4640c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cb500 0x7fa44c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 0x7fa44c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa44c004d40 0x7fa44c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa46406d4e0 con 0x7fa44c003f30 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4640ca5d0 con 0x7fa44c004d40 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.938+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c00b9f0 con 0x7fa4640cb500 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.939+0000 7fa4637fe640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 0x7fa44c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.939+0000 7fa463fff640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cb500 0x7fa44c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.939+0000 7fa468ee8640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa44c004d40 0x7fa44c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.939+0000 7fa4637fe640 1 -- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cb500 msgr2=0x7fa44c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.939+0000 7fa4637fe640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cb500 0x7fa44c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.939+0000 7fa4637fe640 1 -- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa44c004d40 msgr2=0x7fa44c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.939+0000 7fa4637fe640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa44c004d40 0x7fa44c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.939+0000 7fa4637fe640 1 -- 172.21.15.120:0/783279335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c003520 con 0x7fa44c003f30 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.939+0000 7fa4637fe640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 0x7fa44c004800 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa458009ac0 tx=0x7fa44c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731f08e9a889ce53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.940+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458036070 con 0x7fa44c003f30 2024-09-10T10:29:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.940+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa458031400 con 0x7fa44c003f30 2024-09-10T10:29:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.940+0000 7fa462ffd640 1 -- 172.21.15.120:0/783279335 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458009690 con 0x7fa44c003f30 2024-09-10T10:29:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.940+0000 7fa46a94b640 1 -- 172.21.15.120:0/783279335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 msgr2=0x7fa44c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.940+0000 7fa46a94b640 1 --2- 172.21.15.120:0/783279335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 0x7fa44c004800 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa458009ac0 tx=0x7fa44c005830 comp rx=0 tx=0).stop 2024-09-10T10:29:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.940+0000 7fa468ee8640 1 -- 172.21.15.120:0/783279335 reap_dead start 2024-09-10T10:29:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.940+0000 7fa46a94b640 1 -- 172.21.15.120:0/783279335 shutdown_connections 2024-09-10T10:29:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.940+0000 7fa46a94b640 1 -- 172.21.15.120:0/783279335 >> 172.21.15.120:0/783279335 conn(0x7fa46406bc10 msgr2=0x7fa46406fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.940+0000 7fa46a94b640 1 -- 172.21.15.120:0/783279335 shutdown_connections 2024-09-10T10:29:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.940+0000 7fa46a94b640 1 -- 172.21.15.120:0/783279335 wait complete. 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.941+0000 7fa46a94b640 1 Processor -- start 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.942+0000 7fa46a94b640 1 -- start start 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.942+0000 7fa46a94b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 0x7fa46414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.942+0000 7fa46a94b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4641f16b0 0x7fa4640c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.942+0000 7fa46a94b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cd400 0x7fa46414a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.942+0000 7fa46a94b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa46406e550 con 0x7fa44c003f30 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.942+0000 7fa46a94b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa46406d170 con 0x7fa4641f16b0 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.942+0000 7fa46a94b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa46406ecc0 con 0x7fa4640cd400 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.942+0000 7fa463fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 0x7fa46414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.942+0000 7fa463fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 0x7fa46414cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42444/0 (socket says 172.21.15.120:42444) 2024-09-10T10:29:09.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.942+0000 7fa463fff640 1 -- 172.21.15.120:0/3774748907 learned_addr learned my addr 172.21.15.120:0/3774748907 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa468ee8640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cd400 0x7fa46414a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa4637fe640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4641f16b0 0x7fa4640c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa463fff640 1 -- 172.21.15.120:0/3774748907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cd400 msgr2=0x7fa46414a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa463fff640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cd400 0x7fa46414a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa463fff640 1 -- 172.21.15.120:0/3774748907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4641f16b0 msgr2=0x7fa4640c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa463fff640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4641f16b0 0x7fa4640c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa463fff640 1 -- 172.21.15.120:0/3774748907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4640c6ba0 con 0x7fa44c003f30 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa468ee8640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cd400 0x7fa46414a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa463fff640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 0x7fa46414cca0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa454007340 tx=0x7fa454007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa460ff9640 1 -- 172.21.15.120:0/3774748907 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45400f960 con 0x7fa44c003f30 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa460ff9640 1 -- 172.21.15.120:0/3774748907 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa454018070 con 0x7fa44c003f30 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa46a94b640 1 -- 172.21.15.120:0/3774748907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4640c8190 con 0x7fa44c003f30 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa460ff9640 1 -- 172.21.15.120:0/3774748907 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45400fd40 con 0x7fa44c003f30 2024-09-10T10:29:09.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.943+0000 7fa46a94b640 1 -- 172.21.15.120:0/3774748907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4641f0bd0 con 0x7fa44c003f30 2024-09-10T10:29:09.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.944+0000 7fa46a94b640 1 -- 172.21.15.120:0/3774748907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa46406c620 con 0x7fa44c003f30 2024-09-10T10:29:09.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.945+0000 7fa460ff9640 1 -- 172.21.15.120:0/3774748907 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa45401d460 con 0x7fa44c003f30 2024-09-10T10:29:09.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.945+0000 7fa460ff9640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa45006be30 0x7fa45006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:09.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.945+0000 7fa460ff9640 1 -- 172.21.15.120:0/3774748907 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fa45408b3f0 con 0x7fa44c003f30 2024-09-10T10:29:09.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.945+0000 7fa4637fe640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa45006be30 0x7fa45006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:09.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.945+0000 7fa4637fe640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa45006be30 0x7fa45006e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa458033000 tx=0x7fa458008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:09.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:09.949+0000 7fa460ff9640 1 -- 172.21.15.120:0/3774748907 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa454059930 con 0x7fa44c003f30 2024-09-10T10:29:10.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.127+0000 7fa46a94b640 1 -- 172.21.15.120:0/3774748907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa4641f09e0 con 0x7fa44c003f30 2024-09-10T10:29:10.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.128+0000 7fa460ff9640 1 -- 172.21.15.120:0/3774748907 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) ==== 80+0+0 (secure 0 0 0) 0x7fa454059230 con 0x7fa44c003f30 2024-09-10T10:29:10.129 INFO:teuthology.orchestra.run.smithi120.stderr:listed 0 entries 2024-09-10T10:29:10.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 -- 172.21.15.120:0/3774748907 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa45006be30 msgr2=0x7fa45006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa45006be30 0x7fa45006e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa458033000 tx=0x7fa458008030 comp rx=0 tx=0).stop 2024-09-10T10:29:10.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 -- 172.21.15.120:0/3774748907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 msgr2=0x7fa46414cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 0x7fa46414cca0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa454007340 tx=0x7fa454007800 comp rx=0 tx=0).stop 2024-09-10T10:29:10.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 -- 172.21.15.120:0/3774748907 shutdown_connections 2024-09-10T10:29:10.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fa45006be30 0x7fa45006e2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4640cd400 0x7fa46414a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4641f16b0 0x7fa4640c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 --2- 172.21.15.120:0/3774748907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44c003f30 0x7fa46414cca0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 -- 172.21.15.120:0/3774748907 >> 172.21.15.120:0/3774748907 conn(0x7fa46406bc10 msgr2=0x7fa4640ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:10.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 -- 172.21.15.120:0/3774748907 shutdown_connections 2024-09-10T10:29:10.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.130+0000 7fa46a94b640 1 -- 172.21.15.120:0/3774748907 wait complete. 2024-09-10T10:29:10.142 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:29:10.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:29:10.142 DEBUG:teuthology.orchestra.run.smithi120:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:29:10.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:29:10.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:29:10.190 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:29:10.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:29:10.211 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-09-10T10:29:10.266 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:29:10.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:29:10.267 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:29:10.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:29:10.333 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-09-10T10:29:10.428 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T10:29:10.437 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:29:10.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:29:10.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.535+0000 7f53540f1640 1 Processor -- start 2024-09-10T10:29:10.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.536+0000 7f53540f1640 1 -- start start 2024-09-10T10:29:10.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.536+0000 7f53540f1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f534c0c98d0 0x7f534c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.536+0000 7f53540f1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f534c150bf0 0x7f534c151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.536+0000 7f53540f1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f534c1578a0 con 0x7f534c071b30 2024-09-10T10:29:10.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.536+0000 7f53540f1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f534c0c7a80 con 0x7f534c150bf0 2024-09-10T10:29:10.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.536+0000 7f53540f1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f534c072620 con 0x7f534c0c98d0 2024-09-10T10:29:10.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.537+0000 7f535268e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f534c150bf0 0x7f534c151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.537+0000 7f535268e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f534c150bf0 0x7f534c151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42460/0 (socket says 172.21.15.120:42460) 2024-09-10T10:29:10.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.537+0000 7f535268e640 1 -- 172.21.15.120:0/2647595661 learned_addr learned my addr 172.21.15.120:0/2647595661 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:10.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.537+0000 7f535168c640 1 --2- 172.21.15.120:0/2647595661 >> v2:172.21.15.134:3300/0 conn(0x7f534c0c98d0 0x7f534c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.537+0000 7f535268e640 1 -- 172.21.15.120:0/2647595661 >> v2:172.21.15.134:3300/0 conn(0x7f534c0c98d0 msgr2=0x7f534c0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:10.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.537+0000 7f535268e640 1 --2- 172.21.15.120:0/2647595661 >> v2:172.21.15.134:3300/0 conn(0x7f534c0c98d0 0x7f534c0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.537+0000 7f535268e640 1 -- 172.21.15.120:0/2647595661 >> v1:172.21.15.134:6789/0 conn(0x7f534c071b30 legacy=0x7f534c071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:10.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.537+0000 7f535268e640 1 -- 172.21.15.120:0/2647595661 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c072f30 con 0x7f534c150bf0 2024-09-10T10:29:10.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.537+0000 7f535168c640 1 --2- 172.21.15.120:0/2647595661 >> v2:172.21.15.134:3300/0 conn(0x7f534c0c98d0 0x7f534c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:10.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f535268e640 1 --2- 172.21.15.120:0/2647595661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c150bf0 0x7f534c151020 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f534c154990 tx=0x7f533c02b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a12e50654a8f430 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:10.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f5350e8b640 1 -- 172.21.15.120:0/2647595661 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c038070 con 0x7f534c150bf0 2024-09-10T10:29:10.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f5350e8b640 1 -- 172.21.15.120:0/2647595661 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f533c033640 con 0x7f534c150bf0 2024-09-10T10:29:10.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f5350e8b640 1 -- 172.21.15.120:0/2647595661 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c033a10 con 0x7f534c150bf0 2024-09-10T10:29:10.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f53540f1640 1 -- 172.21.15.120:0/2647595661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c150bf0 msgr2=0x7f534c151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f53540f1640 1 --2- 172.21.15.120:0/2647595661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c150bf0 0x7f534c151020 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f534c154990 tx=0x7f533c02b3d0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f53540f1640 1 -- 172.21.15.120:0/2647595661 shutdown_connections 2024-09-10T10:29:10.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f53540f1640 1 --2- 172.21.15.120:0/2647595661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c150bf0 0x7f534c151020 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f53540f1640 1 --2- 172.21.15.120:0/2647595661 >> v2:172.21.15.134:3300/0 conn(0x7f534c0c98d0 0x7f534c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f53540f1640 1 -- 172.21.15.120:0/2647595661 >> 172.21.15.120:0/2647595661 conn(0x7f534c06f3c0 msgr2=0x7f534c06f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f53540f1640 1 -- 172.21.15.120:0/2647595661 shutdown_connections 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.538+0000 7f53540f1640 1 -- 172.21.15.120:0/2647595661 wait complete. 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.539+0000 7f53540f1640 1 Processor -- start 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.540+0000 7f53540f1640 1 -- start start 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.540+0000 7f53540f1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c071b30 0x7f534c1efeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.540+0000 7f53540f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c0c98d0 0x7f534c146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.540+0000 7f53540f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f534c150bf0 0x7f534c14fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.540+0000 7f53540f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f534c0c8cb0 con 0x7f534c0c98d0 2024-09-10T10:29:10.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.540+0000 7f53540f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f534c0731f0 con 0x7f534c150bf0 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.540+0000 7f53540f1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f534c0c8e30 con 0x7f534c071b30 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.540+0000 7f535168c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c0c98d0 0x7f534c146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.540+0000 7f535168c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c0c98d0 0x7f534c146da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42464/0 (socket says 172.21.15.120:42464) 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.540+0000 7f535168c640 1 -- 172.21.15.120:0/64170304 learned_addr learned my addr 172.21.15.120:0/64170304 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f535268e640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f534c150bf0 0x7f534c14fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f5351e8d640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c071b30 0x7f534c1efeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f535168c640 1 -- 172.21.15.120:0/64170304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c071b30 msgr2=0x7f534c1efeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f535168c640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c071b30 0x7f534c1efeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f535168c640 1 -- 172.21.15.120:0/64170304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f534c150bf0 msgr2=0x7f534c14fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f535168c640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f534c150bf0 0x7f534c14fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f535168c640 1 -- 172.21.15.120:0/64170304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c1f03f0 con 0x7f534c0c98d0 2024-09-10T10:29:10.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f535268e640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f534c150bf0 0x7f534c14fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:10.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f535168c640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c0c98d0 0x7f534c146da0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f534000e0d0 tx=0x7f534000e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:10.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f534a7fc640 1 -- 172.21.15.120:0/64170304 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534000c0d0 con 0x7f534c0c98d0 2024-09-10T10:29:10.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f534a7fc640 1 -- 172.21.15.120:0/64170304 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534000c270 con 0x7f534c0c98d0 2024-09-10T10:29:10.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f53540f1640 1 -- 172.21.15.120:0/64170304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c154f20 con 0x7f534c0c98d0 2024-09-10T10:29:10.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f534a7fc640 1 -- 172.21.15.120:0/64170304 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534000a750 con 0x7f534c0c98d0 2024-09-10T10:29:10.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.541+0000 7f53540f1640 1 -- 172.21.15.120:0/64170304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c155840 con 0x7f534c0c98d0 2024-09-10T10:29:10.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.542+0000 7f53540f1640 1 -- 172.21.15.120:0/64170304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534c0c9220 con 0x7f534c0c98d0 2024-09-10T10:29:10.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.542+0000 7f534a7fc640 1 -- 172.21.15.120:0/64170304 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f534000a8f0 con 0x7f534c0c98d0 2024-09-10T10:29:10.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.543+0000 7f534a7fc640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f533806be30 0x7f533806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.543+0000 7f534a7fc640 1 -- 172.21.15.120:0/64170304 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f534008da40 con 0x7f534c0c98d0 2024-09-10T10:29:10.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.543+0000 7f5351e8d640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f533806be30 0x7f533806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.543+0000 7f5351e8d640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f533806be30 0x7f533806e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5344007fd0 tx=0x7f5344009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:10.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.547+0000 7f534a7fc640 1 -- 172.21.15.120:0/64170304 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f534005be60 con 0x7f534c0c98d0 2024-09-10T10:29:10.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.711+0000 7f53540f1640 1 -- 172.21.15.120:0/64170304 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f534c0720b0 con 0x7f533806be30 2024-09-10T10:29:10.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.718+0000 7f534a7fc640 1 -- 172.21.15.120:0/64170304 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332899 (secure 0 0 0) 0x7f534c0720b0 con 0x7f533806be30 2024-09-10T10:29:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:10.724 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:29:10.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 -- 172.21.15.120:0/64170304 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f533806be30 msgr2=0x7f533806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f533806be30 0x7f533806e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5344007fd0 tx=0x7f5344009040 comp rx=0 tx=0).stop 2024-09-10T10:29:10.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 -- 172.21.15.120:0/64170304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c0c98d0 msgr2=0x7f534c146da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c0c98d0 0x7f534c146da0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f534000e0d0 tx=0x7f534000e590 comp rx=0 tx=0).stop 2024-09-10T10:29:10.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 -- 172.21.15.120:0/64170304 shutdown_connections 2024-09-10T10:29:10.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f533806be30 0x7f533806e2f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f534c150bf0 0x7f534c14fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c0c98d0 0x7f534c146da0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 --2- 172.21.15.120:0/64170304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c071b30 0x7f534c1efeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 -- 172.21.15.120:0/64170304 >> 172.21.15.120:0/64170304 conn(0x7f534c06f3c0 msgr2=0x7f534c0c5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:10.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 -- 172.21.15.120:0/64170304 shutdown_connections 2024-09-10T10:29:10.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.722+0000 7f53540f1640 1 -- 172.21.15.120:0/64170304 wait complete. 2024-09-10T10:29:10.734 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-10T10:29:10.120811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634472,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":119,"num_read_kb":103,"num_write":408,"num_write_kb":2539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":228028,"kb_used_data":7616,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754746692,"statfs":{"total":773094113280,"available":772860612608,"internally_reserved":0,"allocated":7798784,"data_stored":5122120,"data_compressed":68176,"data_compressed_allocated":98304,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.656364"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893394+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:29:05.893394+0000","last_peered":"2024-09-10T10:29:05.893394+0000","last_clean":"2024-09-10T10:29:05.893394+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:29:05.893394+0000","last_undegraded":"2024-09-10T10:29:05.893394+0000","last_fullsized":"2024-09-10T10:29:05.893394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831510+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:29:05.831510+0000","last_peered":"2024-09-10T10:29:05.831510+0000","last_clean":"2024-09-10T10:29:05.831510+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:29:05.831510+0000","last_undegraded":"2024-09-10T10:29:05.831510+0000","last_fullsized":"2024-09-10T10:29:05.831510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831742+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:29:05.831742+0000","last_peered":"2024-09-10T10:29:05.831742+0000","last_clean":"2024-09-10T10:29:05.831742+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:29:05.831742+0000","last_undegraded":"2024-09-10T10:29:05.831742+0000","last_fullsized":"2024-09-10T10:29:05.831742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832208+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:29:05.832208+0000","last_peered":"2024-09-10T10:29:05.832208+0000","last_clean":"2024-09-10T10:29:05.832208+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:29:05.832208+0000","last_undegraded":"2024-09-10T10:29:05.832208+0000","last_fullsized":"2024-09-10T10:29:05.832208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892600+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:29:05.892600+0000","last_peered":"2024-09-10T10:29:05.892600+0000","last_clean":"2024-09-10T10:29:05.892600+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:29:05.892600+0000","last_undegraded":"2024-09-10T10:29:05.892600+0000","last_fullsized":"2024-09-10T10:29:05.892600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955501+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:29:05.955501+0000","last_peered":"2024-09-10T10:29:05.955501+0000","last_clean":"2024-09-10T10:29:05.955501+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:29:05.955501+0000","last_undegraded":"2024-09-10T10:29:05.955501+0000","last_fullsized":"2024-09-10T10:29:05.955501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"60'13","reported_seq":158,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521178+0000","last_change":"2024-09-10T10:27:51.060242+0000","last_active":"2024-09-10T10:29:05.521178+0000","last_peered":"2024-09-10T10:29:05.521178+0000","last_clean":"2024-09-10T10:29:05.521178+0000","last_became_active":"2024-09-10T10:27:51.049148+0000","last_became_peered":"2024-09-10T10:27:51.049148+0000","last_unstale":"2024-09-10T10:29:05.521178+0000","last_undegraded":"2024-09-10T10:29:05.521178+0000","last_fullsized":"2024-09-10T10:29:05.521178+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.604144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486386+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:29:05.486386+0000","last_peered":"2024-09-10T10:29:05.486386+0000","last_clean":"2024-09-10T10:29:05.486386+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:29:05.486386+0000","last_undegraded":"2024-09-10T10:29:05.486386+0000","last_fullsized":"2024-09-10T10:29:05.486386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955770+0000","last_change":"2024-09-10T10:28:24.244697+0000","last_active":"2024-09-10T10:29:05.955770+0000","last_peered":"2024-09-10T10:29:05.955770+0000","last_clean":"2024-09-10T10:29:05.955770+0000","last_became_active":"2024-09-10T10:28:24.244262+0000","last_became_peered":"2024-09-10T10:28:24.244262+0000","last_unstale":"2024-09-10T10:29:05.955770+0000","last_undegraded":"2024-09-10T10:29:05.955770+0000","last_fullsized":"2024-09-10T10:29:05.955770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:50.683186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6,2],"acting":[4,0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520262+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:29:05.520262+0000","last_peered":"2024-09-10T10:29:05.520262+0000","last_clean":"2024-09-10T10:29:05.520262+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:29:05.520262+0000","last_undegraded":"2024-09-10T10:29:05.520262+0000","last_fullsized":"2024-09-10T10:29:05.520262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":110,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482709+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:29:05.482709+0000","last_peered":"2024-09-10T10:29:05.482709+0000","last_clean":"2024-09-10T10:29:05.482709+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:29:05.482709+0000","last_undegraded":"2024-09-10T10:29:05.482709+0000","last_fullsized":"2024-09-10T10:29:05.482709+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955722+0000","last_change":"2024-09-10T10:28:59.443607+0000","last_active":"2024-09-10T10:29:05.955722+0000","last_peered":"2024-09-10T10:29:05.955722+0000","last_clean":"2024-09-10T10:29:05.955722+0000","last_became_active":"2024-09-10T10:28:59.443172+0000","last_became_peered":"2024-09-10T10:28:59.443172+0000","last_unstale":"2024-09-10T10:29:05.955722+0000","last_undegraded":"2024-09-10T10:29:05.955722+0000","last_fullsized":"2024-09-10T10:29:05.955722+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:36.379163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485088+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:29:05.485088+0000","last_peered":"2024-09-10T10:29:05.485088+0000","last_clean":"2024-09-10T10:29:05.485088+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:29:05.485088+0000","last_undegraded":"2024-09-10T10:29:05.485088+0000","last_fullsized":"2024-09-10T10:29:05.485088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"60'5","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520220+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:29:05.520220+0000","last_peered":"2024-09-10T10:29:05.520220+0000","last_clean":"2024-09-10T10:29:05.520220+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:29:05.520220+0000","last_undegraded":"2024-09-10T10:29:05.520220+0000","last_fullsized":"2024-09-10T10:29:05.520220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956119+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:29:05.956119+0000","last_peered":"2024-09-10T10:29:05.956119+0000","last_clean":"2024-09-10T10:29:05.956119+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:29:05.956119+0000","last_undegraded":"2024-09-10T10:29:05.956119+0000","last_fullsized":"2024-09-10T10:29:05.956119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831889+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:29:05.831889+0000","last_peered":"2024-09-10T10:29:05.831889+0000","last_clean":"2024-09-10T10:29:05.831889+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:29:05.831889+0000","last_undegraded":"2024-09-10T10:29:05.831889+0000","last_fullsized":"2024-09-10T10:29:05.831889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688805+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:29:05.688805+0000","last_peered":"2024-09-10T10:29:05.688805+0000","last_clean":"2024-09-10T10:29:05.688805+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:29:05.688805+0000","last_undegraded":"2024-09-10T10:29:05.688805+0000","last_fullsized":"2024-09-10T10:29:05.688805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955843+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:29:05.955843+0000","last_peered":"2024-09-10T10:29:05.955843+0000","last_clean":"2024-09-10T10:29:05.955843+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:29:05.955843+0000","last_undegraded":"2024-09-10T10:29:05.955843+0000","last_fullsized":"2024-09-10T10:29:05.955843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831383+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:29:05.831383+0000","last_peered":"2024-09-10T10:29:05.831383+0000","last_clean":"2024-09-10T10:29:05.831383+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:29:05.831383+0000","last_undegraded":"2024-09-10T10:29:05.831383+0000","last_fullsized":"2024-09-10T10:29:05.831383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688223+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:29:05.688223+0000","last_peered":"2024-09-10T10:29:05.688223+0000","last_clean":"2024-09-10T10:29:05.688223+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:29:05.688223+0000","last_undegraded":"2024-09-10T10:29:05.688223+0000","last_fullsized":"2024-09-10T10:29:05.688223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688656+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:29:05.688656+0000","last_peered":"2024-09-10T10:29:05.688656+0000","last_clean":"2024-09-10T10:29:05.688656+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:29:05.688656+0000","last_undegraded":"2024-09-10T10:29:05.688656+0000","last_fullsized":"2024-09-10T10:29:05.688656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955038+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:29:05.955038+0000","last_peered":"2024-09-10T10:29:05.955038+0000","last_clean":"2024-09-10T10:29:05.955038+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:29:05.955038+0000","last_undegraded":"2024-09-10T10:29:05.955038+0000","last_fullsized":"2024-09-10T10:29:05.955038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520435+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:29:05.520435+0000","last_peered":"2024-09-10T10:29:05.520435+0000","last_clean":"2024-09-10T10:29:05.520435+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:29:05.520435+0000","last_undegraded":"2024-09-10T10:29:05.520435+0000","last_fullsized":"2024-09-10T10:29:05.520435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956141+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:29:05.956141+0000","last_peered":"2024-09-10T10:29:05.956141+0000","last_clean":"2024-09-10T10:29:05.956141+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:29:05.956141+0000","last_undegraded":"2024-09-10T10:29:05.956141+0000","last_fullsized":"2024-09-10T10:29:05.956141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892494+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:29:05.892494+0000","last_peered":"2024-09-10T10:29:05.892494+0000","last_clean":"2024-09-10T10:29:05.892494+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:29:05.892494+0000","last_undegraded":"2024-09-10T10:29:05.892494+0000","last_fullsized":"2024-09-10T10:29:05.892494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'8","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485743+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:29:05.485743+0000","last_peered":"2024-09-10T10:29:05.485743+0000","last_clean":"2024-09-10T10:29:05.485743+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:29:05.485743+0000","last_undegraded":"2024-09-10T10:29:05.485743+0000","last_fullsized":"2024-09-10T10:29:05.485743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955720+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:29:05.955720+0000","last_peered":"2024-09-10T10:29:05.955720+0000","last_clean":"2024-09-10T10:29:05.955720+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:29:05.955720+0000","last_undegraded":"2024-09-10T10:29:05.955720+0000","last_fullsized":"2024-09-10T10:29:05.955720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485344+0000","last_change":"2024-09-10T10:28:24.374205+0000","last_active":"2024-09-10T10:29:05.485344+0000","last_peered":"2024-09-10T10:29:05.485344+0000","last_clean":"2024-09-10T10:29:05.485344+0000","last_became_active":"2024-09-10T10:28:24.373921+0000","last_became_peered":"2024-09-10T10:28:24.373921+0000","last_unstale":"2024-09-10T10:29:05.485344+0000","last_undegraded":"2024-09-10T10:29:05.485344+0000","last_fullsized":"2024-09-10T10:29:05.485344+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:36.923293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"64'15","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688969+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:29:05.688969+0000","last_peered":"2024-09-10T10:29:05.688969+0000","last_clean":"2024-09-10T10:29:05.688969+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:29:05.688969+0000","last_undegraded":"2024-09-10T10:29:05.688969+0000","last_fullsized":"2024-09-10T10:29:05.688969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831546+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:29:05.831546+0000","last_peered":"2024-09-10T10:29:05.831546+0000","last_clean":"2024-09-10T10:29:05.831546+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:29:05.831546+0000","last_undegraded":"2024-09-10T10:29:05.831546+0000","last_fullsized":"2024-09-10T10:29:05.831546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688907+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:29:05.688907+0000","last_peered":"2024-09-10T10:29:05.688907+0000","last_clean":"2024-09-10T10:29:05.688907+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:29:05.688907+0000","last_undegraded":"2024-09-10T10:29:05.688907+0000","last_fullsized":"2024-09-10T10:29:05.688907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521226+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:29:05.521226+0000","last_peered":"2024-09-10T10:29:05.521226+0000","last_clean":"2024-09-10T10:29:05.521226+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:29:05.521226+0000","last_undegraded":"2024-09-10T10:29:05.521226+0000","last_fullsized":"2024-09-10T10:29:05.521226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893084+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:29:05.893084+0000","last_peered":"2024-09-10T10:29:05.893084+0000","last_clean":"2024-09-10T10:29:05.893084+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:29:05.893084+0000","last_undegraded":"2024-09-10T10:29:05.893084+0000","last_fullsized":"2024-09-10T10:29:05.893084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486566+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:29:05.486566+0000","last_peered":"2024-09-10T10:29:05.486566+0000","last_clean":"2024-09-10T10:29:05.486566+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:29:05.486566+0000","last_undegraded":"2024-09-10T10:29:05.486566+0000","last_fullsized":"2024-09-10T10:29:05.486566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688951+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:29:05.688951+0000","last_peered":"2024-09-10T10:29:05.688951+0000","last_clean":"2024-09-10T10:29:05.688951+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.688951+0000","last_undegraded":"2024-09-10T10:29:05.688951+0000","last_fullsized":"2024-09-10T10:29:05.688951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688451+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:29:05.688451+0000","last_peered":"2024-09-10T10:29:05.688451+0000","last_clean":"2024-09-10T10:29:05.688451+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:29:05.688451+0000","last_undegraded":"2024-09-10T10:29:05.688451+0000","last_fullsized":"2024-09-10T10:29:05.688451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956082+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:29:05.956082+0000","last_peered":"2024-09-10T10:29:05.956082+0000","last_clean":"2024-09-10T10:29:05.956082+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:29:05.956082+0000","last_undegraded":"2024-09-10T10:29:05.956082+0000","last_fullsized":"2024-09-10T10:29:05.956082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486359+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:29:05.486359+0000","last_peered":"2024-09-10T10:29:05.486359+0000","last_clean":"2024-09-10T10:29:05.486359+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:29:05.486359+0000","last_undegraded":"2024-09-10T10:29:05.486359+0000","last_fullsized":"2024-09-10T10:29:05.486359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486083+0000","last_change":"2024-09-10T10:28:24.244988+0000","last_active":"2024-09-10T10:29:05.486083+0000","last_peered":"2024-09-10T10:29:05.486083+0000","last_clean":"2024-09-10T10:29:05.486083+0000","last_became_active":"2024-09-10T10:28:24.244556+0000","last_became_peered":"2024-09-10T10:28:24.244556+0000","last_unstale":"2024-09-10T10:29:05.486083+0000","last_undegraded":"2024-09-10T10:29:05.486083+0000","last_fullsized":"2024-09-10T10:29:05.486083+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:08.127492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688685+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:29:05.688685+0000","last_peered":"2024-09-10T10:29:05.688685+0000","last_clean":"2024-09-10T10:29:05.688685+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:29:05.688685+0000","last_undegraded":"2024-09-10T10:29:05.688685+0000","last_fullsized":"2024-09-10T10:29:05.688685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'9","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482578+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:29:05.482578+0000","last_peered":"2024-09-10T10:29:05.482578+0000","last_clean":"2024-09-10T10:29:05.482578+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.482578+0000","last_undegraded":"2024-09-10T10:29:05.482578+0000","last_fullsized":"2024-09-10T10:29:05.482578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688718+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:29:05.688718+0000","last_peered":"2024-09-10T10:29:05.688718+0000","last_clean":"2024-09-10T10:29:05.688718+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:29:05.688718+0000","last_undegraded":"2024-09-10T10:29:05.688718+0000","last_fullsized":"2024-09-10T10:29:05.688718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892835+0000","last_change":"2024-09-10T10:28:59.446889+0000","last_active":"2024-09-10T10:29:05.892835+0000","last_peered":"2024-09-10T10:29:05.892835+0000","last_clean":"2024-09-10T10:29:05.892835+0000","last_became_active":"2024-09-10T10:28:59.446544+0000","last_became_peered":"2024-09-10T10:28:59.446544+0000","last_unstale":"2024-09-10T10:29:05.892835+0000","last_undegraded":"2024-09-10T10:29:05.892835+0000","last_fullsized":"2024-09-10T10:29:05.892835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:32.352466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520235+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:29:05.520235+0000","last_peered":"2024-09-10T10:29:05.520235+0000","last_clean":"2024-09-10T10:29:05.520235+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:29:05.520235+0000","last_undegraded":"2024-09-10T10:29:05.520235+0000","last_fullsized":"2024-09-10T10:29:05.520235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893277+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:29:05.893277+0000","last_peered":"2024-09-10T10:29:05.893277+0000","last_clean":"2024-09-10T10:29:05.893277+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:29:05.893277+0000","last_undegraded":"2024-09-10T10:29:05.893277+0000","last_fullsized":"2024-09-10T10:29:05.893277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831398+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:29:05.831398+0000","last_peered":"2024-09-10T10:29:05.831398+0000","last_clean":"2024-09-10T10:29:05.831398+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:29:05.831398+0000","last_undegraded":"2024-09-10T10:29:05.831398+0000","last_fullsized":"2024-09-10T10:29:05.831398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486160+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:29:05.486160+0000","last_peered":"2024-09-10T10:29:05.486160+0000","last_clean":"2024-09-10T10:29:05.486160+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:29:05.486160+0000","last_undegraded":"2024-09-10T10:29:05.486160+0000","last_fullsized":"2024-09-10T10:29:05.486160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":281,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485314+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:29:05.485314+0000","last_peered":"2024-09-10T10:29:05.485314+0000","last_clean":"2024-09-10T10:29:05.485314+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:29:05.485314+0000","last_undegraded":"2024-09-10T10:29:05.485314+0000","last_fullsized":"2024-09-10T10:29:05.485314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521019+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:29:05.521019+0000","last_peered":"2024-09-10T10:29:05.521019+0000","last_clean":"2024-09-10T10:29:05.521019+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:29:05.521019+0000","last_undegraded":"2024-09-10T10:29:05.521019+0000","last_fullsized":"2024-09-10T10:29:05.521019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482817+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:29:05.482817+0000","last_peered":"2024-09-10T10:29:05.482817+0000","last_clean":"2024-09-10T10:29:05.482817+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:29:05.482817+0000","last_undegraded":"2024-09-10T10:29:05.482817+0000","last_fullsized":"2024-09-10T10:29:05.482817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484865+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:29:05.484865+0000","last_peered":"2024-09-10T10:29:05.484865+0000","last_clean":"2024-09-10T10:29:05.484865+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:29:05.484865+0000","last_undegraded":"2024-09-10T10:29:05.484865+0000","last_fullsized":"2024-09-10T10:29:05.484865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486504+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:29:05.486504+0000","last_peered":"2024-09-10T10:29:05.486504+0000","last_clean":"2024-09-10T10:29:05.486504+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:29:05.486504+0000","last_undegraded":"2024-09-10T10:29:05.486504+0000","last_fullsized":"2024-09-10T10:29:05.486504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688628+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:29:05.688628+0000","last_peered":"2024-09-10T10:29:05.688628+0000","last_clean":"2024-09-10T10:29:05.688628+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:29:05.688628+0000","last_undegraded":"2024-09-10T10:29:05.688628+0000","last_fullsized":"2024-09-10T10:29:05.688628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893128+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:29:05.893128+0000","last_peered":"2024-09-10T10:29:05.893128+0000","last_clean":"2024-09-10T10:29:05.893128+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:29:05.893128+0000","last_undegraded":"2024-09-10T10:29:05.893128+0000","last_fullsized":"2024-09-10T10:29:05.893128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521367+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:29:05.521367+0000","last_peered":"2024-09-10T10:29:05.521367+0000","last_clean":"2024-09-10T10:29:05.521367+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:29:05.521367+0000","last_undegraded":"2024-09-10T10:29:05.521367+0000","last_fullsized":"2024-09-10T10:29:05.521367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483041+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:29:05.483041+0000","last_peered":"2024-09-10T10:29:05.483041+0000","last_clean":"2024-09-10T10:29:05.483041+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:29:05.483041+0000","last_undegraded":"2024-09-10T10:29:05.483041+0000","last_fullsized":"2024-09-10T10:29:05.483041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485530+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:29:05.485530+0000","last_peered":"2024-09-10T10:29:05.485530+0000","last_clean":"2024-09-10T10:29:05.485530+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:29:05.485530+0000","last_undegraded":"2024-09-10T10:29:05.485530+0000","last_fullsized":"2024-09-10T10:29:05.485530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521232+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:29:05.521232+0000","last_peered":"2024-09-10T10:29:05.521232+0000","last_clean":"2024-09-10T10:29:05.521232+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:29:05.521232+0000","last_undegraded":"2024-09-10T10:29:05.521232+0000","last_fullsized":"2024-09-10T10:29:05.521232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956264+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:29:05.956264+0000","last_peered":"2024-09-10T10:29:05.956264+0000","last_clean":"2024-09-10T10:29:05.956264+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:29:05.956264+0000","last_undegraded":"2024-09-10T10:29:05.956264+0000","last_fullsized":"2024-09-10T10:29:05.956264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955704+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:29:05.955704+0000","last_peered":"2024-09-10T10:29:05.955704+0000","last_clean":"2024-09-10T10:29:05.955704+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:29:05.955704+0000","last_undegraded":"2024-09-10T10:29:05.955704+0000","last_fullsized":"2024-09-10T10:29:05.955704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'18","reported_seq":147,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482470+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:29:05.482470+0000","last_peered":"2024-09-10T10:29:05.482470+0000","last_clean":"2024-09-10T10:29:05.482470+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:29:05.482470+0000","last_undegraded":"2024-09-10T10:29:05.482470+0000","last_fullsized":"2024-09-10T10:29:05.482470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893225+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:29:05.893225+0000","last_peered":"2024-09-10T10:29:05.893225+0000","last_clean":"2024-09-10T10:29:05.893225+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:29:05.893225+0000","last_undegraded":"2024-09-10T10:29:05.893225+0000","last_fullsized":"2024-09-10T10:29:05.893225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":160,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:28:59.453146+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:28:59.446453+0000","last_became_peered":"2024-09-10T10:28:59.446453+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:31.347510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1176,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832175+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:29:05.832175+0000","last_peered":"2024-09-10T10:29:05.832175+0000","last_clean":"2024-09-10T10:29:05.832175+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:29:05.832175+0000","last_undegraded":"2024-09-10T10:29:05.832175+0000","last_fullsized":"2024-09-10T10:29:05.832175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520795+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:29:05.520795+0000","last_peered":"2024-09-10T10:29:05.520795+0000","last_clean":"2024-09-10T10:29:05.520795+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:29:05.520795+0000","last_undegraded":"2024-09-10T10:29:05.520795+0000","last_fullsized":"2024-09-10T10:29:05.520795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"60'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486347+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:29:05.486347+0000","last_peered":"2024-09-10T10:29:05.486347+0000","last_clean":"2024-09-10T10:29:05.486347+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:29:05.486347+0000","last_undegraded":"2024-09-10T10:29:05.486347+0000","last_fullsized":"2024-09-10T10:29:05.486347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832226+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:29:05.832226+0000","last_peered":"2024-09-10T10:29:05.832226+0000","last_clean":"2024-09-10T10:29:05.832226+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:29:05.832226+0000","last_undegraded":"2024-09-10T10:29:05.832226+0000","last_fullsized":"2024-09-10T10:29:05.832226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892463+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:29:05.892463+0000","last_peered":"2024-09-10T10:29:05.892463+0000","last_clean":"2024-09-10T10:29:05.892463+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:29:05.892463+0000","last_undegraded":"2024-09-10T10:29:05.892463+0000","last_fullsized":"2024-09-10T10:29:05.892463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893348+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:29:05.893348+0000","last_peered":"2024-09-10T10:29:05.893348+0000","last_clean":"2024-09-10T10:29:05.893348+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:29:05.893348+0000","last_undegraded":"2024-09-10T10:29:05.893348+0000","last_fullsized":"2024-09-10T10:29:05.893348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486563+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:29:05.486563+0000","last_peered":"2024-09-10T10:29:05.486563+0000","last_clean":"2024-09-10T10:29:05.486563+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:29:05.486563+0000","last_undegraded":"2024-09-10T10:29:05.486563+0000","last_fullsized":"2024-09-10T10:29:05.486563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520992+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:29:05.520992+0000","last_peered":"2024-09-10T10:29:05.520992+0000","last_clean":"2024-09-10T10:29:05.520992+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:29:05.520992+0000","last_undegraded":"2024-09-10T10:29:05.520992+0000","last_fullsized":"2024-09-10T10:29:05.520992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"69'27","reported_seq":156,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485568+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:29:05.485568+0000","last_peered":"2024-09-10T10:29:05.485568+0000","last_clean":"2024-09-10T10:29:05.485568+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:29:05.485568+0000","last_undegraded":"2024-09-10T10:29:05.485568+0000","last_fullsized":"2024-09-10T10:29:05.485568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37014,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"60'3","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688715+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:29:05.688715+0000","last_peered":"2024-09-10T10:29:05.688715+0000","last_clean":"2024-09-10T10:29:05.688715+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:29:05.688715+0000","last_undegraded":"2024-09-10T10:29:05.688715+0000","last_fullsized":"2024-09-10T10:29:05.688715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955123+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:29:05.955123+0000","last_peered":"2024-09-10T10:29:05.955123+0000","last_clean":"2024-09-10T10:29:05.955123+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:29:05.955123+0000","last_undegraded":"2024-09-10T10:29:05.955123+0000","last_fullsized":"2024-09-10T10:29:05.955123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'8","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688658+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:29:05.688658+0000","last_peered":"2024-09-10T10:29:05.688658+0000","last_clean":"2024-09-10T10:29:05.688658+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:29:05.688658+0000","last_undegraded":"2024-09-10T10:29:05.688658+0000","last_fullsized":"2024-09-10T10:29:05.688658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955996+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:29:05.955996+0000","last_peered":"2024-09-10T10:29:05.955996+0000","last_clean":"2024-09-10T10:29:05.955996+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:29:05.955996+0000","last_undegraded":"2024-09-10T10:29:05.955996+0000","last_fullsized":"2024-09-10T10:29:05.955996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892560+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:29:05.892560+0000","last_peered":"2024-09-10T10:29:05.892560+0000","last_clean":"2024-09-10T10:29:05.892560+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:29:05.892560+0000","last_undegraded":"2024-09-10T10:29:05.892560+0000","last_fullsized":"2024-09-10T10:29:05.892560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486525+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:29:05.486525+0000","last_peered":"2024-09-10T10:29:05.486525+0000","last_clean":"2024-09-10T10:29:05.486525+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:29:05.486525+0000","last_undegraded":"2024-09-10T10:29:05.486525+0000","last_fullsized":"2024-09-10T10:29:05.486525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831619+0000","last_change":"2024-09-10T10:28:24.248827+0000","last_active":"2024-09-10T10:29:05.831619+0000","last_peered":"2024-09-10T10:29:05.831619+0000","last_clean":"2024-09-10T10:29:05.831619+0000","last_became_active":"2024-09-10T10:28:24.248395+0000","last_became_peered":"2024-09-10T10:28:24.248395+0000","last_unstale":"2024-09-10T10:29:05.831619+0000","last_undegraded":"2024-09-10T10:29:05.831619+0000","last_fullsized":"2024-09-10T10:29:05.831619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:05.306669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":106,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688561+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:29:05.688561+0000","last_peered":"2024-09-10T10:29:05.688561+0000","last_clean":"2024-09-10T10:29:05.688561+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:29:05.688561+0000","last_undegraded":"2024-09-10T10:29:05.688561+0000","last_fullsized":"2024-09-10T10:29:05.688561+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"60'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485617+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:29:05.485617+0000","last_peered":"2024-09-10T10:29:05.485617+0000","last_clean":"2024-09-10T10:29:05.485617+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:29:05.485617+0000","last_undegraded":"2024-09-10T10:29:05.485617+0000","last_fullsized":"2024-09-10T10:29:05.485617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831826+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:29:05.831826+0000","last_peered":"2024-09-10T10:29:05.831826+0000","last_clean":"2024-09-10T10:29:05.831826+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:29:05.831826+0000","last_undegraded":"2024-09-10T10:29:05.831826+0000","last_fullsized":"2024-09-10T10:29:05.831826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893171+0000","last_change":"2024-09-10T10:28:24.376473+0000","last_active":"2024-09-10T10:29:05.893171+0000","last_peered":"2024-09-10T10:29:05.893171+0000","last_clean":"2024-09-10T10:29:05.893171+0000","last_became_active":"2024-09-10T10:28:24.376079+0000","last_became_peered":"2024-09-10T10:28:24.376079+0000","last_unstale":"2024-09-10T10:29:05.893171+0000","last_undegraded":"2024-09-10T10:29:05.893171+0000","last_fullsized":"2024-09-10T10:29:05.893171+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:53.600117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,1],"acting":[0,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688397+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:29:05.688397+0000","last_peered":"2024-09-10T10:29:05.688397+0000","last_clean":"2024-09-10T10:29:05.688397+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:29:05.688397+0000","last_undegraded":"2024-09-10T10:29:05.688397+0000","last_fullsized":"2024-09-10T10:29:05.688397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485970+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:29:05.485970+0000","last_peered":"2024-09-10T10:29:05.485970+0000","last_clean":"2024-09-10T10:29:05.485970+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:29:05.485970+0000","last_undegraded":"2024-09-10T10:29:05.485970+0000","last_fullsized":"2024-09-10T10:29:05.485970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485377+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:29:05.485377+0000","last_peered":"2024-09-10T10:29:05.485377+0000","last_clean":"2024-09-10T10:29:05.485377+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:29:05.485377+0000","last_undegraded":"2024-09-10T10:29:05.485377+0000","last_fullsized":"2024-09-10T10:29:05.485377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831708+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:29:05.831708+0000","last_peered":"2024-09-10T10:29:05.831708+0000","last_clean":"2024-09-10T10:29:05.831708+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:29:05.831708+0000","last_undegraded":"2024-09-10T10:29:05.831708+0000","last_fullsized":"2024-09-10T10:29:05.831708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832298+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:29:05.832298+0000","last_peered":"2024-09-10T10:29:05.832298+0000","last_clean":"2024-09-10T10:29:05.832298+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:29:05.832298+0000","last_undegraded":"2024-09-10T10:29:05.832298+0000","last_fullsized":"2024-09-10T10:29:05.832298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485185+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:29:05.485185+0000","last_peered":"2024-09-10T10:29:05.485185+0000","last_clean":"2024-09-10T10:29:05.485185+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:29:05.485185+0000","last_undegraded":"2024-09-10T10:29:05.485185+0000","last_fullsized":"2024-09-10T10:29:05.485185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893219+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:29:05.893219+0000","last_peered":"2024-09-10T10:29:05.893219+0000","last_clean":"2024-09-10T10:29:05.893219+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:29:05.893219+0000","last_undegraded":"2024-09-10T10:29:05.893219+0000","last_fullsized":"2024-09-10T10:29:05.893219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482644+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:29:05.482644+0000","last_peered":"2024-09-10T10:29:05.482644+0000","last_clean":"2024-09-10T10:29:05.482644+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:29:05.482644+0000","last_undegraded":"2024-09-10T10:29:05.482644+0000","last_fullsized":"2024-09-10T10:29:05.482644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'6","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831961+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:29:05.831961+0000","last_peered":"2024-09-10T10:29:05.831961+0000","last_clean":"2024-09-10T10:29:05.831961+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:29:05.831961+0000","last_undegraded":"2024-09-10T10:29:05.831961+0000","last_fullsized":"2024-09-10T10:29:05.831961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955040+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:29:05.955040+0000","last_peered":"2024-09-10T10:29:05.955040+0000","last_clean":"2024-09-10T10:29:05.955040+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:29:05.955040+0000","last_undegraded":"2024-09-10T10:29:05.955040+0000","last_fullsized":"2024-09-10T10:29:05.955040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832152+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:29:05.832152+0000","last_peered":"2024-09-10T10:29:05.832152+0000","last_clean":"2024-09-10T10:29:05.832152+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:29:05.832152+0000","last_undegraded":"2024-09-10T10:29:05.832152+0000","last_fullsized":"2024-09-10T10:29:05.832152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484806+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:29:05.484806+0000","last_peered":"2024-09-10T10:29:05.484806+0000","last_clean":"2024-09-10T10:29:05.484806+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:29:05.484806+0000","last_undegraded":"2024-09-10T10:29:05.484806+0000","last_fullsized":"2024-09-10T10:29:05.484806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482528+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:29:05.482528+0000","last_peered":"2024-09-10T10:29:05.482528+0000","last_clean":"2024-09-10T10:29:05.482528+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:29:05.482528+0000","last_undegraded":"2024-09-10T10:29:05.482528+0000","last_fullsized":"2024-09-10T10:29:05.482528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486220+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:29:05.486220+0000","last_peered":"2024-09-10T10:29:05.486220+0000","last_clean":"2024-09-10T10:29:05.486220+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:29:05.486220+0000","last_undegraded":"2024-09-10T10:29:05.486220+0000","last_fullsized":"2024-09-10T10:29:05.486220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893191+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:29:05.893191+0000","last_peered":"2024-09-10T10:29:05.893191+0000","last_clean":"2024-09-10T10:29:05.893191+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:29:05.893191+0000","last_undegraded":"2024-09-10T10:29:05.893191+0000","last_fullsized":"2024-09-10T10:29:05.893191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832375+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:29:05.832375+0000","last_peered":"2024-09-10T10:29:05.832375+0000","last_clean":"2024-09-10T10:29:05.832375+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:29:05.832375+0000","last_undegraded":"2024-09-10T10:29:05.832375+0000","last_fullsized":"2024-09-10T10:29:05.832375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688489+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:29:05.688489+0000","last_peered":"2024-09-10T10:29:05.688489+0000","last_clean":"2024-09-10T10:29:05.688489+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:29:05.688489+0000","last_undegraded":"2024-09-10T10:29:05.688489+0000","last_fullsized":"2024-09-10T10:29:05.688489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485637+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:29:05.485637+0000","last_peered":"2024-09-10T10:29:05.485637+0000","last_clean":"2024-09-10T10:29:05.485637+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:29:05.485637+0000","last_undegraded":"2024-09-10T10:29:05.485637+0000","last_fullsized":"2024-09-10T10:29:05.485637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832269+0000","last_change":"2024-09-10T10:28:59.446917+0000","last_active":"2024-09-10T10:29:05.832269+0000","last_peered":"2024-09-10T10:29:05.832269+0000","last_clean":"2024-09-10T10:29:05.832269+0000","last_became_active":"2024-09-10T10:28:59.446546+0000","last_became_peered":"2024-09-10T10:28:59.446546+0000","last_unstale":"2024-09-10T10:29:05.832269+0000","last_undegraded":"2024-09-10T10:29:05.832269+0000","last_fullsized":"2024-09-10T10:29:05.832269+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:30.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485795+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:29:05.485795+0000","last_peered":"2024-09-10T10:29:05.485795+0000","last_clean":"2024-09-10T10:29:05.485795+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:29:05.485795+0000","last_undegraded":"2024-09-10T10:29:05.485795+0000","last_fullsized":"2024-09-10T10:29:05.485795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831886+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:29:05.831886+0000","last_peered":"2024-09-10T10:29:05.831886+0000","last_clean":"2024-09-10T10:29:05.831886+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:29:05.831886+0000","last_undegraded":"2024-09-10T10:29:05.831886+0000","last_fullsized":"2024-09-10T10:29:05.831886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486218+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:29:05.486218+0000","last_peered":"2024-09-10T10:29:05.486218+0000","last_clean":"2024-09-10T10:29:05.486218+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:29:05.486218+0000","last_undegraded":"2024-09-10T10:29:05.486218+0000","last_fullsized":"2024-09-10T10:29:05.486218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892472+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:29:05.892472+0000","last_peered":"2024-09-10T10:29:05.892472+0000","last_clean":"2024-09-10T10:29:05.892472+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:29:05.892472+0000","last_undegraded":"2024-09-10T10:29:05.892472+0000","last_fullsized":"2024-09-10T10:29:05.892472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688898+0000","last_change":"2024-09-10T10:28:23.249392+0000","last_active":"2024-09-10T10:29:05.688898+0000","last_peered":"2024-09-10T10:29:05.688898+0000","last_clean":"2024-09-10T10:29:05.688898+0000","last_became_active":"2024-09-10T10:28:23.249039+0000","last_became_peered":"2024-09-10T10:28:23.249039+0000","last_unstale":"2024-09-10T10:29:05.688898+0000","last_undegraded":"2024-09-10T10:29:05.688898+0000","last_fullsized":"2024-09-10T10:29:05.688898+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:46.656488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485234+0000","last_change":"2024-09-10T10:28:24.244885+0000","last_active":"2024-09-10T10:29:05.485234+0000","last_peered":"2024-09-10T10:29:05.485234+0000","last_clean":"2024-09-10T10:29:05.485234+0000","last_became_active":"2024-09-10T10:28:24.244560+0000","last_became_peered":"2024-09-10T10:28:24.244560+0000","last_unstale":"2024-09-10T10:29:05.485234+0000","last_undegraded":"2024-09-10T10:29:05.485234+0000","last_fullsized":"2024-09-10T10:29:05.485234+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:36.642112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483017+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:29:05.483017+0000","last_peered":"2024-09-10T10:29:05.483017+0000","last_clean":"2024-09-10T10:29:05.483017+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:29:05.483017+0000","last_undegraded":"2024-09-10T10:29:05.483017+0000","last_fullsized":"2024-09-10T10:29:05.483017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486768+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:29:05.486768+0000","last_peered":"2024-09-10T10:29:05.486768+0000","last_clean":"2024-09-10T10:29:05.486768+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:29:05.486768+0000","last_undegraded":"2024-09-10T10:29:05.486768+0000","last_fullsized":"2024-09-10T10:29:05.486768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485724+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:29:05.485724+0000","last_peered":"2024-09-10T10:29:05.485724+0000","last_clean":"2024-09-10T10:29:05.485724+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:29:05.485724+0000","last_undegraded":"2024-09-10T10:29:05.485724+0000","last_fullsized":"2024-09-10T10:29:05.485724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688424+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:29:05.688424+0000","last_peered":"2024-09-10T10:29:05.688424+0000","last_clean":"2024-09-10T10:29:05.688424+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:29:05.688424+0000","last_undegraded":"2024-09-10T10:29:05.688424+0000","last_fullsized":"2024-09-10T10:29:05.688424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"60'1","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892613+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:29:05.892613+0000","last_peered":"2024-09-10T10:29:05.892613+0000","last_clean":"2024-09-10T10:29:05.892613+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:29:05.892613+0000","last_undegraded":"2024-09-10T10:29:05.892613+0000","last_fullsized":"2024-09-10T10:29:05.892613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482510+0000","last_change":"2024-09-10T10:28:24.243479+0000","last_active":"2024-09-10T10:29:05.482510+0000","last_peered":"2024-09-10T10:29:05.482510+0000","last_clean":"2024-09-10T10:29:05.482510+0000","last_became_active":"2024-09-10T10:28:24.243088+0000","last_became_peered":"2024-09-10T10:28:24.243088+0000","last_unstale":"2024-09-10T10:29:05.482510+0000","last_undegraded":"2024-09-10T10:29:05.482510+0000","last_fullsized":"2024-09-10T10:29:05.482510+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:39.465491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":86,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831975+0000","last_change":"2024-09-10T10:27:58.327792+0000","last_active":"2024-09-10T10:29:05.831975+0000","last_peered":"2024-09-10T10:29:05.831975+0000","last_clean":"2024-09-10T10:29:05.831975+0000","last_became_active":"2024-09-10T10:27:58.327502+0000","last_became_peered":"2024-09-10T10:27:58.327502+0000","last_unstale":"2024-09-10T10:29:05.831975+0000","last_undegraded":"2024-09-10T10:29:05.831975+0000","last_fullsized":"2024-09-10T10:29:05.831975+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485410+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:29:05.485410+0000","last_peered":"2024-09-10T10:29:05.485410+0000","last_clean":"2024-09-10T10:29:05.485410+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:29:05.485410+0000","last_undegraded":"2024-09-10T10:29:05.485410+0000","last_fullsized":"2024-09-10T10:29:05.485410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485574+0000","last_change":"2024-09-10T10:28:40.345087+0000","last_active":"2024-09-10T10:29:05.485574+0000","last_peered":"2024-09-10T10:29:05.485574+0000","last_clean":"2024-09-10T10:29:05.485574+0000","last_became_active":"2024-09-10T10:28:40.344766+0000","last_became_peered":"2024-09-10T10:28:40.344766+0000","last_unstale":"2024-09-10T10:29:05.485574+0000","last_undegraded":"2024-09-10T10:29:05.485574+0000","last_fullsized":"2024-09-10T10:29:05.485574+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521372+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:29:05.521372+0000","last_peered":"2024-09-10T10:29:05.521372+0000","last_clean":"2024-09-10T10:29:05.521372+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:29:05.521372+0000","last_undegraded":"2024-09-10T10:29:05.521372+0000","last_fullsized":"2024-09-10T10:29:05.521372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831395+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:29:05.831395+0000","last_peered":"2024-09-10T10:29:05.831395+0000","last_clean":"2024-09-10T10:29:05.831395+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:29:05.831395+0000","last_undegraded":"2024-09-10T10:29:05.831395+0000","last_fullsized":"2024-09-10T10:29:05.831395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520601+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:29:05.520601+0000","last_peered":"2024-09-10T10:29:05.520601+0000","last_clean":"2024-09-10T10:29:05.520601+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:29:05.520601+0000","last_undegraded":"2024-09-10T10:29:05.520601+0000","last_fullsized":"2024-09-10T10:29:05.520601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486096+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:29:05.486096+0000","last_peered":"2024-09-10T10:29:05.486096+0000","last_clean":"2024-09-10T10:29:05.486096+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:29:05.486096+0000","last_undegraded":"2024-09-10T10:29:05.486096+0000","last_fullsized":"2024-09-10T10:29:05.486096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520639+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:29:05.520639+0000","last_peered":"2024-09-10T10:29:05.520639+0000","last_clean":"2024-09-10T10:29:05.520639+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:29:05.520639+0000","last_undegraded":"2024-09-10T10:29:05.520639+0000","last_fullsized":"2024-09-10T10:29:05.520639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486354+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:29:05.486354+0000","last_peered":"2024-09-10T10:29:05.486354+0000","last_clean":"2024-09-10T10:29:05.486354+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:29:05.486354+0000","last_undegraded":"2024-09-10T10:29:05.486354+0000","last_fullsized":"2024-09-10T10:29:05.486354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892681+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:29:05.892681+0000","last_peered":"2024-09-10T10:29:05.892681+0000","last_clean":"2024-09-10T10:29:05.892681+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:29:05.892681+0000","last_undegraded":"2024-09-10T10:29:05.892681+0000","last_fullsized":"2024-09-10T10:29:05.892681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688961+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:29:05.688961+0000","last_peered":"2024-09-10T10:29:05.688961+0000","last_clean":"2024-09-10T10:29:05.688961+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:29:05.688961+0000","last_undegraded":"2024-09-10T10:29:05.688961+0000","last_fullsized":"2024-09-10T10:29:05.688961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521230+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:29:05.521230+0000","last_peered":"2024-09-10T10:29:05.521230+0000","last_clean":"2024-09-10T10:29:05.521230+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:29:05.521230+0000","last_undegraded":"2024-09-10T10:29:05.521230+0000","last_fullsized":"2024-09-10T10:29:05.521230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44104,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":247,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":88208,"data_compressed":3324,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1019904,"data_stored":1074371,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":974848,"data_stored":519451,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":966656,"data_stored":485579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":482977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":520153,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":954368,"data_stored":483521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":483179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1011712,"data_stored":1072889,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":38506,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":37804,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:29:10.739 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:29:10.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.872+0000 7f3c81b26640 1 Processor -- start 2024-09-10T10:29:10.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.872+0000 7f3c81b26640 1 -- start start 2024-09-10T10:29:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.873+0000 7f3c81b26640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3c7c152850 0x7f3c7c158670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.873+0000 7f3c81b26640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3c7c158bb0 0x7f3c7c0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.873+0000 7f3c81b26640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c7c072f50 con 0x7f3c7c0c6bb0 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.873+0000 7f3c81b26640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f3c7c073210 con 0x7f3c7c158bb0 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.873+0000 7f3c81b26640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3c7c15cb50 con 0x7f3c7c152850 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.873+0000 7f3c7affd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3c7c158bb0 0x7f3c7c0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.873+0000 7f3c7bfff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f3c7c0c6bb0 0x7f3c7c0c86f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41564/0 (socket says 172.21.15.120:41564) 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.873+0000 7f3c7affd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3c7c158bb0 0x7f3c7c0c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42474/0 (socket says 172.21.15.120:42474) 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.873+0000 7f3c7bfff640 1 -- 172.21.15.120:0/1728725683 learned_addr learned my addr 172.21.15.120:0/1728725683 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.873+0000 7f3c7b7fe640 1 --2- 172.21.15.120:0/1728725683 >> v2:172.21.15.134:3300/0 conn(0x7f3c7c152850 0x7f3c7c158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.874+0000 7f3c7affd640 1 -- 172.21.15.120:0/1728725683 >> v2:172.21.15.134:3300/0 conn(0x7f3c7c152850 msgr2=0x7f3c7c158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.874+0000 7f3c7affd640 1 --2- 172.21.15.120:0/1728725683 >> v2:172.21.15.134:3300/0 conn(0x7f3c7c152850 0x7f3c7c158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.874+0000 7f3c7affd640 1 -- 172.21.15.120:0/1728725683 >> v1:172.21.15.120:6789/0 conn(0x7f3c7c0c6bb0 legacy=0x7f3c7c0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.874+0000 7f3c7a7fc640 1 -- 172.21.15.120:0/1728725683 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487400401 0 0) 0x7f3c7c072f50 con 0x7f3c7c0c6bb0 2024-09-10T10:29:10.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.874+0000 7f3c7affd640 1 -- 172.21.15.120:0/1728725683 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c15bd90 con 0x7f3c7c158bb0 2024-09-10T10:29:10.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.874+0000 7f3c7affd640 1 --2- 172.21.15.120:0/1728725683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 0x7f3c7c0c58c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c073680 tx=0x7f3c7002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9cef0d51b36142 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:10.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.874+0000 7f3c7a7fc640 1 -- 172.21.15.120:0/1728725683 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70037070 con 0x7f3c7c158bb0 2024-09-10T10:29:10.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.874+0000 7f3c7a7fc640 1 -- 172.21.15.120:0/1728725683 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c70032440 con 0x7f3c7c158bb0 2024-09-10T10:29:10.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.874+0000 7f3c7a7fc640 1 -- 172.21.15.120:0/1728725683 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70032810 con 0x7f3c7c158bb0 2024-09-10T10:29:10.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.875+0000 7f3c81b26640 1 -- 172.21.15.120:0/1728725683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 msgr2=0x7f3c7c0c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.875+0000 7f3c81b26640 1 --2- 172.21.15.120:0/1728725683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 0x7f3c7c0c58c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c073680 tx=0x7f3c7002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.875+0000 7f3c81b26640 1 -- 172.21.15.120:0/1728725683 shutdown_connections 2024-09-10T10:29:10.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.875+0000 7f3c81b26640 1 --2- 172.21.15.120:0/1728725683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 0x7f3c7c0c58c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.875+0000 7f3c81b26640 1 --2- 172.21.15.120:0/1728725683 >> v2:172.21.15.134:3300/0 conn(0x7f3c7c152850 0x7f3c7c158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.875+0000 7f3c81b26640 1 -- 172.21.15.120:0/1728725683 >> 172.21.15.120:0/1728725683 conn(0x7f3c7c06f3c0 msgr2=0x7f3c7c06f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:10.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.875+0000 7f3c81b26640 1 -- 172.21.15.120:0/1728725683 shutdown_connections 2024-09-10T10:29:10.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.875+0000 7f3c81b26640 1 -- 172.21.15.120:0/1728725683 wait complete. 2024-09-10T10:29:10.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.876+0000 7f3c81b26640 1 Processor -- start 2024-09-10T10:29:10.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.876+0000 7f3c81b26640 1 -- start start 2024-09-10T10:29:10.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c81b26640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c0c6bb0 0x7f3c7c1eed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c81b26640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c7c152850 0x7f3c7c1f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c81b26640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 0x7f3c7c0ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c81b26640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c15c630 con 0x7f3c7c158bb0 2024-09-10T10:29:10.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c81b26640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c15c350 con 0x7f3c7c0c6bb0 2024-09-10T10:29:10.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c81b26640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7c070200 con 0x7f3c7c152850 2024-09-10T10:29:10.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 0x7f3c7c0ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 0x7f3c7c0ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42484/0 (socket says 172.21.15.120:42484) 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c7bfff640 1 -- 172.21.15.120:0/2960059967 learned_addr learned my addr 172.21.15.120:0/2960059967 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c7b7fe640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c0c6bb0 0x7f3c7c1eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c7affd640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c7c152850 0x7f3c7c1f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c7bfff640 1 -- 172.21.15.120:0/2960059967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c7c152850 msgr2=0x7f3c7c1f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c7bfff640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c7c152850 0x7f3c7c1f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c7bfff640 1 -- 172.21.15.120:0/2960059967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c0c6bb0 msgr2=0x7f3c7c1eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c7bfff640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c0c6bb0 0x7f3c7c1eed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.877+0000 7f3c7bfff640 1 -- 172.21.15.120:0/2960059967 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c14cfa0 con 0x7f3c7c158bb0 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.878+0000 7f3c7bfff640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 0x7f3c7c0ca850 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c450 tx=0x7f3c7400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.878+0000 7f3c63fff640 1 -- 172.21.15.120:0/2960059967 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74018070 con 0x7f3c7c158bb0 2024-09-10T10:29:10.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.878+0000 7f3c63fff640 1 -- 172.21.15.120:0/2960059967 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c74007500 con 0x7f3c7c158bb0 2024-09-10T10:29:10.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.878+0000 7f3c81b26640 1 -- 172.21.15.120:0/2960059967 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7c14d280 con 0x7f3c7c158bb0 2024-09-10T10:29:10.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.878+0000 7f3c63fff640 1 -- 172.21.15.120:0/2960059967 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74004a20 con 0x7f3c7c158bb0 2024-09-10T10:29:10.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.878+0000 7f3c81b26640 1 -- 172.21.15.120:0/2960059967 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7c1ee500 con 0x7f3c7c158bb0 2024-09-10T10:29:10.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.879+0000 7f3c81b26640 1 -- 172.21.15.120:0/2960059967 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c7c0718c0 con 0x7f3c7c158bb0 2024-09-10T10:29:10.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.879+0000 7f3c63fff640 1 -- 172.21.15.120:0/2960059967 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c7401b070 con 0x7f3c7c158bb0 2024-09-10T10:29:10.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.879+0000 7f3c63fff640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3c6806be30 0x7f3c6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:10.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.880+0000 7f3c63fff640 1 -- 172.21.15.120:0/2960059967 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f3c7408a640 con 0x7f3c7c158bb0 2024-09-10T10:29:10.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.880+0000 7f3c7b7fe640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3c6806be30 0x7f3c6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:10.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.880+0000 7f3c7b7fe640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3c6806be30 0x7f3c6806e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c003b80 tx=0x7f3c6c0098a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:10.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:10.884+0000 7f3c63fff640 1 -- 172.21.15.120:0/2960059967 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3c74058410 con 0x7f3c7c158bb0 2024-09-10T10:29:11.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.049+0000 7f3c81b26640 1 -- 172.21.15.120:0/2960059967 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c7c0c8f10 con 0x7f3c6806be30 2024-09-10T10:29:11.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.056+0000 7f3c63fff640 1 -- 172.21.15.120:0/2960059967 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332899 (secure 0 0 0) 0x7f3c7c0c8f10 con 0x7f3c6806be30 2024-09-10T10:29:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:11.063 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:29:11.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 -- 172.21.15.120:0/2960059967 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3c6806be30 msgr2=0x7f3c6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3c6806be30 0x7f3c6806e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c003b80 tx=0x7f3c6c0098a0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 -- 172.21.15.120:0/2960059967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 msgr2=0x7f3c7c0ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 0x7f3c7c0ca850 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c450 tx=0x7f3c7400c910 comp rx=0 tx=0).stop 2024-09-10T10:29:11.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 -- 172.21.15.120:0/2960059967 shutdown_connections 2024-09-10T10:29:11.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f3c6806be30 0x7f3c6806e2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c7c158bb0 0x7f3c7c0ca850 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c7c152850 0x7f3c7c1f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 --2- 172.21.15.120:0/2960059967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c0c6bb0 0x7f3c7c1eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 -- 172.21.15.120:0/2960059967 >> 172.21.15.120:0/2960059967 conn(0x7f3c7c06f3c0 msgr2=0x7f3c7c0c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:11.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 -- 172.21.15.120:0/2960059967 shutdown_connections 2024-09-10T10:29:11.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.060+0000 7f3c81b26640 1 -- 172.21.15.120:0/2960059967 wait complete. 2024-09-10T10:29:11.072 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-10T10:29:10.120811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634472,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":119,"num_read_kb":103,"num_write":408,"num_write_kb":2539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":228028,"kb_used_data":7616,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754746692,"statfs":{"total":773094113280,"available":772860612608,"internally_reserved":0,"allocated":7798784,"data_stored":5122120,"data_compressed":68176,"data_compressed_allocated":98304,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.656364"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893394+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:29:05.893394+0000","last_peered":"2024-09-10T10:29:05.893394+0000","last_clean":"2024-09-10T10:29:05.893394+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:29:05.893394+0000","last_undegraded":"2024-09-10T10:29:05.893394+0000","last_fullsized":"2024-09-10T10:29:05.893394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831510+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:29:05.831510+0000","last_peered":"2024-09-10T10:29:05.831510+0000","last_clean":"2024-09-10T10:29:05.831510+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:29:05.831510+0000","last_undegraded":"2024-09-10T10:29:05.831510+0000","last_fullsized":"2024-09-10T10:29:05.831510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831742+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:29:05.831742+0000","last_peered":"2024-09-10T10:29:05.831742+0000","last_clean":"2024-09-10T10:29:05.831742+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:29:05.831742+0000","last_undegraded":"2024-09-10T10:29:05.831742+0000","last_fullsized":"2024-09-10T10:29:05.831742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832208+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:29:05.832208+0000","last_peered":"2024-09-10T10:29:05.832208+0000","last_clean":"2024-09-10T10:29:05.832208+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:29:05.832208+0000","last_undegraded":"2024-09-10T10:29:05.832208+0000","last_fullsized":"2024-09-10T10:29:05.832208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892600+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:29:05.892600+0000","last_peered":"2024-09-10T10:29:05.892600+0000","last_clean":"2024-09-10T10:29:05.892600+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:29:05.892600+0000","last_undegraded":"2024-09-10T10:29:05.892600+0000","last_fullsized":"2024-09-10T10:29:05.892600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955501+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:29:05.955501+0000","last_peered":"2024-09-10T10:29:05.955501+0000","last_clean":"2024-09-10T10:29:05.955501+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:29:05.955501+0000","last_undegraded":"2024-09-10T10:29:05.955501+0000","last_fullsized":"2024-09-10T10:29:05.955501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"60'13","reported_seq":158,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521178+0000","last_change":"2024-09-10T10:27:51.060242+0000","last_active":"2024-09-10T10:29:05.521178+0000","last_peered":"2024-09-10T10:29:05.521178+0000","last_clean":"2024-09-10T10:29:05.521178+0000","last_became_active":"2024-09-10T10:27:51.049148+0000","last_became_peered":"2024-09-10T10:27:51.049148+0000","last_unstale":"2024-09-10T10:29:05.521178+0000","last_undegraded":"2024-09-10T10:29:05.521178+0000","last_fullsized":"2024-09-10T10:29:05.521178+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.604144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486386+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:29:05.486386+0000","last_peered":"2024-09-10T10:29:05.486386+0000","last_clean":"2024-09-10T10:29:05.486386+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:29:05.486386+0000","last_undegraded":"2024-09-10T10:29:05.486386+0000","last_fullsized":"2024-09-10T10:29:05.486386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955770+0000","last_change":"2024-09-10T10:28:24.244697+0000","last_active":"2024-09-10T10:29:05.955770+0000","last_peered":"2024-09-10T10:29:05.955770+0000","last_clean":"2024-09-10T10:29:05.955770+0000","last_became_active":"2024-09-10T10:28:24.244262+0000","last_became_peered":"2024-09-10T10:28:24.244262+0000","last_unstale":"2024-09-10T10:29:05.955770+0000","last_undegraded":"2024-09-10T10:29:05.955770+0000","last_fullsized":"2024-09-10T10:29:05.955770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:50.683186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6,2],"acting":[4,0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520262+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:29:05.520262+0000","last_peered":"2024-09-10T10:29:05.520262+0000","last_clean":"2024-09-10T10:29:05.520262+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:29:05.520262+0000","last_undegraded":"2024-09-10T10:29:05.520262+0000","last_fullsized":"2024-09-10T10:29:05.520262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":110,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482709+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:29:05.482709+0000","last_peered":"2024-09-10T10:29:05.482709+0000","last_clean":"2024-09-10T10:29:05.482709+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:29:05.482709+0000","last_undegraded":"2024-09-10T10:29:05.482709+0000","last_fullsized":"2024-09-10T10:29:05.482709+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955722+0000","last_change":"2024-09-10T10:28:59.443607+0000","last_active":"2024-09-10T10:29:05.955722+0000","last_peered":"2024-09-10T10:29:05.955722+0000","last_clean":"2024-09-10T10:29:05.955722+0000","last_became_active":"2024-09-10T10:28:59.443172+0000","last_became_peered":"2024-09-10T10:28:59.443172+0000","last_unstale":"2024-09-10T10:29:05.955722+0000","last_undegraded":"2024-09-10T10:29:05.955722+0000","last_fullsized":"2024-09-10T10:29:05.955722+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:36.379163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485088+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:29:05.485088+0000","last_peered":"2024-09-10T10:29:05.485088+0000","last_clean":"2024-09-10T10:29:05.485088+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:29:05.485088+0000","last_undegraded":"2024-09-10T10:29:05.485088+0000","last_fullsized":"2024-09-10T10:29:05.485088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"60'5","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520220+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:29:05.520220+0000","last_peered":"2024-09-10T10:29:05.520220+0000","last_clean":"2024-09-10T10:29:05.520220+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:29:05.520220+0000","last_undegraded":"2024-09-10T10:29:05.520220+0000","last_fullsized":"2024-09-10T10:29:05.520220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956119+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:29:05.956119+0000","last_peered":"2024-09-10T10:29:05.956119+0000","last_clean":"2024-09-10T10:29:05.956119+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:29:05.956119+0000","last_undegraded":"2024-09-10T10:29:05.956119+0000","last_fullsized":"2024-09-10T10:29:05.956119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831889+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:29:05.831889+0000","last_peered":"2024-09-10T10:29:05.831889+0000","last_clean":"2024-09-10T10:29:05.831889+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:29:05.831889+0000","last_undegraded":"2024-09-10T10:29:05.831889+0000","last_fullsized":"2024-09-10T10:29:05.831889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688805+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:29:05.688805+0000","last_peered":"2024-09-10T10:29:05.688805+0000","last_clean":"2024-09-10T10:29:05.688805+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:29:05.688805+0000","last_undegraded":"2024-09-10T10:29:05.688805+0000","last_fullsized":"2024-09-10T10:29:05.688805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955843+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:29:05.955843+0000","last_peered":"2024-09-10T10:29:05.955843+0000","last_clean":"2024-09-10T10:29:05.955843+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:29:05.955843+0000","last_undegraded":"2024-09-10T10:29:05.955843+0000","last_fullsized":"2024-09-10T10:29:05.955843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831383+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:29:05.831383+0000","last_peered":"2024-09-10T10:29:05.831383+0000","last_clean":"2024-09-10T10:29:05.831383+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:29:05.831383+0000","last_undegraded":"2024-09-10T10:29:05.831383+0000","last_fullsized":"2024-09-10T10:29:05.831383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688223+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:29:05.688223+0000","last_peered":"2024-09-10T10:29:05.688223+0000","last_clean":"2024-09-10T10:29:05.688223+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:29:05.688223+0000","last_undegraded":"2024-09-10T10:29:05.688223+0000","last_fullsized":"2024-09-10T10:29:05.688223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688656+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:29:05.688656+0000","last_peered":"2024-09-10T10:29:05.688656+0000","last_clean":"2024-09-10T10:29:05.688656+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:29:05.688656+0000","last_undegraded":"2024-09-10T10:29:05.688656+0000","last_fullsized":"2024-09-10T10:29:05.688656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955038+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:29:05.955038+0000","last_peered":"2024-09-10T10:29:05.955038+0000","last_clean":"2024-09-10T10:29:05.955038+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:29:05.955038+0000","last_undegraded":"2024-09-10T10:29:05.955038+0000","last_fullsized":"2024-09-10T10:29:05.955038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520435+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:29:05.520435+0000","last_peered":"2024-09-10T10:29:05.520435+0000","last_clean":"2024-09-10T10:29:05.520435+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:29:05.520435+0000","last_undegraded":"2024-09-10T10:29:05.520435+0000","last_fullsized":"2024-09-10T10:29:05.520435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956141+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:29:05.956141+0000","last_peered":"2024-09-10T10:29:05.956141+0000","last_clean":"2024-09-10T10:29:05.956141+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:29:05.956141+0000","last_undegraded":"2024-09-10T10:29:05.956141+0000","last_fullsized":"2024-09-10T10:29:05.956141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892494+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:29:05.892494+0000","last_peered":"2024-09-10T10:29:05.892494+0000","last_clean":"2024-09-10T10:29:05.892494+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:29:05.892494+0000","last_undegraded":"2024-09-10T10:29:05.892494+0000","last_fullsized":"2024-09-10T10:29:05.892494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'8","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485743+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:29:05.485743+0000","last_peered":"2024-09-10T10:29:05.485743+0000","last_clean":"2024-09-10T10:29:05.485743+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:29:05.485743+0000","last_undegraded":"2024-09-10T10:29:05.485743+0000","last_fullsized":"2024-09-10T10:29:05.485743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955720+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:29:05.955720+0000","last_peered":"2024-09-10T10:29:05.955720+0000","last_clean":"2024-09-10T10:29:05.955720+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:29:05.955720+0000","last_undegraded":"2024-09-10T10:29:05.955720+0000","last_fullsized":"2024-09-10T10:29:05.955720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485344+0000","last_change":"2024-09-10T10:28:24.374205+0000","last_active":"2024-09-10T10:29:05.485344+0000","last_peered":"2024-09-10T10:29:05.485344+0000","last_clean":"2024-09-10T10:29:05.485344+0000","last_became_active":"2024-09-10T10:28:24.373921+0000","last_became_peered":"2024-09-10T10:28:24.373921+0000","last_unstale":"2024-09-10T10:29:05.485344+0000","last_undegraded":"2024-09-10T10:29:05.485344+0000","last_fullsized":"2024-09-10T10:29:05.485344+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:36.923293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"64'15","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688969+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:29:05.688969+0000","last_peered":"2024-09-10T10:29:05.688969+0000","last_clean":"2024-09-10T10:29:05.688969+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:29:05.688969+0000","last_undegraded":"2024-09-10T10:29:05.688969+0000","last_fullsized":"2024-09-10T10:29:05.688969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831546+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:29:05.831546+0000","last_peered":"2024-09-10T10:29:05.831546+0000","last_clean":"2024-09-10T10:29:05.831546+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:29:05.831546+0000","last_undegraded":"2024-09-10T10:29:05.831546+0000","last_fullsized":"2024-09-10T10:29:05.831546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688907+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:29:05.688907+0000","last_peered":"2024-09-10T10:29:05.688907+0000","last_clean":"2024-09-10T10:29:05.688907+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:29:05.688907+0000","last_undegraded":"2024-09-10T10:29:05.688907+0000","last_fullsized":"2024-09-10T10:29:05.688907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521226+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:29:05.521226+0000","last_peered":"2024-09-10T10:29:05.521226+0000","last_clean":"2024-09-10T10:29:05.521226+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:29:05.521226+0000","last_undegraded":"2024-09-10T10:29:05.521226+0000","last_fullsized":"2024-09-10T10:29:05.521226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893084+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:29:05.893084+0000","last_peered":"2024-09-10T10:29:05.893084+0000","last_clean":"2024-09-10T10:29:05.893084+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:29:05.893084+0000","last_undegraded":"2024-09-10T10:29:05.893084+0000","last_fullsized":"2024-09-10T10:29:05.893084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486566+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:29:05.486566+0000","last_peered":"2024-09-10T10:29:05.486566+0000","last_clean":"2024-09-10T10:29:05.486566+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:29:05.486566+0000","last_undegraded":"2024-09-10T10:29:05.486566+0000","last_fullsized":"2024-09-10T10:29:05.486566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688951+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:29:05.688951+0000","last_peered":"2024-09-10T10:29:05.688951+0000","last_clean":"2024-09-10T10:29:05.688951+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.688951+0000","last_undegraded":"2024-09-10T10:29:05.688951+0000","last_fullsized":"2024-09-10T10:29:05.688951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688451+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:29:05.688451+0000","last_peered":"2024-09-10T10:29:05.688451+0000","last_clean":"2024-09-10T10:29:05.688451+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:29:05.688451+0000","last_undegraded":"2024-09-10T10:29:05.688451+0000","last_fullsized":"2024-09-10T10:29:05.688451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956082+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:29:05.956082+0000","last_peered":"2024-09-10T10:29:05.956082+0000","last_clean":"2024-09-10T10:29:05.956082+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:29:05.956082+0000","last_undegraded":"2024-09-10T10:29:05.956082+0000","last_fullsized":"2024-09-10T10:29:05.956082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486359+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:29:05.486359+0000","last_peered":"2024-09-10T10:29:05.486359+0000","last_clean":"2024-09-10T10:29:05.486359+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:29:05.486359+0000","last_undegraded":"2024-09-10T10:29:05.486359+0000","last_fullsized":"2024-09-10T10:29:05.486359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486083+0000","last_change":"2024-09-10T10:28:24.244988+0000","last_active":"2024-09-10T10:29:05.486083+0000","last_peered":"2024-09-10T10:29:05.486083+0000","last_clean":"2024-09-10T10:29:05.486083+0000","last_became_active":"2024-09-10T10:28:24.244556+0000","last_became_peered":"2024-09-10T10:28:24.244556+0000","last_unstale":"2024-09-10T10:29:05.486083+0000","last_undegraded":"2024-09-10T10:29:05.486083+0000","last_fullsized":"2024-09-10T10:29:05.486083+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:08.127492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688685+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:29:05.688685+0000","last_peered":"2024-09-10T10:29:05.688685+0000","last_clean":"2024-09-10T10:29:05.688685+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:29:05.688685+0000","last_undegraded":"2024-09-10T10:29:05.688685+0000","last_fullsized":"2024-09-10T10:29:05.688685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'9","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482578+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:29:05.482578+0000","last_peered":"2024-09-10T10:29:05.482578+0000","last_clean":"2024-09-10T10:29:05.482578+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.482578+0000","last_undegraded":"2024-09-10T10:29:05.482578+0000","last_fullsized":"2024-09-10T10:29:05.482578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688718+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:29:05.688718+0000","last_peered":"2024-09-10T10:29:05.688718+0000","last_clean":"2024-09-10T10:29:05.688718+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:29:05.688718+0000","last_undegraded":"2024-09-10T10:29:05.688718+0000","last_fullsized":"2024-09-10T10:29:05.688718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892835+0000","last_change":"2024-09-10T10:28:59.446889+0000","last_active":"2024-09-10T10:29:05.892835+0000","last_peered":"2024-09-10T10:29:05.892835+0000","last_clean":"2024-09-10T10:29:05.892835+0000","last_became_active":"2024-09-10T10:28:59.446544+0000","last_became_peered":"2024-09-10T10:28:59.446544+0000","last_unstale":"2024-09-10T10:29:05.892835+0000","last_undegraded":"2024-09-10T10:29:05.892835+0000","last_fullsized":"2024-09-10T10:29:05.892835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:32.352466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520235+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:29:05.520235+0000","last_peered":"2024-09-10T10:29:05.520235+0000","last_clean":"2024-09-10T10:29:05.520235+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:29:05.520235+0000","last_undegraded":"2024-09-10T10:29:05.520235+0000","last_fullsized":"2024-09-10T10:29:05.520235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893277+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:29:05.893277+0000","last_peered":"2024-09-10T10:29:05.893277+0000","last_clean":"2024-09-10T10:29:05.893277+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:29:05.893277+0000","last_undegraded":"2024-09-10T10:29:05.893277+0000","last_fullsized":"2024-09-10T10:29:05.893277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831398+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:29:05.831398+0000","last_peered":"2024-09-10T10:29:05.831398+0000","last_clean":"2024-09-10T10:29:05.831398+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:29:05.831398+0000","last_undegraded":"2024-09-10T10:29:05.831398+0000","last_fullsized":"2024-09-10T10:29:05.831398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486160+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:29:05.486160+0000","last_peered":"2024-09-10T10:29:05.486160+0000","last_clean":"2024-09-10T10:29:05.486160+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:29:05.486160+0000","last_undegraded":"2024-09-10T10:29:05.486160+0000","last_fullsized":"2024-09-10T10:29:05.486160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":281,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485314+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:29:05.485314+0000","last_peered":"2024-09-10T10:29:05.485314+0000","last_clean":"2024-09-10T10:29:05.485314+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:29:05.485314+0000","last_undegraded":"2024-09-10T10:29:05.485314+0000","last_fullsized":"2024-09-10T10:29:05.485314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521019+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:29:05.521019+0000","last_peered":"2024-09-10T10:29:05.521019+0000","last_clean":"2024-09-10T10:29:05.521019+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:29:05.521019+0000","last_undegraded":"2024-09-10T10:29:05.521019+0000","last_fullsized":"2024-09-10T10:29:05.521019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482817+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:29:05.482817+0000","last_peered":"2024-09-10T10:29:05.482817+0000","last_clean":"2024-09-10T10:29:05.482817+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:29:05.482817+0000","last_undegraded":"2024-09-10T10:29:05.482817+0000","last_fullsized":"2024-09-10T10:29:05.482817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484865+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:29:05.484865+0000","last_peered":"2024-09-10T10:29:05.484865+0000","last_clean":"2024-09-10T10:29:05.484865+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:29:05.484865+0000","last_undegraded":"2024-09-10T10:29:05.484865+0000","last_fullsized":"2024-09-10T10:29:05.484865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486504+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:29:05.486504+0000","last_peered":"2024-09-10T10:29:05.486504+0000","last_clean":"2024-09-10T10:29:05.486504+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:29:05.486504+0000","last_undegraded":"2024-09-10T10:29:05.486504+0000","last_fullsized":"2024-09-10T10:29:05.486504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688628+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:29:05.688628+0000","last_peered":"2024-09-10T10:29:05.688628+0000","last_clean":"2024-09-10T10:29:05.688628+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:29:05.688628+0000","last_undegraded":"2024-09-10T10:29:05.688628+0000","last_fullsized":"2024-09-10T10:29:05.688628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893128+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:29:05.893128+0000","last_peered":"2024-09-10T10:29:05.893128+0000","last_clean":"2024-09-10T10:29:05.893128+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:29:05.893128+0000","last_undegraded":"2024-09-10T10:29:05.893128+0000","last_fullsized":"2024-09-10T10:29:05.893128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521367+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:29:05.521367+0000","last_peered":"2024-09-10T10:29:05.521367+0000","last_clean":"2024-09-10T10:29:05.521367+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:29:05.521367+0000","last_undegraded":"2024-09-10T10:29:05.521367+0000","last_fullsized":"2024-09-10T10:29:05.521367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483041+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:29:05.483041+0000","last_peered":"2024-09-10T10:29:05.483041+0000","last_clean":"2024-09-10T10:29:05.483041+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:29:05.483041+0000","last_undegraded":"2024-09-10T10:29:05.483041+0000","last_fullsized":"2024-09-10T10:29:05.483041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485530+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:29:05.485530+0000","last_peered":"2024-09-10T10:29:05.485530+0000","last_clean":"2024-09-10T10:29:05.485530+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:29:05.485530+0000","last_undegraded":"2024-09-10T10:29:05.485530+0000","last_fullsized":"2024-09-10T10:29:05.485530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521232+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:29:05.521232+0000","last_peered":"2024-09-10T10:29:05.521232+0000","last_clean":"2024-09-10T10:29:05.521232+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:29:05.521232+0000","last_undegraded":"2024-09-10T10:29:05.521232+0000","last_fullsized":"2024-09-10T10:29:05.521232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956264+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:29:05.956264+0000","last_peered":"2024-09-10T10:29:05.956264+0000","last_clean":"2024-09-10T10:29:05.956264+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:29:05.956264+0000","last_undegraded":"2024-09-10T10:29:05.956264+0000","last_fullsized":"2024-09-10T10:29:05.956264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955704+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:29:05.955704+0000","last_peered":"2024-09-10T10:29:05.955704+0000","last_clean":"2024-09-10T10:29:05.955704+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:29:05.955704+0000","last_undegraded":"2024-09-10T10:29:05.955704+0000","last_fullsized":"2024-09-10T10:29:05.955704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'18","reported_seq":147,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482470+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:29:05.482470+0000","last_peered":"2024-09-10T10:29:05.482470+0000","last_clean":"2024-09-10T10:29:05.482470+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:29:05.482470+0000","last_undegraded":"2024-09-10T10:29:05.482470+0000","last_fullsized":"2024-09-10T10:29:05.482470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893225+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:29:05.893225+0000","last_peered":"2024-09-10T10:29:05.893225+0000","last_clean":"2024-09-10T10:29:05.893225+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:29:05.893225+0000","last_undegraded":"2024-09-10T10:29:05.893225+0000","last_fullsized":"2024-09-10T10:29:05.893225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":160,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:28:59.453146+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:28:59.446453+0000","last_became_peered":"2024-09-10T10:28:59.446453+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:31.347510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1176,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832175+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:29:05.832175+0000","last_peered":"2024-09-10T10:29:05.832175+0000","last_clean":"2024-09-10T10:29:05.832175+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:29:05.832175+0000","last_undegraded":"2024-09-10T10:29:05.832175+0000","last_fullsized":"2024-09-10T10:29:05.832175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520795+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:29:05.520795+0000","last_peered":"2024-09-10T10:29:05.520795+0000","last_clean":"2024-09-10T10:29:05.520795+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:29:05.520795+0000","last_undegraded":"2024-09-10T10:29:05.520795+0000","last_fullsized":"2024-09-10T10:29:05.520795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"60'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486347+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:29:05.486347+0000","last_peered":"2024-09-10T10:29:05.486347+0000","last_clean":"2024-09-10T10:29:05.486347+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:29:05.486347+0000","last_undegraded":"2024-09-10T10:29:05.486347+0000","last_fullsized":"2024-09-10T10:29:05.486347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832226+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:29:05.832226+0000","last_peered":"2024-09-10T10:29:05.832226+0000","last_clean":"2024-09-10T10:29:05.832226+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:29:05.832226+0000","last_undegraded":"2024-09-10T10:29:05.832226+0000","last_fullsized":"2024-09-10T10:29:05.832226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892463+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:29:05.892463+0000","last_peered":"2024-09-10T10:29:05.892463+0000","last_clean":"2024-09-10T10:29:05.892463+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:29:05.892463+0000","last_undegraded":"2024-09-10T10:29:05.892463+0000","last_fullsized":"2024-09-10T10:29:05.892463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893348+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:29:05.893348+0000","last_peered":"2024-09-10T10:29:05.893348+0000","last_clean":"2024-09-10T10:29:05.893348+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:29:05.893348+0000","last_undegraded":"2024-09-10T10:29:05.893348+0000","last_fullsized":"2024-09-10T10:29:05.893348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486563+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:29:05.486563+0000","last_peered":"2024-09-10T10:29:05.486563+0000","last_clean":"2024-09-10T10:29:05.486563+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:29:05.486563+0000","last_undegraded":"2024-09-10T10:29:05.486563+0000","last_fullsized":"2024-09-10T10:29:05.486563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520992+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:29:05.520992+0000","last_peered":"2024-09-10T10:29:05.520992+0000","last_clean":"2024-09-10T10:29:05.520992+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:29:05.520992+0000","last_undegraded":"2024-09-10T10:29:05.520992+0000","last_fullsized":"2024-09-10T10:29:05.520992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"69'27","reported_seq":156,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485568+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:29:05.485568+0000","last_peered":"2024-09-10T10:29:05.485568+0000","last_clean":"2024-09-10T10:29:05.485568+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:29:05.485568+0000","last_undegraded":"2024-09-10T10:29:05.485568+0000","last_fullsized":"2024-09-10T10:29:05.485568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37014,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"60'3","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688715+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:29:05.688715+0000","last_peered":"2024-09-10T10:29:05.688715+0000","last_clean":"2024-09-10T10:29:05.688715+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:29:05.688715+0000","last_undegraded":"2024-09-10T10:29:05.688715+0000","last_fullsized":"2024-09-10T10:29:05.688715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955123+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:29:05.955123+0000","last_peered":"2024-09-10T10:29:05.955123+0000","last_clean":"2024-09-10T10:29:05.955123+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:29:05.955123+0000","last_undegraded":"2024-09-10T10:29:05.955123+0000","last_fullsized":"2024-09-10T10:29:05.955123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'8","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688658+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:29:05.688658+0000","last_peered":"2024-09-10T10:29:05.688658+0000","last_clean":"2024-09-10T10:29:05.688658+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:29:05.688658+0000","last_undegraded":"2024-09-10T10:29:05.688658+0000","last_fullsized":"2024-09-10T10:29:05.688658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955996+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:29:05.955996+0000","last_peered":"2024-09-10T10:29:05.955996+0000","last_clean":"2024-09-10T10:29:05.955996+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:29:05.955996+0000","last_undegraded":"2024-09-10T10:29:05.955996+0000","last_fullsized":"2024-09-10T10:29:05.955996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892560+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:29:05.892560+0000","last_peered":"2024-09-10T10:29:05.892560+0000","last_clean":"2024-09-10T10:29:05.892560+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:29:05.892560+0000","last_undegraded":"2024-09-10T10:29:05.892560+0000","last_fullsized":"2024-09-10T10:29:05.892560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486525+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:29:05.486525+0000","last_peered":"2024-09-10T10:29:05.486525+0000","last_clean":"2024-09-10T10:29:05.486525+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:29:05.486525+0000","last_undegraded":"2024-09-10T10:29:05.486525+0000","last_fullsized":"2024-09-10T10:29:05.486525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831619+0000","last_change":"2024-09-10T10:28:24.248827+0000","last_active":"2024-09-10T10:29:05.831619+0000","last_peered":"2024-09-10T10:29:05.831619+0000","last_clean":"2024-09-10T10:29:05.831619+0000","last_became_active":"2024-09-10T10:28:24.248395+0000","last_became_peered":"2024-09-10T10:28:24.248395+0000","last_unstale":"2024-09-10T10:29:05.831619+0000","last_undegraded":"2024-09-10T10:29:05.831619+0000","last_fullsized":"2024-09-10T10:29:05.831619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:05.306669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":106,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688561+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:29:05.688561+0000","last_peered":"2024-09-10T10:29:05.688561+0000","last_clean":"2024-09-10T10:29:05.688561+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:29:05.688561+0000","last_undegraded":"2024-09-10T10:29:05.688561+0000","last_fullsized":"2024-09-10T10:29:05.688561+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"60'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485617+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:29:05.485617+0000","last_peered":"2024-09-10T10:29:05.485617+0000","last_clean":"2024-09-10T10:29:05.485617+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:29:05.485617+0000","last_undegraded":"2024-09-10T10:29:05.485617+0000","last_fullsized":"2024-09-10T10:29:05.485617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831826+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:29:05.831826+0000","last_peered":"2024-09-10T10:29:05.831826+0000","last_clean":"2024-09-10T10:29:05.831826+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:29:05.831826+0000","last_undegraded":"2024-09-10T10:29:05.831826+0000","last_fullsized":"2024-09-10T10:29:05.831826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893171+0000","last_change":"2024-09-10T10:28:24.376473+0000","last_active":"2024-09-10T10:29:05.893171+0000","last_peered":"2024-09-10T10:29:05.893171+0000","last_clean":"2024-09-10T10:29:05.893171+0000","last_became_active":"2024-09-10T10:28:24.376079+0000","last_became_peered":"2024-09-10T10:28:24.376079+0000","last_unstale":"2024-09-10T10:29:05.893171+0000","last_undegraded":"2024-09-10T10:29:05.893171+0000","last_fullsized":"2024-09-10T10:29:05.893171+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:53.600117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,1],"acting":[0,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688397+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:29:05.688397+0000","last_peered":"2024-09-10T10:29:05.688397+0000","last_clean":"2024-09-10T10:29:05.688397+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:29:05.688397+0000","last_undegraded":"2024-09-10T10:29:05.688397+0000","last_fullsized":"2024-09-10T10:29:05.688397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485970+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:29:05.485970+0000","last_peered":"2024-09-10T10:29:05.485970+0000","last_clean":"2024-09-10T10:29:05.485970+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:29:05.485970+0000","last_undegraded":"2024-09-10T10:29:05.485970+0000","last_fullsized":"2024-09-10T10:29:05.485970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485377+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:29:05.485377+0000","last_peered":"2024-09-10T10:29:05.485377+0000","last_clean":"2024-09-10T10:29:05.485377+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:29:05.485377+0000","last_undegraded":"2024-09-10T10:29:05.485377+0000","last_fullsized":"2024-09-10T10:29:05.485377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831708+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:29:05.831708+0000","last_peered":"2024-09-10T10:29:05.831708+0000","last_clean":"2024-09-10T10:29:05.831708+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:29:05.831708+0000","last_undegraded":"2024-09-10T10:29:05.831708+0000","last_fullsized":"2024-09-10T10:29:05.831708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832298+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:29:05.832298+0000","last_peered":"2024-09-10T10:29:05.832298+0000","last_clean":"2024-09-10T10:29:05.832298+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:29:05.832298+0000","last_undegraded":"2024-09-10T10:29:05.832298+0000","last_fullsized":"2024-09-10T10:29:05.832298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485185+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:29:05.485185+0000","last_peered":"2024-09-10T10:29:05.485185+0000","last_clean":"2024-09-10T10:29:05.485185+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:29:05.485185+0000","last_undegraded":"2024-09-10T10:29:05.485185+0000","last_fullsized":"2024-09-10T10:29:05.485185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893219+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:29:05.893219+0000","last_peered":"2024-09-10T10:29:05.893219+0000","last_clean":"2024-09-10T10:29:05.893219+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:29:05.893219+0000","last_undegraded":"2024-09-10T10:29:05.893219+0000","last_fullsized":"2024-09-10T10:29:05.893219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482644+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:29:05.482644+0000","last_peered":"2024-09-10T10:29:05.482644+0000","last_clean":"2024-09-10T10:29:05.482644+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:29:05.482644+0000","last_undegraded":"2024-09-10T10:29:05.482644+0000","last_fullsized":"2024-09-10T10:29:05.482644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'6","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831961+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:29:05.831961+0000","last_peered":"2024-09-10T10:29:05.831961+0000","last_clean":"2024-09-10T10:29:05.831961+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:29:05.831961+0000","last_undegraded":"2024-09-10T10:29:05.831961+0000","last_fullsized":"2024-09-10T10:29:05.831961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955040+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:29:05.955040+0000","last_peered":"2024-09-10T10:29:05.955040+0000","last_clean":"2024-09-10T10:29:05.955040+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:29:05.955040+0000","last_undegraded":"2024-09-10T10:29:05.955040+0000","last_fullsized":"2024-09-10T10:29:05.955040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832152+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:29:05.832152+0000","last_peered":"2024-09-10T10:29:05.832152+0000","last_clean":"2024-09-10T10:29:05.832152+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:29:05.832152+0000","last_undegraded":"2024-09-10T10:29:05.832152+0000","last_fullsized":"2024-09-10T10:29:05.832152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484806+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:29:05.484806+0000","last_peered":"2024-09-10T10:29:05.484806+0000","last_clean":"2024-09-10T10:29:05.484806+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:29:05.484806+0000","last_undegraded":"2024-09-10T10:29:05.484806+0000","last_fullsized":"2024-09-10T10:29:05.484806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482528+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:29:05.482528+0000","last_peered":"2024-09-10T10:29:05.482528+0000","last_clean":"2024-09-10T10:29:05.482528+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:29:05.482528+0000","last_undegraded":"2024-09-10T10:29:05.482528+0000","last_fullsized":"2024-09-10T10:29:05.482528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486220+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:29:05.486220+0000","last_peered":"2024-09-10T10:29:05.486220+0000","last_clean":"2024-09-10T10:29:05.486220+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:29:05.486220+0000","last_undegraded":"2024-09-10T10:29:05.486220+0000","last_fullsized":"2024-09-10T10:29:05.486220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893191+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:29:05.893191+0000","last_peered":"2024-09-10T10:29:05.893191+0000","last_clean":"2024-09-10T10:29:05.893191+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:29:05.893191+0000","last_undegraded":"2024-09-10T10:29:05.893191+0000","last_fullsized":"2024-09-10T10:29:05.893191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832375+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:29:05.832375+0000","last_peered":"2024-09-10T10:29:05.832375+0000","last_clean":"2024-09-10T10:29:05.832375+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:29:05.832375+0000","last_undegraded":"2024-09-10T10:29:05.832375+0000","last_fullsized":"2024-09-10T10:29:05.832375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688489+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:29:05.688489+0000","last_peered":"2024-09-10T10:29:05.688489+0000","last_clean":"2024-09-10T10:29:05.688489+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:29:05.688489+0000","last_undegraded":"2024-09-10T10:29:05.688489+0000","last_fullsized":"2024-09-10T10:29:05.688489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485637+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:29:05.485637+0000","last_peered":"2024-09-10T10:29:05.485637+0000","last_clean":"2024-09-10T10:29:05.485637+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:29:05.485637+0000","last_undegraded":"2024-09-10T10:29:05.485637+0000","last_fullsized":"2024-09-10T10:29:05.485637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832269+0000","last_change":"2024-09-10T10:28:59.446917+0000","last_active":"2024-09-10T10:29:05.832269+0000","last_peered":"2024-09-10T10:29:05.832269+0000","last_clean":"2024-09-10T10:29:05.832269+0000","last_became_active":"2024-09-10T10:28:59.446546+0000","last_became_peered":"2024-09-10T10:28:59.446546+0000","last_unstale":"2024-09-10T10:29:05.832269+0000","last_undegraded":"2024-09-10T10:29:05.832269+0000","last_fullsized":"2024-09-10T10:29:05.832269+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:30.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485795+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:29:05.485795+0000","last_peered":"2024-09-10T10:29:05.485795+0000","last_clean":"2024-09-10T10:29:05.485795+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:29:05.485795+0000","last_undegraded":"2024-09-10T10:29:05.485795+0000","last_fullsized":"2024-09-10T10:29:05.485795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831886+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:29:05.831886+0000","last_peered":"2024-09-10T10:29:05.831886+0000","last_clean":"2024-09-10T10:29:05.831886+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:29:05.831886+0000","last_undegraded":"2024-09-10T10:29:05.831886+0000","last_fullsized":"2024-09-10T10:29:05.831886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486218+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:29:05.486218+0000","last_peered":"2024-09-10T10:29:05.486218+0000","last_clean":"2024-09-10T10:29:05.486218+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:29:05.486218+0000","last_undegraded":"2024-09-10T10:29:05.486218+0000","last_fullsized":"2024-09-10T10:29:05.486218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892472+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:29:05.892472+0000","last_peered":"2024-09-10T10:29:05.892472+0000","last_clean":"2024-09-10T10:29:05.892472+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:29:05.892472+0000","last_undegraded":"2024-09-10T10:29:05.892472+0000","last_fullsized":"2024-09-10T10:29:05.892472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688898+0000","last_change":"2024-09-10T10:28:23.249392+0000","last_active":"2024-09-10T10:29:05.688898+0000","last_peered":"2024-09-10T10:29:05.688898+0000","last_clean":"2024-09-10T10:29:05.688898+0000","last_became_active":"2024-09-10T10:28:23.249039+0000","last_became_peered":"2024-09-10T10:28:23.249039+0000","last_unstale":"2024-09-10T10:29:05.688898+0000","last_undegraded":"2024-09-10T10:29:05.688898+0000","last_fullsized":"2024-09-10T10:29:05.688898+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:46.656488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485234+0000","last_change":"2024-09-10T10:28:24.244885+0000","last_active":"2024-09-10T10:29:05.485234+0000","last_peered":"2024-09-10T10:29:05.485234+0000","last_clean":"2024-09-10T10:29:05.485234+0000","last_became_active":"2024-09-10T10:28:24.244560+0000","last_became_peered":"2024-09-10T10:28:24.244560+0000","last_unstale":"2024-09-10T10:29:05.485234+0000","last_undegraded":"2024-09-10T10:29:05.485234+0000","last_fullsized":"2024-09-10T10:29:05.485234+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:36.642112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483017+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:29:05.483017+0000","last_peered":"2024-09-10T10:29:05.483017+0000","last_clean":"2024-09-10T10:29:05.483017+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:29:05.483017+0000","last_undegraded":"2024-09-10T10:29:05.483017+0000","last_fullsized":"2024-09-10T10:29:05.483017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486768+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:29:05.486768+0000","last_peered":"2024-09-10T10:29:05.486768+0000","last_clean":"2024-09-10T10:29:05.486768+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:29:05.486768+0000","last_undegraded":"2024-09-10T10:29:05.486768+0000","last_fullsized":"2024-09-10T10:29:05.486768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485724+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:29:05.485724+0000","last_peered":"2024-09-10T10:29:05.485724+0000","last_clean":"2024-09-10T10:29:05.485724+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:29:05.485724+0000","last_undegraded":"2024-09-10T10:29:05.485724+0000","last_fullsized":"2024-09-10T10:29:05.485724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688424+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:29:05.688424+0000","last_peered":"2024-09-10T10:29:05.688424+0000","last_clean":"2024-09-10T10:29:05.688424+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:29:05.688424+0000","last_undegraded":"2024-09-10T10:29:05.688424+0000","last_fullsized":"2024-09-10T10:29:05.688424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"60'1","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892613+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:29:05.892613+0000","last_peered":"2024-09-10T10:29:05.892613+0000","last_clean":"2024-09-10T10:29:05.892613+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:29:05.892613+0000","last_undegraded":"2024-09-10T10:29:05.892613+0000","last_fullsized":"2024-09-10T10:29:05.892613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482510+0000","last_change":"2024-09-10T10:28:24.243479+0000","last_active":"2024-09-10T10:29:05.482510+0000","last_peered":"2024-09-10T10:29:05.482510+0000","last_clean":"2024-09-10T10:29:05.482510+0000","last_became_active":"2024-09-10T10:28:24.243088+0000","last_became_peered":"2024-09-10T10:28:24.243088+0000","last_unstale":"2024-09-10T10:29:05.482510+0000","last_undegraded":"2024-09-10T10:29:05.482510+0000","last_fullsized":"2024-09-10T10:29:05.482510+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:39.465491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":86,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831975+0000","last_change":"2024-09-10T10:27:58.327792+0000","last_active":"2024-09-10T10:29:05.831975+0000","last_peered":"2024-09-10T10:29:05.831975+0000","last_clean":"2024-09-10T10:29:05.831975+0000","last_became_active":"2024-09-10T10:27:58.327502+0000","last_became_peered":"2024-09-10T10:27:58.327502+0000","last_unstale":"2024-09-10T10:29:05.831975+0000","last_undegraded":"2024-09-10T10:29:05.831975+0000","last_fullsized":"2024-09-10T10:29:05.831975+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485410+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:29:05.485410+0000","last_peered":"2024-09-10T10:29:05.485410+0000","last_clean":"2024-09-10T10:29:05.485410+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:29:05.485410+0000","last_undegraded":"2024-09-10T10:29:05.485410+0000","last_fullsized":"2024-09-10T10:29:05.485410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485574+0000","last_change":"2024-09-10T10:28:40.345087+0000","last_active":"2024-09-10T10:29:05.485574+0000","last_peered":"2024-09-10T10:29:05.485574+0000","last_clean":"2024-09-10T10:29:05.485574+0000","last_became_active":"2024-09-10T10:28:40.344766+0000","last_became_peered":"2024-09-10T10:28:40.344766+0000","last_unstale":"2024-09-10T10:29:05.485574+0000","last_undegraded":"2024-09-10T10:29:05.485574+0000","last_fullsized":"2024-09-10T10:29:05.485574+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521372+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:29:05.521372+0000","last_peered":"2024-09-10T10:29:05.521372+0000","last_clean":"2024-09-10T10:29:05.521372+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:29:05.521372+0000","last_undegraded":"2024-09-10T10:29:05.521372+0000","last_fullsized":"2024-09-10T10:29:05.521372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831395+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:29:05.831395+0000","last_peered":"2024-09-10T10:29:05.831395+0000","last_clean":"2024-09-10T10:29:05.831395+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:29:05.831395+0000","last_undegraded":"2024-09-10T10:29:05.831395+0000","last_fullsized":"2024-09-10T10:29:05.831395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520601+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:29:05.520601+0000","last_peered":"2024-09-10T10:29:05.520601+0000","last_clean":"2024-09-10T10:29:05.520601+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:29:05.520601+0000","last_undegraded":"2024-09-10T10:29:05.520601+0000","last_fullsized":"2024-09-10T10:29:05.520601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486096+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:29:05.486096+0000","last_peered":"2024-09-10T10:29:05.486096+0000","last_clean":"2024-09-10T10:29:05.486096+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:29:05.486096+0000","last_undegraded":"2024-09-10T10:29:05.486096+0000","last_fullsized":"2024-09-10T10:29:05.486096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520639+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:29:05.520639+0000","last_peered":"2024-09-10T10:29:05.520639+0000","last_clean":"2024-09-10T10:29:05.520639+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:29:05.520639+0000","last_undegraded":"2024-09-10T10:29:05.520639+0000","last_fullsized":"2024-09-10T10:29:05.520639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486354+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:29:05.486354+0000","last_peered":"2024-09-10T10:29:05.486354+0000","last_clean":"2024-09-10T10:29:05.486354+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:29:05.486354+0000","last_undegraded":"2024-09-10T10:29:05.486354+0000","last_fullsized":"2024-09-10T10:29:05.486354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892681+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:29:05.892681+0000","last_peered":"2024-09-10T10:29:05.892681+0000","last_clean":"2024-09-10T10:29:05.892681+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:29:05.892681+0000","last_undegraded":"2024-09-10T10:29:05.892681+0000","last_fullsized":"2024-09-10T10:29:05.892681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688961+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:29:05.688961+0000","last_peered":"2024-09-10T10:29:05.688961+0000","last_clean":"2024-09-10T10:29:05.688961+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:29:05.688961+0000","last_undegraded":"2024-09-10T10:29:05.688961+0000","last_fullsized":"2024-09-10T10:29:05.688961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521230+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:29:05.521230+0000","last_peered":"2024-09-10T10:29:05.521230+0000","last_clean":"2024-09-10T10:29:05.521230+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:29:05.521230+0000","last_undegraded":"2024-09-10T10:29:05.521230+0000","last_fullsized":"2024-09-10T10:29:05.521230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44104,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":247,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":88208,"data_compressed":3324,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1019904,"data_stored":1074371,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":974848,"data_stored":519451,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":966656,"data_stored":485579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":482977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":520153,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":954368,"data_stored":483521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":483179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1011712,"data_stored":1072889,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":38506,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":37804,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:29:11.120 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:29:11.121 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:29:11.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.219+0000 7f53895e8640 1 Processor -- start 2024-09-10T10:29:11.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.219+0000 7f53895e8640 1 -- start start 2024-09-10T10:29:11.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f53895e8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f53840cc0c0 0x7f538415aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f53895e8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53840cb2f0 0x7f53840cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f53895e8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f538415b1d0 con 0x7f538415e9d0 2024-09-10T10:29:11.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f53895e8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f53840cbd00 con 0x7f53840cc0c0 2024-09-10T10:29:11.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f53895e8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f538415ba70 con 0x7f53840cb2f0 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f5382ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f53840cc0c0 0x7f538415aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f5382ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f53840cc0c0 0x7f538415aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42488/0 (socket says 172.21.15.120:42488) 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f5382ffd640 1 -- 172.21.15.120:0/3954238044 learned_addr learned my addr 172.21.15.120:0/3954238044 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f537bfff640 1 --2- 172.21.15.120:0/3954238044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53840cb2f0 0x7f53840cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f5382ffd640 1 -- 172.21.15.120:0/3954238044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53840cb2f0 msgr2=0x7f53840cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f5382ffd640 1 --2- 172.21.15.120:0/3954238044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53840cb2f0 0x7f53840cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f5382ffd640 1 -- 172.21.15.120:0/3954238044 >> v1:172.21.15.134:6789/0 conn(0x7f538415e9d0 legacy=0x7f53840cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f5382ffd640 1 -- 172.21.15.120:0/3954238044 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538415b4b0 con 0x7f53840cc0c0 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.220+0000 7f537bfff640 1 --2- 172.21.15.120:0/3954238044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53840cb2f0 0x7f53840cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.221+0000 7f5382ffd640 1 --2- 172.21.15.120:0/3954238044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cc0c0 0x7f538415aab0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5374009870 tx=0x7f537402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f20686c74d01338 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.221+0000 7f53827fc640 1 -- 172.21.15.120:0/3954238044 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374037070 con 0x7f53840cc0c0 2024-09-10T10:29:11.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.221+0000 7f53827fc640 1 -- 172.21.15.120:0/3954238044 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5374032440 con 0x7f53840cc0c0 2024-09-10T10:29:11.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.221+0000 7f53827fc640 1 -- 172.21.15.120:0/3954238044 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374032810 con 0x7f53840cc0c0 2024-09-10T10:29:11.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.222+0000 7f53895e8640 1 -- 172.21.15.120:0/3954238044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cc0c0 msgr2=0x7f538415aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.222+0000 7f53895e8640 1 --2- 172.21.15.120:0/3954238044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cc0c0 0x7f538415aab0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5374009870 tx=0x7f537402a210 comp rx=0 tx=0).stop 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.222+0000 7f53895e8640 1 -- 172.21.15.120:0/3954238044 shutdown_connections 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.222+0000 7f53895e8640 1 --2- 172.21.15.120:0/3954238044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53840cb2f0 0x7f53840cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.222+0000 7f53895e8640 1 --2- 172.21.15.120:0/3954238044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cc0c0 0x7f538415aab0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.222+0000 7f53895e8640 1 -- 172.21.15.120:0/3954238044 >> 172.21.15.120:0/3954238044 conn(0x7f538406d560 msgr2=0x7f538406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.222+0000 7f53895e8640 1 -- 172.21.15.120:0/3954238044 shutdown_connections 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.222+0000 7f53895e8640 1 -- 172.21.15.120:0/3954238044 wait complete. 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.223+0000 7f53895e8640 1 Processor -- start 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.223+0000 7f53895e8640 1 -- start start 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.223+0000 7f53895e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cb2f0 0x7f5384150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.223+0000 7f53895e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53840cc0c0 0x7f5384149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.223+0000 7f53895e8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f538415e9d0 0x7f53841f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.223+0000 7f53895e8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f538415e1d0 con 0x7f53840cb2f0 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.223+0000 7f53895e8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f538415e540 con 0x7f53840cc0c0 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.223+0000 7f53895e8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53840c5560 con 0x7f538415e9d0 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f5382ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cb2f0 0x7f5384150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f5382ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cb2f0 0x7f5384150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42496/0 (socket says 172.21.15.120:42496) 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f5382ffd640 1 -- 172.21.15.120:0/488843307 learned_addr learned my addr 172.21.15.120:0/488843307 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f537bfff640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53840cc0c0 0x7f5384149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f53837fe640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f538415e9d0 0x7f53841f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f5382ffd640 1 -- 172.21.15.120:0/488843307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f538415e9d0 msgr2=0x7f53841f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f5382ffd640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f538415e9d0 0x7f53841f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f5382ffd640 1 -- 172.21.15.120:0/488843307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53840cc0c0 msgr2=0x7f5384149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f5382ffd640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53840cc0c0 0x7f5384149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f5382ffd640 1 -- 172.21.15.120:0/488843307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5384159fd0 con 0x7f53840cb2f0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f5382ffd640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cb2f0 0x7f5384150530 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5374009870 tx=0x7f53740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f537b7fe640 1 -- 172.21.15.120:0/488843307 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374037040 con 0x7f53840cb2f0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f537b7fe640 1 -- 172.21.15.120:0/488843307 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5374042070 con 0x7f53840cb2f0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f53895e8640 1 -- 172.21.15.120:0/488843307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5384151050 con 0x7f53840cb2f0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f537b7fe640 1 -- 172.21.15.120:0/488843307 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374030bb0 con 0x7f53840cb2f0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.224+0000 7f53895e8640 1 -- 172.21.15.120:0/488843307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538415c0d0 con 0x7f53840cb2f0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.225+0000 7f53895e8640 1 -- 172.21.15.120:0/488843307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5384151750 con 0x7f53840cb2f0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.226+0000 7f537b7fe640 1 -- 172.21.15.120:0/488843307 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5374047460 con 0x7f53840cb2f0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.226+0000 7f537b7fe640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f536406be60 0x7f536406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.226+0000 7f537b7fe640 1 -- 172.21.15.120:0/488843307 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f53740abe30 con 0x7f53840cb2f0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.226+0000 7f537bfff640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f536406be60 0x7f536406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.226+0000 7f537bfff640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f536406be60 0x7f536406e320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f53840cac70 tx=0x7f536c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:11.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.230+0000 7f537b7fe640 1 -- 172.21.15.120:0/488843307 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5374079c90 con 0x7f53840cb2f0 2024-09-10T10:29:11.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.391+0000 7f53895e8640 1 -- 172.21.15.120:0/488843307 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53840cbea0 con 0x7f536406be60 2024-09-10T10:29:11.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.399+0000 7f537b7fe640 1 -- 172.21.15.120:0/488843307 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332899 (secure 0 0 0) 0x7f53840cbea0 con 0x7f536406be60 2024-09-10T10:29:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:11.405 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:29:11.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 -- 172.21.15.120:0/488843307 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f536406be60 msgr2=0x7f536406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f536406be60 0x7f536406e320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f53840cac70 tx=0x7f536c008040 comp rx=0 tx=0).stop 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 -- 172.21.15.120:0/488843307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cb2f0 msgr2=0x7f5384150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cb2f0 0x7f5384150530 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5374009870 tx=0x7f53740046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 -- 172.21.15.120:0/488843307 shutdown_connections 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f536406be60 0x7f536406e320 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f538415e9d0 0x7f53841f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53840cc0c0 0x7f5384149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 --2- 172.21.15.120:0/488843307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53840cb2f0 0x7f5384150530 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 -- 172.21.15.120:0/488843307 >> 172.21.15.120:0/488843307 conn(0x7f538406d560 msgr2=0x7f5384152010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 -- 172.21.15.120:0/488843307 shutdown_connections 2024-09-10T10:29:11.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.402+0000 7f53895e8640 1 -- 172.21.15.120:0/488843307 wait complete. 2024-09-10T10:29:11.414 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-10T10:29:10.120811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634472,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":119,"num_read_kb":103,"num_write":408,"num_write_kb":2539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":228028,"kb_used_data":7616,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754746692,"statfs":{"total":773094113280,"available":772860612608,"internally_reserved":0,"allocated":7798784,"data_stored":5122120,"data_compressed":68176,"data_compressed_allocated":98304,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.656364"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893394+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:29:05.893394+0000","last_peered":"2024-09-10T10:29:05.893394+0000","last_clean":"2024-09-10T10:29:05.893394+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:29:05.893394+0000","last_undegraded":"2024-09-10T10:29:05.893394+0000","last_fullsized":"2024-09-10T10:29:05.893394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:48.798549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831510+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:29:05.831510+0000","last_peered":"2024-09-10T10:29:05.831510+0000","last_clean":"2024-09-10T10:29:05.831510+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:29:05.831510+0000","last_undegraded":"2024-09-10T10:29:05.831510+0000","last_fullsized":"2024-09-10T10:29:05.831510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831742+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:29:05.831742+0000","last_peered":"2024-09-10T10:29:05.831742+0000","last_clean":"2024-09-10T10:29:05.831742+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:29:05.831742+0000","last_undegraded":"2024-09-10T10:29:05.831742+0000","last_fullsized":"2024-09-10T10:29:05.831742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832208+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:29:05.832208+0000","last_peered":"2024-09-10T10:29:05.832208+0000","last_clean":"2024-09-10T10:29:05.832208+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:29:05.832208+0000","last_undegraded":"2024-09-10T10:29:05.832208+0000","last_fullsized":"2024-09-10T10:29:05.832208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892600+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:29:05.892600+0000","last_peered":"2024-09-10T10:29:05.892600+0000","last_clean":"2024-09-10T10:29:05.892600+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:29:05.892600+0000","last_undegraded":"2024-09-10T10:29:05.892600+0000","last_fullsized":"2024-09-10T10:29:05.892600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955501+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:29:05.955501+0000","last_peered":"2024-09-10T10:29:05.955501+0000","last_clean":"2024-09-10T10:29:05.955501+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:29:05.955501+0000","last_undegraded":"2024-09-10T10:29:05.955501+0000","last_fullsized":"2024-09-10T10:29:05.955501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"60'13","reported_seq":158,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521178+0000","last_change":"2024-09-10T10:27:51.060242+0000","last_active":"2024-09-10T10:29:05.521178+0000","last_peered":"2024-09-10T10:29:05.521178+0000","last_clean":"2024-09-10T10:29:05.521178+0000","last_became_active":"2024-09-10T10:27:51.049148+0000","last_became_peered":"2024-09-10T10:27:51.049148+0000","last_unstale":"2024-09-10T10:29:05.521178+0000","last_undegraded":"2024-09-10T10:29:05.521178+0000","last_fullsized":"2024-09-10T10:29:05.521178+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.604144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486386+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:29:05.486386+0000","last_peered":"2024-09-10T10:29:05.486386+0000","last_clean":"2024-09-10T10:29:05.486386+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:29:05.486386+0000","last_undegraded":"2024-09-10T10:29:05.486386+0000","last_fullsized":"2024-09-10T10:29:05.486386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955770+0000","last_change":"2024-09-10T10:28:24.244697+0000","last_active":"2024-09-10T10:29:05.955770+0000","last_peered":"2024-09-10T10:29:05.955770+0000","last_clean":"2024-09-10T10:29:05.955770+0000","last_became_active":"2024-09-10T10:28:24.244262+0000","last_became_peered":"2024-09-10T10:28:24.244262+0000","last_unstale":"2024-09-10T10:29:05.955770+0000","last_undegraded":"2024-09-10T10:29:05.955770+0000","last_fullsized":"2024-09-10T10:29:05.955770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:50.683186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6,2],"acting":[4,0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520262+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:29:05.520262+0000","last_peered":"2024-09-10T10:29:05.520262+0000","last_clean":"2024-09-10T10:29:05.520262+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:29:05.520262+0000","last_undegraded":"2024-09-10T10:29:05.520262+0000","last_fullsized":"2024-09-10T10:29:05.520262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":110,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482709+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:29:05.482709+0000","last_peered":"2024-09-10T10:29:05.482709+0000","last_clean":"2024-09-10T10:29:05.482709+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:29:05.482709+0000","last_undegraded":"2024-09-10T10:29:05.482709+0000","last_fullsized":"2024-09-10T10:29:05.482709+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955722+0000","last_change":"2024-09-10T10:28:59.443607+0000","last_active":"2024-09-10T10:29:05.955722+0000","last_peered":"2024-09-10T10:29:05.955722+0000","last_clean":"2024-09-10T10:29:05.955722+0000","last_became_active":"2024-09-10T10:28:59.443172+0000","last_became_peered":"2024-09-10T10:28:59.443172+0000","last_unstale":"2024-09-10T10:29:05.955722+0000","last_undegraded":"2024-09-10T10:29:05.955722+0000","last_fullsized":"2024-09-10T10:29:05.955722+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:36.379163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485088+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:29:05.485088+0000","last_peered":"2024-09-10T10:29:05.485088+0000","last_clean":"2024-09-10T10:29:05.485088+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:29:05.485088+0000","last_undegraded":"2024-09-10T10:29:05.485088+0000","last_fullsized":"2024-09-10T10:29:05.485088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"60'5","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520220+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:29:05.520220+0000","last_peered":"2024-09-10T10:29:05.520220+0000","last_clean":"2024-09-10T10:29:05.520220+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:29:05.520220+0000","last_undegraded":"2024-09-10T10:29:05.520220+0000","last_fullsized":"2024-09-10T10:29:05.520220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956119+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:29:05.956119+0000","last_peered":"2024-09-10T10:29:05.956119+0000","last_clean":"2024-09-10T10:29:05.956119+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:29:05.956119+0000","last_undegraded":"2024-09-10T10:29:05.956119+0000","last_fullsized":"2024-09-10T10:29:05.956119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831889+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:29:05.831889+0000","last_peered":"2024-09-10T10:29:05.831889+0000","last_clean":"2024-09-10T10:29:05.831889+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:29:05.831889+0000","last_undegraded":"2024-09-10T10:29:05.831889+0000","last_fullsized":"2024-09-10T10:29:05.831889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688805+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:29:05.688805+0000","last_peered":"2024-09-10T10:29:05.688805+0000","last_clean":"2024-09-10T10:29:05.688805+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:29:05.688805+0000","last_undegraded":"2024-09-10T10:29:05.688805+0000","last_fullsized":"2024-09-10T10:29:05.688805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955843+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:29:05.955843+0000","last_peered":"2024-09-10T10:29:05.955843+0000","last_clean":"2024-09-10T10:29:05.955843+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:29:05.955843+0000","last_undegraded":"2024-09-10T10:29:05.955843+0000","last_fullsized":"2024-09-10T10:29:05.955843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831383+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:29:05.831383+0000","last_peered":"2024-09-10T10:29:05.831383+0000","last_clean":"2024-09-10T10:29:05.831383+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:29:05.831383+0000","last_undegraded":"2024-09-10T10:29:05.831383+0000","last_fullsized":"2024-09-10T10:29:05.831383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688223+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:29:05.688223+0000","last_peered":"2024-09-10T10:29:05.688223+0000","last_clean":"2024-09-10T10:29:05.688223+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:29:05.688223+0000","last_undegraded":"2024-09-10T10:29:05.688223+0000","last_fullsized":"2024-09-10T10:29:05.688223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688656+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:29:05.688656+0000","last_peered":"2024-09-10T10:29:05.688656+0000","last_clean":"2024-09-10T10:29:05.688656+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:29:05.688656+0000","last_undegraded":"2024-09-10T10:29:05.688656+0000","last_fullsized":"2024-09-10T10:29:05.688656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955038+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:29:05.955038+0000","last_peered":"2024-09-10T10:29:05.955038+0000","last_clean":"2024-09-10T10:29:05.955038+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:29:05.955038+0000","last_undegraded":"2024-09-10T10:29:05.955038+0000","last_fullsized":"2024-09-10T10:29:05.955038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520435+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:29:05.520435+0000","last_peered":"2024-09-10T10:29:05.520435+0000","last_clean":"2024-09-10T10:29:05.520435+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:29:05.520435+0000","last_undegraded":"2024-09-10T10:29:05.520435+0000","last_fullsized":"2024-09-10T10:29:05.520435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956141+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:29:05.956141+0000","last_peered":"2024-09-10T10:29:05.956141+0000","last_clean":"2024-09-10T10:29:05.956141+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:29:05.956141+0000","last_undegraded":"2024-09-10T10:29:05.956141+0000","last_fullsized":"2024-09-10T10:29:05.956141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892494+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:29:05.892494+0000","last_peered":"2024-09-10T10:29:05.892494+0000","last_clean":"2024-09-10T10:29:05.892494+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:29:05.892494+0000","last_undegraded":"2024-09-10T10:29:05.892494+0000","last_fullsized":"2024-09-10T10:29:05.892494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'8","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485743+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:29:05.485743+0000","last_peered":"2024-09-10T10:29:05.485743+0000","last_clean":"2024-09-10T10:29:05.485743+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:29:05.485743+0000","last_undegraded":"2024-09-10T10:29:05.485743+0000","last_fullsized":"2024-09-10T10:29:05.485743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955720+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:29:05.955720+0000","last_peered":"2024-09-10T10:29:05.955720+0000","last_clean":"2024-09-10T10:29:05.955720+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:29:05.955720+0000","last_undegraded":"2024-09-10T10:29:05.955720+0000","last_fullsized":"2024-09-10T10:29:05.955720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485344+0000","last_change":"2024-09-10T10:28:24.374205+0000","last_active":"2024-09-10T10:29:05.485344+0000","last_peered":"2024-09-10T10:29:05.485344+0000","last_clean":"2024-09-10T10:29:05.485344+0000","last_became_active":"2024-09-10T10:28:24.373921+0000","last_became_peered":"2024-09-10T10:28:24.373921+0000","last_unstale":"2024-09-10T10:29:05.485344+0000","last_undegraded":"2024-09-10T10:29:05.485344+0000","last_fullsized":"2024-09-10T10:29:05.485344+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:36.923293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"64'15","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688969+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:29:05.688969+0000","last_peered":"2024-09-10T10:29:05.688969+0000","last_clean":"2024-09-10T10:29:05.688969+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:29:05.688969+0000","last_undegraded":"2024-09-10T10:29:05.688969+0000","last_fullsized":"2024-09-10T10:29:05.688969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:32.753892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831546+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:29:05.831546+0000","last_peered":"2024-09-10T10:29:05.831546+0000","last_clean":"2024-09-10T10:29:05.831546+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:29:05.831546+0000","last_undegraded":"2024-09-10T10:29:05.831546+0000","last_fullsized":"2024-09-10T10:29:05.831546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688907+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:29:05.688907+0000","last_peered":"2024-09-10T10:29:05.688907+0000","last_clean":"2024-09-10T10:29:05.688907+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:29:05.688907+0000","last_undegraded":"2024-09-10T10:29:05.688907+0000","last_fullsized":"2024-09-10T10:29:05.688907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521226+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:29:05.521226+0000","last_peered":"2024-09-10T10:29:05.521226+0000","last_clean":"2024-09-10T10:29:05.521226+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:29:05.521226+0000","last_undegraded":"2024-09-10T10:29:05.521226+0000","last_fullsized":"2024-09-10T10:29:05.521226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893084+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:29:05.893084+0000","last_peered":"2024-09-10T10:29:05.893084+0000","last_clean":"2024-09-10T10:29:05.893084+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:29:05.893084+0000","last_undegraded":"2024-09-10T10:29:05.893084+0000","last_fullsized":"2024-09-10T10:29:05.893084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:41.875855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486566+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:29:05.486566+0000","last_peered":"2024-09-10T10:29:05.486566+0000","last_clean":"2024-09-10T10:29:05.486566+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:29:05.486566+0000","last_undegraded":"2024-09-10T10:29:05.486566+0000","last_fullsized":"2024-09-10T10:29:05.486566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688951+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:29:05.688951+0000","last_peered":"2024-09-10T10:29:05.688951+0000","last_clean":"2024-09-10T10:29:05.688951+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.688951+0000","last_undegraded":"2024-09-10T10:29:05.688951+0000","last_fullsized":"2024-09-10T10:29:05.688951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688451+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:29:05.688451+0000","last_peered":"2024-09-10T10:29:05.688451+0000","last_clean":"2024-09-10T10:29:05.688451+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:29:05.688451+0000","last_undegraded":"2024-09-10T10:29:05.688451+0000","last_fullsized":"2024-09-10T10:29:05.688451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956082+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:29:05.956082+0000","last_peered":"2024-09-10T10:29:05.956082+0000","last_clean":"2024-09-10T10:29:05.956082+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:29:05.956082+0000","last_undegraded":"2024-09-10T10:29:05.956082+0000","last_fullsized":"2024-09-10T10:29:05.956082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486359+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:29:05.486359+0000","last_peered":"2024-09-10T10:29:05.486359+0000","last_clean":"2024-09-10T10:29:05.486359+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:29:05.486359+0000","last_undegraded":"2024-09-10T10:29:05.486359+0000","last_fullsized":"2024-09-10T10:29:05.486359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486083+0000","last_change":"2024-09-10T10:28:24.244988+0000","last_active":"2024-09-10T10:29:05.486083+0000","last_peered":"2024-09-10T10:29:05.486083+0000","last_clean":"2024-09-10T10:29:05.486083+0000","last_became_active":"2024-09-10T10:28:24.244556+0000","last_became_peered":"2024-09-10T10:28:24.244556+0000","last_unstale":"2024-09-10T10:29:05.486083+0000","last_undegraded":"2024-09-10T10:29:05.486083+0000","last_fullsized":"2024-09-10T10:29:05.486083+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:08.127492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688685+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:29:05.688685+0000","last_peered":"2024-09-10T10:29:05.688685+0000","last_clean":"2024-09-10T10:29:05.688685+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:29:05.688685+0000","last_undegraded":"2024-09-10T10:29:05.688685+0000","last_fullsized":"2024-09-10T10:29:05.688685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'9","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482578+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:29:05.482578+0000","last_peered":"2024-09-10T10:29:05.482578+0000","last_clean":"2024-09-10T10:29:05.482578+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.482578+0000","last_undegraded":"2024-09-10T10:29:05.482578+0000","last_fullsized":"2024-09-10T10:29:05.482578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688718+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:29:05.688718+0000","last_peered":"2024-09-10T10:29:05.688718+0000","last_clean":"2024-09-10T10:29:05.688718+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:29:05.688718+0000","last_undegraded":"2024-09-10T10:29:05.688718+0000","last_fullsized":"2024-09-10T10:29:05.688718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892835+0000","last_change":"2024-09-10T10:28:59.446889+0000","last_active":"2024-09-10T10:29:05.892835+0000","last_peered":"2024-09-10T10:29:05.892835+0000","last_clean":"2024-09-10T10:29:05.892835+0000","last_became_active":"2024-09-10T10:28:59.446544+0000","last_became_peered":"2024-09-10T10:28:59.446544+0000","last_unstale":"2024-09-10T10:29:05.892835+0000","last_undegraded":"2024-09-10T10:29:05.892835+0000","last_fullsized":"2024-09-10T10:29:05.892835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:32.352466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520235+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:29:05.520235+0000","last_peered":"2024-09-10T10:29:05.520235+0000","last_clean":"2024-09-10T10:29:05.520235+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:29:05.520235+0000","last_undegraded":"2024-09-10T10:29:05.520235+0000","last_fullsized":"2024-09-10T10:29:05.520235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893277+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:29:05.893277+0000","last_peered":"2024-09-10T10:29:05.893277+0000","last_clean":"2024-09-10T10:29:05.893277+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:29:05.893277+0000","last_undegraded":"2024-09-10T10:29:05.893277+0000","last_fullsized":"2024-09-10T10:29:05.893277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831398+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:29:05.831398+0000","last_peered":"2024-09-10T10:29:05.831398+0000","last_clean":"2024-09-10T10:29:05.831398+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:29:05.831398+0000","last_undegraded":"2024-09-10T10:29:05.831398+0000","last_fullsized":"2024-09-10T10:29:05.831398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486160+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:29:05.486160+0000","last_peered":"2024-09-10T10:29:05.486160+0000","last_clean":"2024-09-10T10:29:05.486160+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:29:05.486160+0000","last_undegraded":"2024-09-10T10:29:05.486160+0000","last_fullsized":"2024-09-10T10:29:05.486160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":281,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485314+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:29:05.485314+0000","last_peered":"2024-09-10T10:29:05.485314+0000","last_clean":"2024-09-10T10:29:05.485314+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:29:05.485314+0000","last_undegraded":"2024-09-10T10:29:05.485314+0000","last_fullsized":"2024-09-10T10:29:05.485314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521019+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:29:05.521019+0000","last_peered":"2024-09-10T10:29:05.521019+0000","last_clean":"2024-09-10T10:29:05.521019+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:29:05.521019+0000","last_undegraded":"2024-09-10T10:29:05.521019+0000","last_fullsized":"2024-09-10T10:29:05.521019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482817+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:29:05.482817+0000","last_peered":"2024-09-10T10:29:05.482817+0000","last_clean":"2024-09-10T10:29:05.482817+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:29:05.482817+0000","last_undegraded":"2024-09-10T10:29:05.482817+0000","last_fullsized":"2024-09-10T10:29:05.482817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484865+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:29:05.484865+0000","last_peered":"2024-09-10T10:29:05.484865+0000","last_clean":"2024-09-10T10:29:05.484865+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:29:05.484865+0000","last_undegraded":"2024-09-10T10:29:05.484865+0000","last_fullsized":"2024-09-10T10:29:05.484865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486504+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:29:05.486504+0000","last_peered":"2024-09-10T10:29:05.486504+0000","last_clean":"2024-09-10T10:29:05.486504+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:29:05.486504+0000","last_undegraded":"2024-09-10T10:29:05.486504+0000","last_fullsized":"2024-09-10T10:29:05.486504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688628+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:29:05.688628+0000","last_peered":"2024-09-10T10:29:05.688628+0000","last_clean":"2024-09-10T10:29:05.688628+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:29:05.688628+0000","last_undegraded":"2024-09-10T10:29:05.688628+0000","last_fullsized":"2024-09-10T10:29:05.688628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893128+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:29:05.893128+0000","last_peered":"2024-09-10T10:29:05.893128+0000","last_clean":"2024-09-10T10:29:05.893128+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:29:05.893128+0000","last_undegraded":"2024-09-10T10:29:05.893128+0000","last_fullsized":"2024-09-10T10:29:05.893128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521367+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:29:05.521367+0000","last_peered":"2024-09-10T10:29:05.521367+0000","last_clean":"2024-09-10T10:29:05.521367+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:29:05.521367+0000","last_undegraded":"2024-09-10T10:29:05.521367+0000","last_fullsized":"2024-09-10T10:29:05.521367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483041+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:29:05.483041+0000","last_peered":"2024-09-10T10:29:05.483041+0000","last_clean":"2024-09-10T10:29:05.483041+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:29:05.483041+0000","last_undegraded":"2024-09-10T10:29:05.483041+0000","last_fullsized":"2024-09-10T10:29:05.483041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485530+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:29:05.485530+0000","last_peered":"2024-09-10T10:29:05.485530+0000","last_clean":"2024-09-10T10:29:05.485530+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:29:05.485530+0000","last_undegraded":"2024-09-10T10:29:05.485530+0000","last_fullsized":"2024-09-10T10:29:05.485530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521232+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:29:05.521232+0000","last_peered":"2024-09-10T10:29:05.521232+0000","last_clean":"2024-09-10T10:29:05.521232+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:29:05.521232+0000","last_undegraded":"2024-09-10T10:29:05.521232+0000","last_fullsized":"2024-09-10T10:29:05.521232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956264+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:29:05.956264+0000","last_peered":"2024-09-10T10:29:05.956264+0000","last_clean":"2024-09-10T10:29:05.956264+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:29:05.956264+0000","last_undegraded":"2024-09-10T10:29:05.956264+0000","last_fullsized":"2024-09-10T10:29:05.956264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955704+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:29:05.955704+0000","last_peered":"2024-09-10T10:29:05.955704+0000","last_clean":"2024-09-10T10:29:05.955704+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:29:05.955704+0000","last_undegraded":"2024-09-10T10:29:05.955704+0000","last_fullsized":"2024-09-10T10:29:05.955704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'18","reported_seq":147,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482470+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:29:05.482470+0000","last_peered":"2024-09-10T10:29:05.482470+0000","last_clean":"2024-09-10T10:29:05.482470+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:29:05.482470+0000","last_undegraded":"2024-09-10T10:29:05.482470+0000","last_fullsized":"2024-09-10T10:29:05.482470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893225+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:29:05.893225+0000","last_peered":"2024-09-10T10:29:05.893225+0000","last_clean":"2024-09-10T10:29:05.893225+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:29:05.893225+0000","last_undegraded":"2024-09-10T10:29:05.893225+0000","last_fullsized":"2024-09-10T10:29:05.893225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:01.355916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":160,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:28:59.453146+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:28:59.446453+0000","last_became_peered":"2024-09-10T10:28:59.446453+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:31.347510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1176,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832175+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:29:05.832175+0000","last_peered":"2024-09-10T10:29:05.832175+0000","last_clean":"2024-09-10T10:29:05.832175+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:29:05.832175+0000","last_undegraded":"2024-09-10T10:29:05.832175+0000","last_fullsized":"2024-09-10T10:29:05.832175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520795+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:29:05.520795+0000","last_peered":"2024-09-10T10:29:05.520795+0000","last_clean":"2024-09-10T10:29:05.520795+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:29:05.520795+0000","last_undegraded":"2024-09-10T10:29:05.520795+0000","last_fullsized":"2024-09-10T10:29:05.520795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"60'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486347+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:29:05.486347+0000","last_peered":"2024-09-10T10:29:05.486347+0000","last_clean":"2024-09-10T10:29:05.486347+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:29:05.486347+0000","last_undegraded":"2024-09-10T10:29:05.486347+0000","last_fullsized":"2024-09-10T10:29:05.486347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832226+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:29:05.832226+0000","last_peered":"2024-09-10T10:29:05.832226+0000","last_clean":"2024-09-10T10:29:05.832226+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:29:05.832226+0000","last_undegraded":"2024-09-10T10:29:05.832226+0000","last_fullsized":"2024-09-10T10:29:05.832226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892463+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:29:05.892463+0000","last_peered":"2024-09-10T10:29:05.892463+0000","last_clean":"2024-09-10T10:29:05.892463+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:29:05.892463+0000","last_undegraded":"2024-09-10T10:29:05.892463+0000","last_fullsized":"2024-09-10T10:29:05.892463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:26.715166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893348+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:29:05.893348+0000","last_peered":"2024-09-10T10:29:05.893348+0000","last_clean":"2024-09-10T10:29:05.893348+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:29:05.893348+0000","last_undegraded":"2024-09-10T10:29:05.893348+0000","last_fullsized":"2024-09-10T10:29:05.893348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486563+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:29:05.486563+0000","last_peered":"2024-09-10T10:29:05.486563+0000","last_clean":"2024-09-10T10:29:05.486563+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:29:05.486563+0000","last_undegraded":"2024-09-10T10:29:05.486563+0000","last_fullsized":"2024-09-10T10:29:05.486563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520992+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:29:05.520992+0000","last_peered":"2024-09-10T10:29:05.520992+0000","last_clean":"2024-09-10T10:29:05.520992+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:29:05.520992+0000","last_undegraded":"2024-09-10T10:29:05.520992+0000","last_fullsized":"2024-09-10T10:29:05.520992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"69'27","reported_seq":156,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485568+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:29:05.485568+0000","last_peered":"2024-09-10T10:29:05.485568+0000","last_clean":"2024-09-10T10:29:05.485568+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:29:05.485568+0000","last_undegraded":"2024-09-10T10:29:05.485568+0000","last_fullsized":"2024-09-10T10:29:05.485568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37014,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"60'3","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688715+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:29:05.688715+0000","last_peered":"2024-09-10T10:29:05.688715+0000","last_clean":"2024-09-10T10:29:05.688715+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:29:05.688715+0000","last_undegraded":"2024-09-10T10:29:05.688715+0000","last_fullsized":"2024-09-10T10:29:05.688715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955123+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:29:05.955123+0000","last_peered":"2024-09-10T10:29:05.955123+0000","last_clean":"2024-09-10T10:29:05.955123+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:29:05.955123+0000","last_undegraded":"2024-09-10T10:29:05.955123+0000","last_fullsized":"2024-09-10T10:29:05.955123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'8","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688658+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:29:05.688658+0000","last_peered":"2024-09-10T10:29:05.688658+0000","last_clean":"2024-09-10T10:29:05.688658+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:29:05.688658+0000","last_undegraded":"2024-09-10T10:29:05.688658+0000","last_fullsized":"2024-09-10T10:29:05.688658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955996+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:29:05.955996+0000","last_peered":"2024-09-10T10:29:05.955996+0000","last_clean":"2024-09-10T10:29:05.955996+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:29:05.955996+0000","last_undegraded":"2024-09-10T10:29:05.955996+0000","last_fullsized":"2024-09-10T10:29:05.955996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892560+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:29:05.892560+0000","last_peered":"2024-09-10T10:29:05.892560+0000","last_clean":"2024-09-10T10:29:05.892560+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:29:05.892560+0000","last_undegraded":"2024-09-10T10:29:05.892560+0000","last_fullsized":"2024-09-10T10:29:05.892560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486525+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:29:05.486525+0000","last_peered":"2024-09-10T10:29:05.486525+0000","last_clean":"2024-09-10T10:29:05.486525+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:29:05.486525+0000","last_undegraded":"2024-09-10T10:29:05.486525+0000","last_fullsized":"2024-09-10T10:29:05.486525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831619+0000","last_change":"2024-09-10T10:28:24.248827+0000","last_active":"2024-09-10T10:29:05.831619+0000","last_peered":"2024-09-10T10:29:05.831619+0000","last_clean":"2024-09-10T10:29:05.831619+0000","last_became_active":"2024-09-10T10:28:24.248395+0000","last_became_peered":"2024-09-10T10:28:24.248395+0000","last_unstale":"2024-09-10T10:29:05.831619+0000","last_undegraded":"2024-09-10T10:29:05.831619+0000","last_fullsized":"2024-09-10T10:29:05.831619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:05.306669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":106,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688561+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:29:05.688561+0000","last_peered":"2024-09-10T10:29:05.688561+0000","last_clean":"2024-09-10T10:29:05.688561+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:29:05.688561+0000","last_undegraded":"2024-09-10T10:29:05.688561+0000","last_fullsized":"2024-09-10T10:29:05.688561+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:57.377112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"60'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485617+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:29:05.485617+0000","last_peered":"2024-09-10T10:29:05.485617+0000","last_clean":"2024-09-10T10:29:05.485617+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:29:05.485617+0000","last_undegraded":"2024-09-10T10:29:05.485617+0000","last_fullsized":"2024-09-10T10:29:05.485617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831826+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:29:05.831826+0000","last_peered":"2024-09-10T10:29:05.831826+0000","last_clean":"2024-09-10T10:29:05.831826+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:29:05.831826+0000","last_undegraded":"2024-09-10T10:29:05.831826+0000","last_fullsized":"2024-09-10T10:29:05.831826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893171+0000","last_change":"2024-09-10T10:28:24.376473+0000","last_active":"2024-09-10T10:29:05.893171+0000","last_peered":"2024-09-10T10:29:05.893171+0000","last_clean":"2024-09-10T10:29:05.893171+0000","last_became_active":"2024-09-10T10:28:24.376079+0000","last_became_peered":"2024-09-10T10:28:24.376079+0000","last_unstale":"2024-09-10T10:29:05.893171+0000","last_undegraded":"2024-09-10T10:29:05.893171+0000","last_fullsized":"2024-09-10T10:29:05.893171+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:53.600117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,1],"acting":[0,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688397+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:29:05.688397+0000","last_peered":"2024-09-10T10:29:05.688397+0000","last_clean":"2024-09-10T10:29:05.688397+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:29:05.688397+0000","last_undegraded":"2024-09-10T10:29:05.688397+0000","last_fullsized":"2024-09-10T10:29:05.688397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485970+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:29:05.485970+0000","last_peered":"2024-09-10T10:29:05.485970+0000","last_clean":"2024-09-10T10:29:05.485970+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:29:05.485970+0000","last_undegraded":"2024-09-10T10:29:05.485970+0000","last_fullsized":"2024-09-10T10:29:05.485970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485377+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:29:05.485377+0000","last_peered":"2024-09-10T10:29:05.485377+0000","last_clean":"2024-09-10T10:29:05.485377+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:29:05.485377+0000","last_undegraded":"2024-09-10T10:29:05.485377+0000","last_fullsized":"2024-09-10T10:29:05.485377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831708+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:29:05.831708+0000","last_peered":"2024-09-10T10:29:05.831708+0000","last_clean":"2024-09-10T10:29:05.831708+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:29:05.831708+0000","last_undegraded":"2024-09-10T10:29:05.831708+0000","last_fullsized":"2024-09-10T10:29:05.831708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832298+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:29:05.832298+0000","last_peered":"2024-09-10T10:29:05.832298+0000","last_clean":"2024-09-10T10:29:05.832298+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:29:05.832298+0000","last_undegraded":"2024-09-10T10:29:05.832298+0000","last_fullsized":"2024-09-10T10:29:05.832298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485185+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:29:05.485185+0000","last_peered":"2024-09-10T10:29:05.485185+0000","last_clean":"2024-09-10T10:29:05.485185+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:29:05.485185+0000","last_undegraded":"2024-09-10T10:29:05.485185+0000","last_fullsized":"2024-09-10T10:29:05.485185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893219+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:29:05.893219+0000","last_peered":"2024-09-10T10:29:05.893219+0000","last_clean":"2024-09-10T10:29:05.893219+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:29:05.893219+0000","last_undegraded":"2024-09-10T10:29:05.893219+0000","last_fullsized":"2024-09-10T10:29:05.893219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482644+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:29:05.482644+0000","last_peered":"2024-09-10T10:29:05.482644+0000","last_clean":"2024-09-10T10:29:05.482644+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:29:05.482644+0000","last_undegraded":"2024-09-10T10:29:05.482644+0000","last_fullsized":"2024-09-10T10:29:05.482644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'6","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831961+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:29:05.831961+0000","last_peered":"2024-09-10T10:29:05.831961+0000","last_clean":"2024-09-10T10:29:05.831961+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:29:05.831961+0000","last_undegraded":"2024-09-10T10:29:05.831961+0000","last_fullsized":"2024-09-10T10:29:05.831961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955040+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:29:05.955040+0000","last_peered":"2024-09-10T10:29:05.955040+0000","last_clean":"2024-09-10T10:29:05.955040+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:29:05.955040+0000","last_undegraded":"2024-09-10T10:29:05.955040+0000","last_fullsized":"2024-09-10T10:29:05.955040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832152+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:29:05.832152+0000","last_peered":"2024-09-10T10:29:05.832152+0000","last_clean":"2024-09-10T10:29:05.832152+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:29:05.832152+0000","last_undegraded":"2024-09-10T10:29:05.832152+0000","last_fullsized":"2024-09-10T10:29:05.832152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484806+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:29:05.484806+0000","last_peered":"2024-09-10T10:29:05.484806+0000","last_clean":"2024-09-10T10:29:05.484806+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:29:05.484806+0000","last_undegraded":"2024-09-10T10:29:05.484806+0000","last_fullsized":"2024-09-10T10:29:05.484806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482528+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:29:05.482528+0000","last_peered":"2024-09-10T10:29:05.482528+0000","last_clean":"2024-09-10T10:29:05.482528+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:29:05.482528+0000","last_undegraded":"2024-09-10T10:29:05.482528+0000","last_fullsized":"2024-09-10T10:29:05.482528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486220+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:29:05.486220+0000","last_peered":"2024-09-10T10:29:05.486220+0000","last_clean":"2024-09-10T10:29:05.486220+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:29:05.486220+0000","last_undegraded":"2024-09-10T10:29:05.486220+0000","last_fullsized":"2024-09-10T10:29:05.486220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893191+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:29:05.893191+0000","last_peered":"2024-09-10T10:29:05.893191+0000","last_clean":"2024-09-10T10:29:05.893191+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:29:05.893191+0000","last_undegraded":"2024-09-10T10:29:05.893191+0000","last_fullsized":"2024-09-10T10:29:05.893191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832375+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:29:05.832375+0000","last_peered":"2024-09-10T10:29:05.832375+0000","last_clean":"2024-09-10T10:29:05.832375+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:29:05.832375+0000","last_undegraded":"2024-09-10T10:29:05.832375+0000","last_fullsized":"2024-09-10T10:29:05.832375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688489+0000","last_change":"2024-09-10T10:26:41.792496+0000","last_active":"2024-09-10T10:29:05.688489+0000","last_peered":"2024-09-10T10:29:05.688489+0000","last_clean":"2024-09-10T10:29:05.688489+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:29:05.688489+0000","last_undegraded":"2024-09-10T10:29:05.688489+0000","last_fullsized":"2024-09-10T10:29:05.688489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485637+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:29:05.485637+0000","last_peered":"2024-09-10T10:29:05.485637+0000","last_clean":"2024-09-10T10:29:05.485637+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:29:05.485637+0000","last_undegraded":"2024-09-10T10:29:05.485637+0000","last_fullsized":"2024-09-10T10:29:05.485637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832269+0000","last_change":"2024-09-10T10:28:59.446917+0000","last_active":"2024-09-10T10:29:05.832269+0000","last_peered":"2024-09-10T10:29:05.832269+0000","last_clean":"2024-09-10T10:29:05.832269+0000","last_became_active":"2024-09-10T10:28:59.446546+0000","last_became_peered":"2024-09-10T10:28:59.446546+0000","last_unstale":"2024-09-10T10:29:05.832269+0000","last_undegraded":"2024-09-10T10:29:05.832269+0000","last_fullsized":"2024-09-10T10:29:05.832269+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:30.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485795+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:29:05.485795+0000","last_peered":"2024-09-10T10:29:05.485795+0000","last_clean":"2024-09-10T10:29:05.485795+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:29:05.485795+0000","last_undegraded":"2024-09-10T10:29:05.485795+0000","last_fullsized":"2024-09-10T10:29:05.485795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831886+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:29:05.831886+0000","last_peered":"2024-09-10T10:29:05.831886+0000","last_clean":"2024-09-10T10:29:05.831886+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:29:05.831886+0000","last_undegraded":"2024-09-10T10:29:05.831886+0000","last_fullsized":"2024-09-10T10:29:05.831886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486218+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:29:05.486218+0000","last_peered":"2024-09-10T10:29:05.486218+0000","last_clean":"2024-09-10T10:29:05.486218+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:29:05.486218+0000","last_undegraded":"2024-09-10T10:29:05.486218+0000","last_fullsized":"2024-09-10T10:29:05.486218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892472+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:29:05.892472+0000","last_peered":"2024-09-10T10:29:05.892472+0000","last_clean":"2024-09-10T10:29:05.892472+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:29:05.892472+0000","last_undegraded":"2024-09-10T10:29:05.892472+0000","last_fullsized":"2024-09-10T10:29:05.892472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688898+0000","last_change":"2024-09-10T10:28:23.249392+0000","last_active":"2024-09-10T10:29:05.688898+0000","last_peered":"2024-09-10T10:29:05.688898+0000","last_clean":"2024-09-10T10:29:05.688898+0000","last_became_active":"2024-09-10T10:28:23.249039+0000","last_became_peered":"2024-09-10T10:28:23.249039+0000","last_unstale":"2024-09-10T10:29:05.688898+0000","last_undegraded":"2024-09-10T10:29:05.688898+0000","last_fullsized":"2024-09-10T10:29:05.688898+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:46.656488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485234+0000","last_change":"2024-09-10T10:28:24.244885+0000","last_active":"2024-09-10T10:29:05.485234+0000","last_peered":"2024-09-10T10:29:05.485234+0000","last_clean":"2024-09-10T10:29:05.485234+0000","last_became_active":"2024-09-10T10:28:24.244560+0000","last_became_peered":"2024-09-10T10:28:24.244560+0000","last_unstale":"2024-09-10T10:29:05.485234+0000","last_undegraded":"2024-09-10T10:29:05.485234+0000","last_fullsized":"2024-09-10T10:29:05.485234+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:36.642112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483017+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:29:05.483017+0000","last_peered":"2024-09-10T10:29:05.483017+0000","last_clean":"2024-09-10T10:29:05.483017+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:29:05.483017+0000","last_undegraded":"2024-09-10T10:29:05.483017+0000","last_fullsized":"2024-09-10T10:29:05.483017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486768+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:29:05.486768+0000","last_peered":"2024-09-10T10:29:05.486768+0000","last_clean":"2024-09-10T10:29:05.486768+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:29:05.486768+0000","last_undegraded":"2024-09-10T10:29:05.486768+0000","last_fullsized":"2024-09-10T10:29:05.486768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485724+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:29:05.485724+0000","last_peered":"2024-09-10T10:29:05.485724+0000","last_clean":"2024-09-10T10:29:05.485724+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:29:05.485724+0000","last_undegraded":"2024-09-10T10:29:05.485724+0000","last_fullsized":"2024-09-10T10:29:05.485724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688424+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:29:05.688424+0000","last_peered":"2024-09-10T10:29:05.688424+0000","last_clean":"2024-09-10T10:29:05.688424+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:29:05.688424+0000","last_undegraded":"2024-09-10T10:29:05.688424+0000","last_fullsized":"2024-09-10T10:29:05.688424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"60'1","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892613+0000","last_change":"2024-09-10T10:26:41.797405+0000","last_active":"2024-09-10T10:29:05.892613+0000","last_peered":"2024-09-10T10:29:05.892613+0000","last_clean":"2024-09-10T10:29:05.892613+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:29:05.892613+0000","last_undegraded":"2024-09-10T10:29:05.892613+0000","last_fullsized":"2024-09-10T10:29:05.892613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:24.938095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482510+0000","last_change":"2024-09-10T10:28:24.243479+0000","last_active":"2024-09-10T10:29:05.482510+0000","last_peered":"2024-09-10T10:29:05.482510+0000","last_clean":"2024-09-10T10:29:05.482510+0000","last_became_active":"2024-09-10T10:28:24.243088+0000","last_became_peered":"2024-09-10T10:28:24.243088+0000","last_unstale":"2024-09-10T10:29:05.482510+0000","last_undegraded":"2024-09-10T10:29:05.482510+0000","last_fullsized":"2024-09-10T10:29:05.482510+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:39.465491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":86,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831975+0000","last_change":"2024-09-10T10:27:58.327792+0000","last_active":"2024-09-10T10:29:05.831975+0000","last_peered":"2024-09-10T10:29:05.831975+0000","last_clean":"2024-09-10T10:29:05.831975+0000","last_became_active":"2024-09-10T10:27:58.327502+0000","last_became_peered":"2024-09-10T10:27:58.327502+0000","last_unstale":"2024-09-10T10:29:05.831975+0000","last_undegraded":"2024-09-10T10:29:05.831975+0000","last_fullsized":"2024-09-10T10:29:05.831975+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485410+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:29:05.485410+0000","last_peered":"2024-09-10T10:29:05.485410+0000","last_clean":"2024-09-10T10:29:05.485410+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:29:05.485410+0000","last_undegraded":"2024-09-10T10:29:05.485410+0000","last_fullsized":"2024-09-10T10:29:05.485410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485574+0000","last_change":"2024-09-10T10:28:40.345087+0000","last_active":"2024-09-10T10:29:05.485574+0000","last_peered":"2024-09-10T10:29:05.485574+0000","last_clean":"2024-09-10T10:29:05.485574+0000","last_became_active":"2024-09-10T10:28:40.344766+0000","last_became_peered":"2024-09-10T10:28:40.344766+0000","last_unstale":"2024-09-10T10:29:05.485574+0000","last_undegraded":"2024-09-10T10:29:05.485574+0000","last_fullsized":"2024-09-10T10:29:05.485574+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521372+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:29:05.521372+0000","last_peered":"2024-09-10T10:29:05.521372+0000","last_clean":"2024-09-10T10:29:05.521372+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:29:05.521372+0000","last_undegraded":"2024-09-10T10:29:05.521372+0000","last_fullsized":"2024-09-10T10:29:05.521372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831395+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:29:05.831395+0000","last_peered":"2024-09-10T10:29:05.831395+0000","last_clean":"2024-09-10T10:29:05.831395+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:29:05.831395+0000","last_undegraded":"2024-09-10T10:29:05.831395+0000","last_fullsized":"2024-09-10T10:29:05.831395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520601+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:29:05.520601+0000","last_peered":"2024-09-10T10:29:05.520601+0000","last_clean":"2024-09-10T10:29:05.520601+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:29:05.520601+0000","last_undegraded":"2024-09-10T10:29:05.520601+0000","last_fullsized":"2024-09-10T10:29:05.520601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486096+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:29:05.486096+0000","last_peered":"2024-09-10T10:29:05.486096+0000","last_clean":"2024-09-10T10:29:05.486096+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:29:05.486096+0000","last_undegraded":"2024-09-10T10:29:05.486096+0000","last_fullsized":"2024-09-10T10:29:05.486096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520639+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:29:05.520639+0000","last_peered":"2024-09-10T10:29:05.520639+0000","last_clean":"2024-09-10T10:29:05.520639+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:29:05.520639+0000","last_undegraded":"2024-09-10T10:29:05.520639+0000","last_fullsized":"2024-09-10T10:29:05.520639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486354+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:29:05.486354+0000","last_peered":"2024-09-10T10:29:05.486354+0000","last_clean":"2024-09-10T10:29:05.486354+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:29:05.486354+0000","last_undegraded":"2024-09-10T10:29:05.486354+0000","last_fullsized":"2024-09-10T10:29:05.486354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":124,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.892681+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:29:05.892681+0000","last_peered":"2024-09-10T10:29:05.892681+0000","last_clean":"2024-09-10T10:29:05.892681+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:29:05.892681+0000","last_undegraded":"2024-09-10T10:29:05.892681+0000","last_fullsized":"2024-09-10T10:29:05.892681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:06.012091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688961+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:29:05.688961+0000","last_peered":"2024-09-10T10:29:05.688961+0000","last_clean":"2024-09-10T10:29:05.688961+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:29:05.688961+0000","last_undegraded":"2024-09-10T10:29:05.688961+0000","last_fullsized":"2024-09-10T10:29:05.688961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521230+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:29:05.521230+0000","last_peered":"2024-09-10T10:29:05.521230+0000","last_clean":"2024-09-10T10:29:05.521230+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:29:05.521230+0000","last_undegraded":"2024-09-10T10:29:05.521230+0000","last_fullsized":"2024-09-10T10:29:05.521230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44104,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":247,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":88208,"data_compressed":3324,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1019904,"data_stored":1074371,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":974848,"data_stored":519451,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":966656,"data_stored":485579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":482977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":520153,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":954368,"data_stored":483521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":483179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1011712,"data_stored":1072889,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":38506,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":37804,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:29:11.419 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:29:11.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.551+0000 7fe3c87c4640 1 Processor -- start 2024-09-10T10:29:11.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.551+0000 7fe3c87c4640 1 -- start start 2024-09-10T10:29:11.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.552+0000 7fe3c87c4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe3c0152d60 0x7fe3c00c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.552+0000 7fe3c87c4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe3c00c3df0 0x7fe3c00c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.552+0000 7fe3c87c4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3c015c230 con 0x7fe3c00c8df0 2024-09-10T10:29:11.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.552+0000 7fe3c87c4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c015cdb0 con 0x7fe3c0152d60 2024-09-10T10:29:11.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.552+0000 7fe3c87c4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c00cf3c0 con 0x7fe3c00c3df0 2024-09-10T10:29:11.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.552+0000 7fe3c6560640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe3c0152d60 0x7fe3c00c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.553+0000 7fe3c6560640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe3c0152d60 0x7fe3c00c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42498/0 (socket says 172.21.15.120:42498) 2024-09-10T10:29:11.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.553+0000 7fe3c5d5f640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fe3c00c8df0 0x7fe3c0159020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41578/0 (socket says 172.21.15.120:41578) 2024-09-10T10:29:11.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.553+0000 7fe3c6560640 1 -- 172.21.15.120:0/1300175054 learned_addr learned my addr 172.21.15.120:0/1300175054 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:11.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.553+0000 7fe3c6d61640 1 --2- 172.21.15.120:0/1300175054 >> v2:172.21.15.134:3300/0 conn(0x7fe3c00c3df0 0x7fe3c00c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.553+0000 7fe3c6560640 1 -- 172.21.15.120:0/1300175054 >> v2:172.21.15.134:3300/0 conn(0x7fe3c00c3df0 msgr2=0x7fe3c00c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:11.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.553+0000 7fe3c6560640 1 --2- 172.21.15.120:0/1300175054 >> v2:172.21.15.134:3300/0 conn(0x7fe3c00c3df0 0x7fe3c00c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.553+0000 7fe3c6560640 1 -- 172.21.15.120:0/1300175054 >> v1:172.21.15.120:6789/0 conn(0x7fe3c00c8df0 legacy=0x7fe3c0159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.553+0000 7fe3c6560640 1 -- 172.21.15.120:0/1300175054 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c015c510 con 0x7fe3c0152d60 2024-09-10T10:29:11.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.553+0000 7fe3c555e640 1 -- 172.21.15.120:0/1300175054 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525072701 0 0) 0x7fe3c015c230 con 0x7fe3c00c8df0 2024-09-10T10:29:11.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c6560640 1 --2- 172.21.15.120:0/1300175054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 0x7fe3c00c3410 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0072680 tx=0x7fe3bc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f8c0a85aa4356 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:11.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c555e640 1 -- 172.21.15.120:0/1300175054 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc037070 con 0x7fe3c0152d60 2024-09-10T10:29:11.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c555e640 1 -- 172.21.15.120:0/1300175054 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3bc02ae50 con 0x7fe3c0152d60 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c555e640 1 -- 172.21.15.120:0/1300175054 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc032860 con 0x7fe3c0152d60 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c87c4640 1 -- 172.21.15.120:0/1300175054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 msgr2=0x7fe3c00c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c87c4640 1 --2- 172.21.15.120:0/1300175054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 0x7fe3c00c3410 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0072680 tx=0x7fe3bc02a300 comp rx=0 tx=0).stop 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c87c4640 1 -- 172.21.15.120:0/1300175054 shutdown_connections 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c87c4640 1 --2- 172.21.15.120:0/1300175054 >> v2:172.21.15.134:3300/0 conn(0x7fe3c00c3df0 0x7fe3c00c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c87c4640 1 --2- 172.21.15.120:0/1300175054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 0x7fe3c00c3410 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c87c4640 1 -- 172.21.15.120:0/1300175054 >> 172.21.15.120:0/1300175054 conn(0x7fe3c00716e0 msgr2=0x7fe3c0071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c87c4640 1 -- 172.21.15.120:0/1300175054 shutdown_connections 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.554+0000 7fe3c87c4640 1 -- 172.21.15.120:0/1300175054 wait complete. 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.555+0000 7fe3c87c4640 1 Processor -- start 2024-09-10T10:29:11.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.555+0000 7fe3c87c4640 1 -- start start 2024-09-10T10:29:11.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c87c4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3c00c3df0 0x7fe3c01518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c87c4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c00c8df0 0x7fe3c0151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c87c4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 0x7fe3c00c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c87c4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c015cad0 con 0x7fe3c0152d60 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c87c4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0158b90 con 0x7fe3c00c8df0 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c87c4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c0159730 con 0x7fe3c00c3df0 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c6d61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 0x7fe3c00c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c6d61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 0x7fe3c00c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42510/0 (socket says 172.21.15.120:42510) 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c6d61640 1 -- 172.21.15.120:0/164076515 learned_addr learned my addr 172.21.15.120:0/164076515 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c5d5f640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c00c8df0 0x7fe3c0151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.556+0000 7fe3c6560640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3c00c3df0 0x7fe3c01518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3c6d61640 1 -- 172.21.15.120:0/164076515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3c00c3df0 msgr2=0x7fe3c01518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3c6d61640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3c00c3df0 0x7fe3c01518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3c6d61640 1 -- 172.21.15.120:0/164076515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c00c8df0 msgr2=0x7fe3c0151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3c6d61640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c00c8df0 0x7fe3c0151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3c6d61640 1 -- 172.21.15.120:0/164076515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c00c72f0 con 0x7fe3c0152d60 2024-09-10T10:29:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3c6d61640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 0x7fe3c00c7ab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8007e70 tx=0x7fe3b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3b6ffd640 1 -- 172.21.15.120:0/164076515 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8014420 con 0x7fe3c0152d60 2024-09-10T10:29:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3b6ffd640 1 -- 172.21.15.120:0/164076515 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b8019070 con 0x7fe3c0152d60 2024-09-10T10:29:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3c87c4640 1 -- 172.21.15.120:0/164076515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c0153440 con 0x7fe3c0152d60 2024-09-10T10:29:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3b6ffd640 1 -- 172.21.15.120:0/164076515 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8014820 con 0x7fe3c0152d60 2024-09-10T10:29:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.557+0000 7fe3c87c4640 1 -- 172.21.15.120:0/164076515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c00c7590 con 0x7fe3c0152d60 2024-09-10T10:29:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.558+0000 7fe3c87c4640 1 -- 172.21.15.120:0/164076515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c01587d0 con 0x7fe3c0152d60 2024-09-10T10:29:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.558+0000 7fe3b6ffd640 1 -- 172.21.15.120:0/164076515 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3b80149c0 con 0x7fe3c0152d60 2024-09-10T10:29:11.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.559+0000 7fe3b6ffd640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe3a406be30 0x7fe3a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:11.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.559+0000 7fe3b6ffd640 1 -- 172.21.15.120:0/164076515 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fe3b808bbf0 con 0x7fe3c0152d60 2024-09-10T10:29:11.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.559+0000 7fe3c6560640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe3a406be30 0x7fe3a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:11.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.559+0000 7fe3c6560640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe3a406be30 0x7fe3a406e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc034000 tx=0x7fe3bc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:11.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.563+0000 7fe3b6ffd640 1 -- 172.21.15.120:0/164076515 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe3c01587d0 con 0x7fe3c0152d60 2024-09-10T10:29:11.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.730+0000 7fe3c87c4640 1 -- 172.21.15.120:0/164076515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3c015be60 con 0x7fe3c0152d60 2024-09-10T10:29:11.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.731+0000 7fe3b6ffd640 1 -- 172.21.15.120:0/164076515 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+19282 (secure 0 0 0) 0x7fe3b805a0a0 con 0x7fe3c0152d60 2024-09-10T10:29:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:11.733 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":78,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:29:05.476388+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2c","osds":[1,7]},{"pgid":"3.34","osds":[3,1]}],"pg_upmap_items":[{"pgid":"2.1d","mappings":[{"from":6,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":6}]},{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":6}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":3,"to":6},{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2},{"from":3,"to":0}]},{"pgid":"4.1e","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:29:11.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 -- 172.21.15.120:0/164076515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe3a406be30 msgr2=0x7fe3a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe3a406be30 0x7fe3a406e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc034000 tx=0x7fe3bc02e000 comp rx=0 tx=0).stop 2024-09-10T10:29:11.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 -- 172.21.15.120:0/164076515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 msgr2=0x7fe3c00c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 0x7fe3c00c7ab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8007e70 tx=0x7fe3b800e4b0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 -- 172.21.15.120:0/164076515 shutdown_connections 2024-09-10T10:29:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe3a406be30 0x7fe3a406e2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3c0152d60 0x7fe3c00c7ab0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c00c8df0 0x7fe3c0151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 --2- 172.21.15.120:0/164076515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3c00c3df0 0x7fe3c01518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 -- 172.21.15.120:0/164076515 >> 172.21.15.120:0/164076515 conn(0x7fe3c00716e0 msgr2=0x7fe3c00cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 -- 172.21.15.120:0/164076515 shutdown_connections 2024-09-10T10:29:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:11.734+0000 7fe3c87c4640 1 -- 172.21.15.120:0/164076515 wait complete. 2024-09-10T10:29:12.062 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:29:12.065 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:29:12.748 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T10:29:12.748 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:29:12.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.856+0000 7f8218f03640 1 Processor -- start 2024-09-10T10:29:12.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.857+0000 7f8218f03640 1 -- start start 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.857+0000 7f8218f03640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f82140cd120 0x7f821415ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.857+0000 7f8218f03640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82140cd500 0x7f82141549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f8218f03640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f821415acf0 con 0x7f82140b3690 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f8218f03640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f821415f010 con 0x7f82140cd120 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f8218f03640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f82140b3430 con 0x7f82140cd500 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f8212d9d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f82140b3690 0x7f82140b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41580/0 (socket says 172.21.15.120:41580) 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f8212d9d640 1 -- 172.21.15.120:0/2386638719 learned_addr learned my addr 172.21.15.120:0/2386638719 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f8211d9b640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82140cd500 0x7f82141549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821259c640 1 --2- 172.21.15.120:0/2386638719 >> v2:172.21.15.134:3300/0 conn(0x7f82140cd120 0x7f821415ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3264376210 0 0) 0x7f821415acf0 con 0x7f82140b3690 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8200003520 con 0x7f82140b3690 2024-09-10T10:29:12.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2087691553 0 0) 0x7f8200003520 con 0x7f82140b3690 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82140cd500 msgr2=0x7f82141549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82140cd500 0x7f82141549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 >> v2:172.21.15.134:3300/0 conn(0x7f82140cd120 msgr2=0x7f821415ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 --2- 172.21.15.120:0/2386638719 >> v2:172.21.15.134:3300/0 conn(0x7f82140cd120 0x7f821415ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821415afd0 con 0x7f82140b3690 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f820c0019f0 con 0x7f82140b3690 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 >> v1:172.21.15.120:6789/0 conn(0x7f82140b3690 legacy=0x7f82140b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82140cd500 0x7f82000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 0x7f8200004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8200004d40 0x7f82000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f821415f010 con 0x7f8200003f30 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82140b3430 con 0x7f8200004d40 2024-09-10T10:29:12.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f820000b9f0 con 0x7f82140cd500 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f8211d9b640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 0x7f8200004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f821259c640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82140cd500 0x7f82000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.858+0000 7f8212d9d640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8200004d40 0x7f82000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.859+0000 7f8211d9b640 1 -- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82140cd500 msgr2=0x7f82000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.859+0000 7f8211d9b640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82140cd500 0x7f82000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.859+0000 7f8211d9b640 1 -- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8200004d40 msgr2=0x7f82000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.859+0000 7f8211d9b640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8200004d40 0x7f82000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.859+0000 7f8211d9b640 1 -- 172.21.15.120:0/2386638719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8200003520 con 0x7f8200003f30 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.859+0000 7f8211d9b640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 0x7f8200004800 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8204009a90 tx=0x7f8200005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a031f2d7ddb9793 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.859+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204036070 con 0x7f8200003f30 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.859+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8204031400 con 0x7f8200003f30 2024-09-10T10:29:12.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.859+0000 7f821159a640 1 -- 172.21.15.120:0/2386638719 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204009690 con 0x7f8200003f30 2024-09-10T10:29:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.860+0000 7f8218f03640 1 -- 172.21.15.120:0/2386638719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 msgr2=0x7f8200004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.860+0000 7f8218f03640 1 --2- 172.21.15.120:0/2386638719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 0x7f8200004800 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8204009a90 tx=0x7f8200005830 comp rx=0 tx=0).stop 2024-09-10T10:29:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.860+0000 7f8212d9d640 1 -- 172.21.15.120:0/2386638719 reap_dead start 2024-09-10T10:29:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.860+0000 7f8218f03640 1 -- 172.21.15.120:0/2386638719 shutdown_connections 2024-09-10T10:29:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.860+0000 7f8218f03640 1 -- 172.21.15.120:0/2386638719 >> 172.21.15.120:0/2386638719 conn(0x7f8214067950 msgr2=0x7f8214067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.860+0000 7f8218f03640 1 -- 172.21.15.120:0/2386638719 shutdown_connections 2024-09-10T10:29:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.860+0000 7f8218f03640 1 -- 172.21.15.120:0/2386638719 wait complete. 2024-09-10T10:29:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.861+0000 7f8218f03640 1 Processor -- start 2024-09-10T10:29:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.861+0000 7f8218f03640 1 -- start start 2024-09-10T10:29:12.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f8218f03640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 0x7f8214149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f8218f03640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8214157120 0x7f821414b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f8218f03640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8214149dd0 0x7f821414b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f8218f03640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f821415b590 con 0x7f8200003f30 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f8218f03640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f821415b900 con 0x7f8214157120 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f8218f03640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f82140b4880 con 0x7f8214149dd0 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f821259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 0x7f8214149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f821259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 0x7f8214149890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42528/0 (socket says 172.21.15.120:42528) 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f821259c640 1 -- 172.21.15.120:0/3649657996 learned_addr learned my addr 172.21.15.120:0/3649657996 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f8211d9b640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8214157120 0x7f821414b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.862+0000 7f8212d9d640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8214149dd0 0x7f821414b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f821259c640 1 -- 172.21.15.120:0/3649657996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8214149dd0 msgr2=0x7f821414b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f821259c640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8214149dd0 0x7f821414b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f821259c640 1 -- 172.21.15.120:0/3649657996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8214157120 msgr2=0x7f821414b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f821259c640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8214157120 0x7f821414b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f821259c640 1 -- 172.21.15.120:0/3649657996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8214157eb0 con 0x7f8200003f30 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f8212d9d640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8214149dd0 0x7f821414b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f821259c640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 0x7f8214149890 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f81fc011a70 tx=0x7f81fc011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f820affd640 1 -- 172.21.15.120:0/3649657996 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00fb50 con 0x7f8200003f30 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f820affd640 1 -- 172.21.15.120:0/3649657996 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81fc00fcf0 con 0x7f8200003f30 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f820affd640 1 -- 172.21.15.120:0/3649657996 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00fea0 con 0x7f8200003f30 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8214158190 con 0x7f8200003f30 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.863+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8214158fe0 con 0x7f8200003f30 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.864+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82140b2e60 con 0x7f8200003f30 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.864+0000 7f820affd640 1 -- 172.21.15.120:0/3649657996 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81fc012040 con 0x7f8200003f30 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.865+0000 7f820affd640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f81f406be60 0x7f81f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.865+0000 7f820affd640 1 -- 172.21.15.120:0/3649657996 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f81fc08b9d0 con 0x7f8200003f30 2024-09-10T10:29:12.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.865+0000 7f8211d9b640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f81f406be60 0x7f81f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.865+0000 7f820affd640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f81f4075c80 0x7f81f4078120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.865+0000 7f820affd640 1 -- 172.21.15.120:0/3649657996 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81f40787f0 con 0x7f81f4075c80 2024-09-10T10:29:12.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.865+0000 7f820affd640 1 -- 172.21.15.120:0/3649657996 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f81fc0902c0 con 0x7f8200003f30 2024-09-10T10:29:12.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.865+0000 7f8212d9d640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f81f4075c80 0x7f81f4078120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.865+0000 7f8211d9b640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f81f406be60 0x7f81f406e320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8204033000 tx=0x7f8204008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:12.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.865+0000 7f8212d9d640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f81f4075c80 0x7f81f4078120 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:12.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.866+0000 7f820affd640 1 -- 172.21.15.120:0/3649657996 <== osd.0 v2:172.21.15.120:6808/3664783490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81f40787f0 con 0x7f81f4075c80 2024-09-10T10:29:12.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.878+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 --> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f82140cc5b0 con 0x7f81f4075c80 2024-09-10T10:29:12.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.878+0000 7f820affd640 1 -- 172.21.15.120:0/3649657996 <== osd.0 v2:172.21.15.120:6808/3664783490 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f82140cc5b0 con 0x7f81f4075c80 2024-09-10T10:29:12.880 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:29:12.880 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:29:12.880 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:29:12.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f81f4075c80 msgr2=0x7f81f4078120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8218f03640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:6808/3664783490,v1:172.21.15.120:6809/3664783490] conn(0x7f81f4075c80 0x7f81f4078120 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f81f406be60 msgr2=0x7f81f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8218f03640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f81f406be60 0x7f81f406e320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8204033000 tx=0x7f8204008030 comp rx=0 tx=0).stop 2024-09-10T10:29:12.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 msgr2=0x7f8214149890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8218f03640 1 --2- 172.21.15.120:0/3649657996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8200003f30 0x7f8214149890 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f81fc011a70 tx=0x7f81fc011f30 comp rx=0 tx=0).stop 2024-09-10T10:29:12.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8212d9d640 1 -- 172.21.15.120:0/3649657996 reap_dead start 2024-09-10T10:29:12.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 shutdown_connections 2024-09-10T10:29:12.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 >> 172.21.15.120:0/3649657996 conn(0x7f8214067950 msgr2=0x7f82141588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:12.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 shutdown_connections 2024-09-10T10:29:12.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.879+0000 7f8218f03640 1 -- 172.21.15.120:0/3649657996 wait complete. 2024-09-10T10:29:12.890 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T10:29:12.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.989+0000 7ff589e2e640 1 Processor -- start 2024-09-10T10:29:12.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.989+0000 7ff589e2e640 1 -- start start 2024-09-10T10:29:12.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff589e2e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff584154d00 0x7ff5840c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff589e2e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff58415e040 0x7ff5840c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff589e2e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff58406c950 con 0x7ff584071010 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff589e2e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff5840707e0 con 0x7ff58415e040 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff589e2e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff5840c90f0 con 0x7ff584154d00 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff582ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff58415e040 0x7ff5840c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff582ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff58415e040 0x7ff5840c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42540/0 (socket says 172.21.15.120:42540) 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff582ffd640 1 -- 172.21.15.120:0/2819877434 learned_addr learned my addr 172.21.15.120:0/2819877434 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff5837fe640 1 --2- 172.21.15.120:0/2819877434 >> v2:172.21.15.134:3300/0 conn(0x7ff584154d00 0x7ff5840c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff582ffd640 1 -- 172.21.15.120:0/2819877434 >> v2:172.21.15.134:3300/0 conn(0x7ff584154d00 msgr2=0x7ff5840c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff582ffd640 1 --2- 172.21.15.120:0/2819877434 >> v2:172.21.15.134:3300/0 conn(0x7ff584154d00 0x7ff5840c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff582ffd640 1 -- 172.21.15.120:0/2819877434 >> v1:172.21.15.134:6789/0 conn(0x7ff584071010 legacy=0x7ff5840cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.990+0000 7ff582ffd640 1 -- 172.21.15.120:0/2819877434 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff584070a60 con 0x7ff58415e040 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.991+0000 7ff582ffd640 1 --2- 172.21.15.120:0/2819877434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 0x7ff5840c8bb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff5840cdf50 tx=0x7ff57802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d4a731170720a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.991+0000 7ff5827fc640 1 -- 172.21.15.120:0/2819877434 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578037070 con 0x7ff58415e040 2024-09-10T10:29:12.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.991+0000 7ff5827fc640 1 -- 172.21.15.120:0/2819877434 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff578032440 con 0x7ff58415e040 2024-09-10T10:29:12.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.991+0000 7ff5827fc640 1 -- 172.21.15.120:0/2819877434 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578032810 con 0x7ff58415e040 2024-09-10T10:29:12.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.992+0000 7ff589e2e640 1 -- 172.21.15.120:0/2819877434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 msgr2=0x7ff5840c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.992+0000 7ff589e2e640 1 --2- 172.21.15.120:0/2819877434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 0x7ff5840c8bb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff5840cdf50 tx=0x7ff57802a270 comp rx=0 tx=0).stop 2024-09-10T10:29:12.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.992+0000 7ff589e2e640 1 -- 172.21.15.120:0/2819877434 shutdown_connections 2024-09-10T10:29:12.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.992+0000 7ff589e2e640 1 --2- 172.21.15.120:0/2819877434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 0x7ff5840c8bb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.992+0000 7ff589e2e640 1 --2- 172.21.15.120:0/2819877434 >> v2:172.21.15.134:3300/0 conn(0x7ff584154d00 0x7ff5840c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.992+0000 7ff589e2e640 1 -- 172.21.15.120:0/2819877434 >> 172.21.15.120:0/2819877434 conn(0x7ff584067950 msgr2=0x7ff584067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.992+0000 7ff589e2e640 1 -- 172.21.15.120:0/2819877434 shutdown_connections 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.992+0000 7ff589e2e640 1 -- 172.21.15.120:0/2819877434 wait complete. 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.993+0000 7ff589e2e640 1 Processor -- start 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.993+0000 7ff589e2e640 1 -- start start 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff589e2e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff584071010 0x7ff5841f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff589e2e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff584154d00 0x7ff58415c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff589e2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 0x7ff584153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff589e2e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff584070d40 con 0x7ff58415e040 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff589e2e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff58406c670 con 0x7ff584071010 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff589e2e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff5840c75a0 con 0x7ff584154d00 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff583fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 0x7ff584153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff583fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 0x7ff584153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42542/0 (socket says 172.21.15.120:42542) 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff583fff640 1 -- 172.21.15.120:0/3820021852 learned_addr learned my addr 172.21.15.120:0/3820021852 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff5837fe640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff584071010 0x7ff5841f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff582ffd640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff584154d00 0x7ff58415c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff583fff640 1 -- 172.21.15.120:0/3820021852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff584154d00 msgr2=0x7ff58415c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff583fff640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff584154d00 0x7ff58415c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff583fff640 1 -- 172.21.15.120:0/3820021852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff584071010 msgr2=0x7ff5841f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff583fff640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff584071010 0x7ff5841f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff583fff640 1 -- 172.21.15.120:0/3820021852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff584155b60 con 0x7ff58415e040 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff5837fe640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff584071010 0x7ff5841f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.994+0000 7ff583fff640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 0x7ff584153420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c7d0 tx=0x7ff57c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.995+0000 7ff56bfff640 1 -- 172.21.15.120:0/3820021852 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c003b70 con 0x7ff58415e040 2024-09-10T10:29:12.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.995+0000 7ff56bfff640 1 -- 172.21.15.120:0/3820021852 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c003d30 con 0x7ff58415e040 2024-09-10T10:29:12.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.995+0000 7ff589e2e640 1 -- 172.21.15.120:0/3820021852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff584155e40 con 0x7ff58415e040 2024-09-10T10:29:12.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.995+0000 7ff56bfff640 1 -- 172.21.15.120:0/3820021852 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00aa90 con 0x7ff58415e040 2024-09-10T10:29:12.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.995+0000 7ff589e2e640 1 -- 172.21.15.120:0/3820021852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff584156400 con 0x7ff58415e040 2024-09-10T10:29:12.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.996+0000 7ff589e2e640 1 -- 172.21.15.120:0/3820021852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5841550e0 con 0x7ff58415e040 2024-09-10T10:29:13.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.996+0000 7ff56bfff640 1 -- 172.21.15.120:0/3820021852 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff57c00ac30 con 0x7ff58415e040 2024-09-10T10:29:13.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.997+0000 7ff56bfff640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff57006be60 0x7ff57006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.997+0000 7ff5837fe640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff57006be60 0x7ff57006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.997+0000 7ff56bfff640 1 -- 172.21.15.120:0/3820021852 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7ff57c08b290 con 0x7ff58415e040 2024-09-10T10:29:13.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:12.997+0000 7ff5837fe640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff57006be60 0x7ff57006e320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff574006fd0 tx=0x7ff574008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.001+0000 7ff56bfff640 1 -- 172.21.15.120:0/3820021852 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff57c0596b0 con 0x7ff58415e040 2024-09-10T10:29:13.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.167+0000 7ff589e2e640 1 -- 172.21.15.120:0/3820021852 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7ff5840cdf70 con 0x7ff57006be60 2024-09-10T10:29:13.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.167+0000 7ff56bfff640 1 -- 172.21.15.120:0/3820021852 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff5840cdf70 con 0x7ff57006be60 2024-09-10T10:29:13.169 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T10:29:13.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.169+0000 7ff589e2e640 1 -- 172.21.15.120:0/3820021852 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff57006be60 msgr2=0x7ff57006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.169+0000 7ff589e2e640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff57006be60 0x7ff57006e320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff574006fd0 tx=0x7ff574008040 comp rx=0 tx=0).stop 2024-09-10T10:29:13.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.169+0000 7ff589e2e640 1 -- 172.21.15.120:0/3820021852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 msgr2=0x7ff584153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.169+0000 7ff589e2e640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 0x7ff584153420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c7d0 tx=0x7ff57c00cc90 comp rx=0 tx=0).stop 2024-09-10T10:29:13.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.169+0000 7ff589e2e640 1 -- 172.21.15.120:0/3820021852 shutdown_connections 2024-09-10T10:29:13.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.169+0000 7ff589e2e640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff57006be60 0x7ff57006e320 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.169+0000 7ff589e2e640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58415e040 0x7ff584153420 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.169+0000 7ff589e2e640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff584154d00 0x7ff58415c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.169+0000 7ff589e2e640 1 --2- 172.21.15.120:0/3820021852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff584071010 0x7ff5841f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.169+0000 7ff589e2e640 1 -- 172.21.15.120:0/3820021852 >> 172.21.15.120:0/3820021852 conn(0x7ff584067950 msgr2=0x7ff5840cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:13.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.170+0000 7ff589e2e640 1 -- 172.21.15.120:0/3820021852 shutdown_connections 2024-09-10T10:29:13.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.170+0000 7ff589e2e640 1 -- 172.21.15.120:0/3820021852 wait complete. 2024-09-10T10:29:13.182 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T10:29:13.182 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:29:13.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.282+0000 7f13e201e640 1 Processor -- start 2024-09-10T10:29:13.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.282+0000 7f13e201e640 1 -- start start 2024-09-10T10:29:13.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13e201e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f13dc06ee80 0x7f13dc06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13e201e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13dc15ac40 con 0x7f13dc0c46c0 2024-09-10T10:29:13.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13e201e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13dc15af20 con 0x7f13dc151490 2024-09-10T10:29:13.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13e201e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f13dc15b200 con 0x7f13dc06ee80 2024-09-10T10:29:13.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13daffd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f13dc0c46c0 0x7f13dc0c6270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41588/0 (socket says 172.21.15.120:41588) 2024-09-10T10:29:13.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13daffd640 1 -- 172.21.15.120:0/1354860596 learned_addr learned my addr 172.21.15.120:0/1354860596 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:13.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2255349118 0 0) 0x7f13dc15ac40 con 0x7f13dc0c46c0 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13c4003520 con 0x7f13dc0c46c0 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13dbfff640 1 --2- 172.21.15.120:0/1354860596 >> v2:172.21.15.134:3300/0 conn(0x7f13dc06ee80 0x7f13dc06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4103150690 0 0) 0x7f13c4003520 con 0x7f13dc0c46c0 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 >> v2:172.21.15.134:3300/0 conn(0x7f13dc06ee80 msgr2=0x7f13dc06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 --2- 172.21.15.120:0/1354860596 >> v2:172.21.15.134:3300/0 conn(0x7f13dc06ee80 0x7f13dc06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 >> v1:172.21.15.134:6789/0 conn(0x7f13dc151490 legacy=0x7f13dc157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc15b4e0 con 0x7f13dc0c46c0 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f13cc001670 con 0x7f13dc0c46c0 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 >> v1:172.21.15.120:6789/0 conn(0x7f13dc0c46c0 legacy=0x7f13dc0c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c40039b0 0x7f13c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13c4004800 0x7f13c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13c40051c0 0x7f13c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc15b200 con 0x7f13c4004800 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13c400c060 con 0x7f13c40039b0 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13c400c260 con 0x7f13c40051c0 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.283+0000 7f13daffd640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13c4004800 0x7f13c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.284+0000 7f13db7fe640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c40039b0 0x7f13c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.284+0000 7f13dbfff640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13c40051c0 0x7f13c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.284+0000 7f13daffd640 1 -- 172.21.15.120:0/1354860596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13c40051c0 msgr2=0x7f13c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.284+0000 7f13daffd640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13c40051c0 0x7f13c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.284+0000 7f13daffd640 1 -- 172.21.15.120:0/1354860596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c40039b0 msgr2=0x7f13c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.284+0000 7f13daffd640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c40039b0 0x7f13c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.284+0000 7f13daffd640 1 -- 172.21.15.120:0/1354860596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c4003520 con 0x7f13c4004800 2024-09-10T10:29:13.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.284+0000 7f13dbfff640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13c40051c0 0x7f13c400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.284+0000 7f13dbfff640 1 -- 172.21.15.120:0/1354860596 reap_dead start 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.284+0000 7f13daffd640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13c4004800 0x7f13c4004c80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00c240 tx=0x7f13c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454688709e364fad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.285+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc007cf0 con 0x7f13c4004800 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.285+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc005ad0 con 0x7f13c4004800 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.285+0000 7f13da7fc640 1 -- 172.21.15.120:0/1354860596 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc007070 con 0x7f13c4004800 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.285+0000 7f13e201e640 1 -- 172.21.15.120:0/1354860596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13c4004800 msgr2=0x7f13c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.285+0000 7f13e201e640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13c4004800 0x7f13c4004c80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00c240 tx=0x7f13c4005830 comp rx=0 tx=0).stop 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.285+0000 7f13e201e640 1 -- 172.21.15.120:0/1354860596 shutdown_connections 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.285+0000 7f13e201e640 1 --2- 172.21.15.120:0/1354860596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13c4004800 0x7f13c4004c80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.285+0000 7f13e201e640 1 -- 172.21.15.120:0/1354860596 >> 172.21.15.120:0/1354860596 conn(0x7f13dc068170 msgr2=0x7f13dc070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.285+0000 7f13e201e640 1 -- 172.21.15.120:0/1354860596 shutdown_connections 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.285+0000 7f13e201e640 1 -- 172.21.15.120:0/1354860596 wait complete. 2024-09-10T10:29:13.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.286+0000 7f13e201e640 1 Processor -- start 2024-09-10T10:29:13.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13e201e640 1 -- start start 2024-09-10T10:29:13.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13e201e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c4004800 0x7f13dc15a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13e201e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc159ce0 0x7f13dc0b8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13e201e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13dc14ad50 0x7f13dc1f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13e201e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1519f0 con 0x7f13dc159ce0 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13e201e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc151870 con 0x7f13c4004800 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13e201e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13dc0c7340 con 0x7f13dc14ad50 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13daffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc159ce0 0x7f13dc0b8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13daffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc159ce0 0x7f13dc0b8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42562/0 (socket says 172.21.15.120:42562) 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13daffd640 1 -- 172.21.15.120:0/4038767115 learned_addr learned my addr 172.21.15.120:0/4038767115 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.287+0000 7f13db7fe640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c4004800 0x7f13dc15a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13dbfff640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13dc14ad50 0x7f13dc1f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13daffd640 1 -- 172.21.15.120:0/4038767115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13dc14ad50 msgr2=0x7f13dc1f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13daffd640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13dc14ad50 0x7f13dc1f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13daffd640 1 -- 172.21.15.120:0/4038767115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c4004800 msgr2=0x7f13dc15a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13daffd640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c4004800 0x7f13dc15a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13daffd640 1 -- 172.21.15.120:0/4038767115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc0cd670 con 0x7f13dc159ce0 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13daffd640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc159ce0 0x7f13dc0b8fd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00c360 tx=0x7f13dc15b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13c3fff640 1 -- 172.21.15.120:0/4038767115 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc037040 con 0x7f13dc159ce0 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13c3fff640 1 -- 172.21.15.120:0/4038767115 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc001470 con 0x7f13dc159ce0 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13dc0cd8f0 con 0x7f13dc159ce0 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13c3fff640 1 -- 172.21.15.120:0/4038767115 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0047e0 con 0x7f13dc159ce0 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.288+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13dc1f0de0 con 0x7f13dc159ce0 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.289+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13dc1f11e0 con 0x7f13dc159ce0 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.289+0000 7f13c3fff640 1 -- 172.21.15.120:0/4038767115 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13cc043030 con 0x7f13dc159ce0 2024-09-10T10:29:13.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.290+0000 7f13c3fff640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f13c806be30 0x7f13c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.290+0000 7f13c3fff640 1 -- 172.21.15.120:0/4038767115 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f13cc0aa400 con 0x7f13dc159ce0 2024-09-10T10:29:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.290+0000 7f13db7fe640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f13c806be30 0x7f13c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.290+0000 7f13c3fff640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f13c8075c50 0x7f13c80780f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.290+0000 7f13c3fff640 1 -- 172.21.15.120:0/4038767115 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13cc034e50 con 0x7f13c8075c50 2024-09-10T10:29:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.290+0000 7f13c3fff640 1 -- 172.21.15.120:0/4038767115 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f13cc0b1050 con 0x7f13dc159ce0 2024-09-10T10:29:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.290+0000 7f13dbfff640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f13c8075c50 0x7f13c80780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.290+0000 7f13db7fe640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f13c806be30 0x7f13c806e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f13d0005590 tx=0x7f13d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.290+0000 7f13dbfff640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f13c8075c50 0x7f13c80780f0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.291+0000 7f13c3fff640 1 -- 172.21.15.120:0/4038767115 <== osd.1 v2:172.21.15.120:6800/3972761964 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13cc034e50 con 0x7f13c8075c50 2024-09-10T10:29:13.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.303+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 --> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f13dc0c4f10 con 0x7f13c8075c50 2024-09-10T10:29:13.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.303+0000 7f13c3fff640 1 -- 172.21.15.120:0/4038767115 <== osd.1 v2:172.21.15.120:6800/3972761964 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f13dc0c4f10 con 0x7f13c8075c50 2024-09-10T10:29:13.305 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:29:13.305 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:29:13.305 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f13c8075c50 msgr2=0x7f13c80780f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13e201e640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:6800/3972761964,v1:172.21.15.120:6801/3972761964] conn(0x7f13c8075c50 0x7f13c80780f0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f13c806be30 msgr2=0x7f13c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13e201e640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f13c806be30 0x7f13c806e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f13d0005590 tx=0x7f13d0009040 comp rx=0 tx=0).stop 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc159ce0 msgr2=0x7f13dc0b8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13e201e640 1 --2- 172.21.15.120:0/4038767115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13dc159ce0 0x7f13dc0b8fd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00c360 tx=0x7f13dc15b7c0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13dbfff640 1 -- 172.21.15.120:0/4038767115 reap_dead start 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 shutdown_connections 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 >> 172.21.15.120:0/4038767115 conn(0x7f13dc068170 msgr2=0x7f13dc14d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 shutdown_connections 2024-09-10T10:29:13.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.304+0000 7f13e201e640 1 -- 172.21.15.120:0/4038767115 wait complete. 2024-09-10T10:29:13.315 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T10:29:13.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.414+0000 7f53bd41e640 1 Processor -- start 2024-09-10T10:29:13.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.414+0000 7f53bd41e640 1 -- start start 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53bd41e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f53b80cc0c0 0x7f53b815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53bd41e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53b815b1d0 con 0x7f53b80cc4a0 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53bd41e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53b815b4b0 con 0x7f53b80cb2f0 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53bd41e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f53b80cbd00 con 0x7f53b80cc0c0 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53b6ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f53b80cc0c0 0x7f53b815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53b6ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f53b80cc0c0 0x7f53b815aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42568/0 (socket says 172.21.15.120:42568) 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53b77fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f53b80cc4a0 0x7f53b8165430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41598/0 (socket says 172.21.15.120:41598) 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53b6ffd640 1 -- 172.21.15.120:0/3500004663 learned_addr learned my addr 172.21.15.120:0/3500004663 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53b6ffd640 1 -- 172.21.15.120:0/3500004663 >> v1:172.21.15.134:6789/0 conn(0x7f53b80cb2f0 legacy=0x7f53b815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53b6ffd640 1 -- 172.21.15.120:0/3500004663 >> v1:172.21.15.120:6789/0 conn(0x7f53b80cc4a0 legacy=0x7f53b8165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53b67fc640 1 -- 172.21.15.120:0/3500004663 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550560426 0 0) 0x7f53b815b1d0 con 0x7f53b80cc4a0 2024-09-10T10:29:13.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.415+0000 7f53b6ffd640 1 -- 172.21.15.120:0/3500004663 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b815b790 con 0x7f53b80cc0c0 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53b6ffd640 1 --2- 172.21.15.120:0/3500004663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 0x7f53b815aab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009870 tx=0x7f53a802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962d6e8ba8b5fe0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53b67fc640 1 -- 172.21.15.120:0/3500004663 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a8037070 con 0x7f53b80cc0c0 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53b67fc640 1 -- 172.21.15.120:0/3500004663 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53a802ae50 con 0x7f53b80cc0c0 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53b67fc640 1 -- 172.21.15.120:0/3500004663 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a8032860 con 0x7f53b80cc0c0 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53bd41e640 1 -- 172.21.15.120:0/3500004663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 msgr2=0x7f53b815aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53bd41e640 1 --2- 172.21.15.120:0/3500004663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 0x7f53b815aab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009870 tx=0x7f53a802a300 comp rx=0 tx=0).stop 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53bd41e640 1 -- 172.21.15.120:0/3500004663 shutdown_connections 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53bd41e640 1 --2- 172.21.15.120:0/3500004663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 0x7f53b815aab0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53bd41e640 1 -- 172.21.15.120:0/3500004663 >> 172.21.15.120:0/3500004663 conn(0x7f53b806d560 msgr2=0x7f53b806d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53bd41e640 1 -- 172.21.15.120:0/3500004663 shutdown_connections 2024-09-10T10:29:13.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.416+0000 7f53bd41e640 1 -- 172.21.15.120:0/3500004663 wait complete. 2024-09-10T10:29:13.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.417+0000 7f53bd41e640 1 Processor -- start 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.418+0000 7f53bd41e640 1 -- start start 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.418+0000 7f53bd41e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80cb2f0 0x7f53b814f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.418+0000 7f53bd41e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 0x7f53b81f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.418+0000 7f53bd41e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b80cc4a0 0x7f53b80b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.418+0000 7f53bd41e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53b815e540 con 0x7f53b80cc0c0 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.418+0000 7f53bd41e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53b815ba70 con 0x7f53b80cb2f0 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.418+0000 7f53bd41e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53b80cc8b0 con 0x7f53b80cc4a0 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.418+0000 7f53ae7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 0x7f53b81f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.418+0000 7f53ae7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 0x7f53b81f14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42584/0 (socket says 172.21.15.120:42584) 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53ae7fc640 1 -- 172.21.15.120:0/1580878377 learned_addr learned my addr 172.21.15.120:0/1580878377 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53b6ffd640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80cb2f0 0x7f53b814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53b77fe640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b80cc4a0 0x7f53b80b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53ae7fc640 1 -- 172.21.15.120:0/1580878377 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b80cc4a0 msgr2=0x7f53b80b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53ae7fc640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b80cc4a0 0x7f53b80b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53ae7fc640 1 -- 172.21.15.120:0/1580878377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80cb2f0 msgr2=0x7f53b814f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53ae7fc640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80cb2f0 0x7f53b814f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53ae7fc640 1 -- 172.21.15.120:0/1580878377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b814ed10 con 0x7f53b80cc0c0 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53b6ffd640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80cb2f0 0x7f53b814f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53ae7fc640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 0x7f53b81f14d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c400 tx=0x7f53a000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53affff640 1 -- 172.21.15.120:0/1580878377 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0017070 con 0x7f53b80cc0c0 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53affff640 1 -- 172.21.15.120:0/1580878377 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53a00052d0 con 0x7f53b80cc0c0 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53affff640 1 -- 172.21.15.120:0/1580878377 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0003830 con 0x7f53b80cc0c0 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53bd41e640 1 -- 172.21.15.120:0/1580878377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b8150f30 con 0x7f53b80cc0c0 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.419+0000 7f53bd41e640 1 -- 172.21.15.120:0/1580878377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b81f0cd0 con 0x7f53b80cc0c0 2024-09-10T10:29:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.420+0000 7f53bd41e640 1 -- 172.21.15.120:0/1580878377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b80c5560 con 0x7f53b80cc0c0 2024-09-10T10:29:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.421+0000 7f53affff640 1 -- 172.21.15.120:0/1580878377 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53a001a070 con 0x7f53b80cc0c0 2024-09-10T10:29:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.421+0000 7f53affff640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f539806be30 0x7f539806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.421+0000 7f53affff640 1 -- 172.21.15.120:0/1580878377 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f53a008a660 con 0x7f53b80cc0c0 2024-09-10T10:29:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.421+0000 7f53b6ffd640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f539806be30 0x7f539806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.422+0000 7f53b6ffd640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f539806be30 0x7f539806e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f53a8034000 tx=0x7f53a802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.425+0000 7f53affff640 1 -- 172.21.15.120:0/1580878377 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f53a0058b10 con 0x7f53b80cc0c0 2024-09-10T10:29:13.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.589+0000 7f53bd41e640 1 -- 172.21.15.120:0/1580878377 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f53b815af10 con 0x7f539806be30 2024-09-10T10:29:13.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.589+0000 7f53affff640 1 -- 172.21.15.120:0/1580878377 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f53b815af10 con 0x7f539806be30 2024-09-10T10:29:13.591 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T10:29:13.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.591+0000 7f53bd41e640 1 -- 172.21.15.120:0/1580878377 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f539806be30 msgr2=0x7f539806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.591+0000 7f53bd41e640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f539806be30 0x7f539806e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f53a8034000 tx=0x7f53a802e000 comp rx=0 tx=0).stop 2024-09-10T10:29:13.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.591+0000 7f53bd41e640 1 -- 172.21.15.120:0/1580878377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 msgr2=0x7f53b81f14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.591+0000 7f53bd41e640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 0x7f53b81f14d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c400 tx=0x7f53a000c8c0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.591+0000 7f53bd41e640 1 -- 172.21.15.120:0/1580878377 shutdown_connections 2024-09-10T10:29:13.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.591+0000 7f53bd41e640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f539806be30 0x7f539806e2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.591+0000 7f53bd41e640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b80cc4a0 0x7f53b80b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.591+0000 7f53bd41e640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53b80cc0c0 0x7f53b81f14d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.591+0000 7f53bd41e640 1 --2- 172.21.15.120:0/1580878377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80cb2f0 0x7f53b814f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.591+0000 7f53bd41e640 1 -- 172.21.15.120:0/1580878377 >> 172.21.15.120:0/1580878377 conn(0x7f53b806d560 msgr2=0x7f53b8152050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:13.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.592+0000 7f53bd41e640 1 -- 172.21.15.120:0/1580878377 shutdown_connections 2024-09-10T10:29:13.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.592+0000 7f53bd41e640 1 -- 172.21.15.120:0/1580878377 wait complete. 2024-09-10T10:29:13.604 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T10:29:13.604 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:29:13.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.704+0000 7f83cc3af640 1 Processor -- start 2024-09-10T10:29:13.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.704+0000 7f83cc3af640 1 -- start start 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83cc3af640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f83c4151900 0x7f83c40c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83cc3af640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c415b480 con 0x7f83c4072ea0 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83cc3af640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c4071b40 con 0x7f83c4151ce0 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83cc3af640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f83c406f250 con 0x7f83c4151900 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83ca14b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f83c4151900 0x7f83c40c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83ca14b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f83c4151900 0x7f83c40c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42596/0 (socket says 172.21.15.120:42596) 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83c994a640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f83c4072ea0 0x7f83c415ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41600/0 (socket says 172.21.15.120:41600) 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83ca14b640 1 -- 172.21.15.120:0/2938675971 learned_addr learned my addr 172.21.15.120:0/2938675971 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83ca14b640 1 -- 172.21.15.120:0/2938675971 >> v1:172.21.15.134:6789/0 conn(0x7f83c4151ce0 legacy=0x7f83c40c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83ca14b640 1 -- 172.21.15.120:0/2938675971 >> v1:172.21.15.120:6789/0 conn(0x7f83c4072ea0 legacy=0x7f83c415ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83ca14b640 1 -- 172.21.15.120:0/2938675971 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c4071e20 con 0x7f83c4151900 2024-09-10T10:29:13.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.705+0000 7f83c9149640 1 -- 172.21.15.120:0/2938675971 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916987917 0 0) 0x7f83c415b480 con 0x7f83c4072ea0 2024-09-10T10:29:13.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.706+0000 7f83ca14b640 1 --2- 172.21.15.120:0/2938675971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83c4151900 0x7f83c40c35b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009e50 tx=0x7f83b402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2204aba8395cd50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.706+0000 7f83c9149640 1 -- 172.21.15.120:0/2938675971 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4036070 con 0x7f83c4151900 2024-09-10T10:29:13.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.706+0000 7f83c9149640 1 -- 172.21.15.120:0/2938675971 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b4031600 con 0x7f83c4151900 2024-09-10T10:29:13.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.706+0000 7f83c9149640 1 -- 172.21.15.120:0/2938675971 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40319d0 con 0x7f83c4151900 2024-09-10T10:29:13.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.707+0000 7f83cc3af640 1 -- 172.21.15.120:0/2938675971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83c4151900 msgr2=0x7f83c40c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.707+0000 7f83cc3af640 1 --2- 172.21.15.120:0/2938675971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83c4151900 0x7f83c40c35b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009e50 tx=0x7f83b402a6d0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.707+0000 7f83cc3af640 1 -- 172.21.15.120:0/2938675971 shutdown_connections 2024-09-10T10:29:13.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.707+0000 7f83cc3af640 1 --2- 172.21.15.120:0/2938675971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83c4151900 0x7f83c40c35b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.707+0000 7f83cc3af640 1 -- 172.21.15.120:0/2938675971 >> 172.21.15.120:0/2938675971 conn(0x7f83c40cc0b0 msgr2=0x7f83c40cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:13.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.707+0000 7f83cc3af640 1 -- 172.21.15.120:0/2938675971 shutdown_connections 2024-09-10T10:29:13.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.707+0000 7f83cc3af640 1 -- 172.21.15.120:0/2938675971 wait complete. 2024-09-10T10:29:13.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.708+0000 7f83cc3af640 1 Processor -- start 2024-09-10T10:29:13.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.708+0000 7f83cc3af640 1 -- start start 2024-09-10T10:29:13.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83cc3af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c4072ea0 0x7f83c41ef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83cc3af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c4151900 0x7f83c406f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83cc3af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83c4151ce0 0x7f83c406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83cc3af640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40c3bd0 con 0x7f83c4151ce0 2024-09-10T10:29:13.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83cc3af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83c4073860 con 0x7f83c4151900 2024-09-10T10:29:13.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83cc3af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f83c4073cf0 con 0x7f83c4072ea0 2024-09-10T10:29:13.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83ca94c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83c4151ce0 0x7f83c406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83ca94c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83c4151ce0 0x7f83c406fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42608/0 (socket says 172.21.15.120:42608) 2024-09-10T10:29:13.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83ca94c640 1 -- 172.21.15.120:0/3710176047 learned_addr learned my addr 172.21.15.120:0/3710176047 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:13.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83ca14b640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c4072ea0 0x7f83c41ef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83c994a640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c4151900 0x7f83c406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83ca94c640 1 -- 172.21.15.120:0/3710176047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c4072ea0 msgr2=0x7f83c41ef0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83ca94c640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c4072ea0 0x7f83c41ef0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83ca94c640 1 -- 172.21.15.120:0/3710176047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c4151900 msgr2=0x7f83c406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83ca94c640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c4151900 0x7f83c406f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83ca94c640 1 -- 172.21.15.120:0/3710176047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c40cb1f0 con 0x7f83c4151ce0 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83ca94c640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83c4151ce0 0x7f83c406fbc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00b410 tx=0x7f83bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83c2ffd640 1 -- 172.21.15.120:0/3710176047 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc015030 con 0x7f83c4151ce0 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83c2ffd640 1 -- 172.21.15.120:0/3710176047 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83bc004020 con 0x7f83c4151ce0 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c41583e0 con 0x7f83c4151ce0 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83c2ffd640 1 -- 172.21.15.120:0/3710176047 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc004450 con 0x7f83c4151ce0 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.709+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c41e32d0 con 0x7f83c4151ce0 2024-09-10T10:29:13.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.710+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83c41e2da0 con 0x7f83c4151ce0 2024-09-10T10:29:13.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.711+0000 7f83c2ffd640 1 -- 172.21.15.120:0/3710176047 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83bc018070 con 0x7f83c4151ce0 2024-09-10T10:29:13.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.711+0000 7f83c2ffd640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83b006be30 0x7f83b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.711+0000 7f83ca14b640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83b006be30 0x7f83b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.711+0000 7f83c2ffd640 1 -- 172.21.15.120:0/3710176047 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f83bc08ae30 con 0x7f83c4151ce0 2024-09-10T10:29:13.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.712+0000 7f83c2ffd640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f83b0075bd0 0x7f83b0078070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.712+0000 7f83c2ffd640 1 -- 172.21.15.120:0/3710176047 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83bc006770 con 0x7f83b0075bd0 2024-09-10T10:29:13.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.712+0000 7f83c994a640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f83b0075bd0 0x7f83b0078070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.712+0000 7f83ca14b640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83b006be30 0x7f83b006e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f83b4033000 tx=0x7f83b4009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.712+0000 7f83c994a640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f83b0075bd0 0x7f83b0078070 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.713+0000 7f83c2ffd640 1 -- 172.21.15.120:0/3710176047 <== osd.2 v2:172.21.15.120:6816/2916410859 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83bc006770 con 0x7f83b0075bd0 2024-09-10T10:29:13.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.715+0000 7f83c2ffd640 1 -- 172.21.15.120:0/3710176047 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f83bc054e80 con 0x7f83c4151ce0 2024-09-10T10:29:13.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.725+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 --> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f83c406e950 con 0x7f83b0075bd0 2024-09-10T10:29:13.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.725+0000 7f83c2ffd640 1 -- 172.21.15.120:0/3710176047 <== osd.2 v2:172.21.15.120:6816/2916410859 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f83c406e950 con 0x7f83b0075bd0 2024-09-10T10:29:13.726 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:29:13.726 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:29:13.727 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:29:13.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.725+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f83b0075bd0 msgr2=0x7f83b0078070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.725+0000 7f83cc3af640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:6816/2916410859,v1:172.21.15.120:6817/2916410859] conn(0x7f83b0075bd0 0x7f83b0078070 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.725+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83b006be30 msgr2=0x7f83b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.726+0000 7f83cc3af640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f83b006be30 0x7f83b006e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f83b4033000 tx=0x7f83b4009000 comp rx=0 tx=0).stop 2024-09-10T10:29:13.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.726+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83c4151ce0 msgr2=0x7f83c406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.726+0000 7f83cc3af640 1 --2- 172.21.15.120:0/3710176047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83c4151ce0 0x7f83c406fbc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00b410 tx=0x7f83bc00b8d0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.726+0000 7f83ca94c640 1 -- 172.21.15.120:0/3710176047 reap_dead start 2024-09-10T10:29:13.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.726+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 shutdown_connections 2024-09-10T10:29:13.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.726+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 >> 172.21.15.120:0/3710176047 conn(0x7f83c40cc0b0 msgr2=0x7f83c415a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:13.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.726+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 shutdown_connections 2024-09-10T10:29:13.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.726+0000 7f83cc3af640 1 -- 172.21.15.120:0/3710176047 wait complete. 2024-09-10T10:29:13.737 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T10:29:13.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.838+0000 7f3a027e4640 1 Processor -- start 2024-09-10T10:29:13.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.838+0000 7f3a027e4640 1 -- start start 2024-09-10T10:29:13.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f3a027e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc154f50 0x7f39fc0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f3a027e4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39fc0c9db0 con 0x7f39fc0cb500 2024-09-10T10:29:13.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f3a027e4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39fc0ca5d0 con 0x7f39fc154b70 2024-09-10T10:29:13.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f3a027e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f39fc06d4e0 con 0x7f39fc154f50 2024-09-10T10:29:13.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f3a00d81640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f39fc0cb500 0x7f39fc15e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41612/0 (socket says 172.21.15.120:41612) 2024-09-10T10:29:13.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f3a00d81640 1 -- 172.21.15.120:0/1784409209 learned_addr learned my addr 172.21.15.120:0/1784409209 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:13.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f39fb7fe640 1 --2- 172.21.15.120:0/1784409209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc154f50 0x7f39fc0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f39faffd640 1 -- 172.21.15.120:0/1784409209 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 528128494 0 0) 0x7f39fc0c9db0 con 0x7f39fc0cb500 2024-09-10T10:29:13.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f39faffd640 1 -- 172.21.15.120:0/1784409209 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39e4003520 con 0x7f39fc0cb500 2024-09-10T10:29:13.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f39faffd640 1 -- 172.21.15.120:0/1784409209 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 353003269 0 0) 0x7f39fc0ca5d0 con 0x7f39fc154b70 2024-09-10T10:29:13.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f39faffd640 1 -- 172.21.15.120:0/1784409209 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39fc0c9db0 con 0x7f39fc154b70 2024-09-10T10:29:13.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f39fb7fe640 1 -- 172.21.15.120:0/1784409209 >> v1:172.21.15.134:6789/0 conn(0x7f39fc154b70 legacy=0x7f39fc0c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f39fb7fe640 1 -- 172.21.15.120:0/1784409209 >> v1:172.21.15.120:6789/0 conn(0x7f39fc0cb500 legacy=0x7f39fc15e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.839+0000 7f39fb7fe640 1 -- 172.21.15.120:0/1784409209 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc0ca890 con 0x7f39fc154f50 2024-09-10T10:29:13.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.840+0000 7f39fb7fe640 1 --2- 172.21.15.120:0/1784409209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc154f50 0x7f39fc0cafc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f39f00097d0 tx=0x7f39f002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb37ebb6bbcb9521 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.840+0000 7f39faffd640 1 -- 172.21.15.120:0/1784409209 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0037070 con 0x7f39fc154f50 2024-09-10T10:29:13.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.840+0000 7f39faffd640 1 -- 172.21.15.120:0/1784409209 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f0032440 con 0x7f39fc154f50 2024-09-10T10:29:13.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.840+0000 7f39faffd640 1 -- 172.21.15.120:0/1784409209 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f00327f0 con 0x7f39fc154f50 2024-09-10T10:29:13.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.841+0000 7f3a027e4640 1 -- 172.21.15.120:0/1784409209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc154f50 msgr2=0x7f39fc0cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.841+0000 7f3a027e4640 1 --2- 172.21.15.120:0/1784409209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc154f50 0x7f39fc0cafc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f39f00097d0 tx=0x7f39f002a290 comp rx=0 tx=0).stop 2024-09-10T10:29:13.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.841+0000 7f3a027e4640 1 -- 172.21.15.120:0/1784409209 shutdown_connections 2024-09-10T10:29:13.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.841+0000 7f3a027e4640 1 --2- 172.21.15.120:0/1784409209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc154f50 0x7f39fc0cafc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.841+0000 7f3a027e4640 1 -- 172.21.15.120:0/1784409209 >> 172.21.15.120:0/1784409209 conn(0x7f39fc06bc10 msgr2=0x7f39fc06fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:13.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.841+0000 7f3a027e4640 1 -- 172.21.15.120:0/1784409209 shutdown_connections 2024-09-10T10:29:13.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.841+0000 7f3a027e4640 1 -- 172.21.15.120:0/1784409209 wait complete. 2024-09-10T10:29:13.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.842+0000 7f3a027e4640 1 Processor -- start 2024-09-10T10:29:13.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.842+0000 7f3a027e4640 1 -- start start 2024-09-10T10:29:13.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a027e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc0cb500 0x7f39fc148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a027e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39fc154b70 0x7f39fc14fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a027e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc154f50 0x7f39fc1470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a027e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc06e550 con 0x7f39fc154f50 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a027e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc06d170 con 0x7f39fc154b70 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a027e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f39fc06ecc0 con 0x7f39fc0cb500 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a00d81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc154f50 0x7f39fc1470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a00d81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc154f50 0x7f39fc1470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42618/0 (socket says 172.21.15.120:42618) 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a00d81640 1 -- 172.21.15.120:0/1681519943 learned_addr learned my addr 172.21.15.120:0/1681519943 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f39fb7fe640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39fc154b70 0x7f39fc14fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f39fbfff640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc0cb500 0x7f39fc148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a00d81640 1 -- 172.21.15.120:0/1681519943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc0cb500 msgr2=0x7f39fc148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a00d81640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc0cb500 0x7f39fc148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a00d81640 1 -- 172.21.15.120:0/1681519943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39fc154b70 msgr2=0x7f39fc14fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a00d81640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39fc154b70 0x7f39fc14fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a00d81640 1 -- 172.21.15.120:0/1681519943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc152eb0 con 0x7f39fc154f50 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.843+0000 7f3a00d81640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc154f50 0x7f39fc1470c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f39f400cb20 tx=0x7f39f400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.844+0000 7f39f8ff9640 1 -- 172.21.15.120:0/1681519943 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4017070 con 0x7f39fc154f50 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.844+0000 7f39f8ff9640 1 -- 172.21.15.120:0/1681519943 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f400ac40 con 0x7f39fc154f50 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.844+0000 7f3a027e4640 1 -- 172.21.15.120:0/1681519943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc14ef40 con 0x7f39fc154f50 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.844+0000 7f39f8ff9640 1 -- 172.21.15.120:0/1681519943 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4012400 con 0x7f39fc154f50 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.844+0000 7f3a027e4640 1 -- 172.21.15.120:0/1681519943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc0cd130 con 0x7f39fc154f50 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.845+0000 7f3a027e4640 1 -- 172.21.15.120:0/1681519943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc0ca220 con 0x7f39fc154f50 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.845+0000 7f39f8ff9640 1 -- 172.21.15.120:0/1681519943 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f39f401a070 con 0x7f39fc154f50 2024-09-10T10:29:13.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.846+0000 7f39f8ff9640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39e806be30 0x7f39e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:13.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.846+0000 7f39fbfff640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39e806be30 0x7f39e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:13.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.846+0000 7f39f8ff9640 1 -- 172.21.15.120:0/1681519943 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f39f408afe0 con 0x7f39fc154f50 2024-09-10T10:29:13.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.846+0000 7f39fbfff640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39e806be30 0x7f39e806e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39ec007fd0 tx=0x7f39ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:13.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:13.850+0000 7f39f8ff9640 1 -- 172.21.15.120:0/1681519943 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f39f4058e40 con 0x7f39fc154f50 2024-09-10T10:29:14.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.018+0000 7f3a027e4640 1 -- 172.21.15.120:0/1681519943 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f39fc0cbae0 con 0x7f39e806be30 2024-09-10T10:29:14.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.018+0000 7f39f8ff9640 1 -- 172.21.15.120:0/1681519943 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f39fc0cbae0 con 0x7f39e806be30 2024-09-10T10:29:14.020 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T10:29:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.020+0000 7f3a027e4640 1 -- 172.21.15.120:0/1681519943 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39e806be30 msgr2=0x7f39e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39e806be30 0x7f39e806e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39ec007fd0 tx=0x7f39ec009040 comp rx=0 tx=0).stop 2024-09-10T10:29:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 -- 172.21.15.120:0/1681519943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc154f50 msgr2=0x7f39fc1470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc154f50 0x7f39fc1470c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f39f400cb20 tx=0x7f39f400a060 comp rx=0 tx=0).stop 2024-09-10T10:29:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 -- 172.21.15.120:0/1681519943 shutdown_connections 2024-09-10T10:29:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f39e806be30 0x7f39e806e2f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc154f50 0x7f39fc1470c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39fc154b70 0x7f39fc14fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 --2- 172.21.15.120:0/1681519943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39fc0cb500 0x7f39fc148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 -- 172.21.15.120:0/1681519943 >> 172.21.15.120:0/1681519943 conn(0x7f39fc06bc10 msgr2=0x7f39fc0ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 -- 172.21.15.120:0/1681519943 shutdown_connections 2024-09-10T10:29:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.021+0000 7f3a027e4640 1 -- 172.21.15.120:0/1681519943 wait complete. 2024-09-10T10:29:14.033 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T10:29:14.033 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:29:14.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.134+0000 7f00e56ba640 1 Processor -- start 2024-09-10T10:29:14.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.134+0000 7f00e56ba640 1 -- start start 2024-09-10T10:29:14.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.135+0000 7f00e56ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 0x7f00e006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.135+0000 7f00e56ba640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f00e00c9c10 0x7f00e00ca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.135+0000 7f00e56ba640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e00c7d10 con 0x7f00e006bc10 2024-09-10T10:29:14.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.135+0000 7f00e56ba640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f00e006c590 con 0x7f00e00c9c10 2024-09-10T10:29:14.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.135+0000 7f00e56ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f00e006c870 con 0x7f00e0154a30 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.135+0000 7f00de7fc640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f00e006bc10 0x7f00e00c6de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41614/0 (socket says 172.21.15.120:41614) 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.135+0000 7f00de7fc640 1 -- 172.21.15.120:0/3564159909 learned_addr learned my addr 172.21.15.120:0/3564159909 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.136+0000 7f00deffd640 1 --2- 172.21.15.120:0/3564159909 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 0x7f00e006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.136+0000 7f00df7fe640 1 --2- 172.21.15.120:0/3564159909 >> v2:172.21.15.134:3300/0 conn(0x7f00e00c9c10 0x7f00e00ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.136+0000 7f00ddffb640 1 -- 172.21.15.120:0/3564159909 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2230144100 0 0) 0x7f00e00c7d10 con 0x7f00e006bc10 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.136+0000 7f00ddffb640 1 -- 172.21.15.120:0/3564159909 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00c8003520 con 0x7f00e006bc10 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.136+0000 7f00df7fe640 1 -- 172.21.15.120:0/3564159909 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 msgr2=0x7f00e006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.136+0000 7f00df7fe640 1 --2- 172.21.15.120:0/3564159909 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 0x7f00e006cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.136+0000 7f00df7fe640 1 -- 172.21.15.120:0/3564159909 >> v1:172.21.15.120:6789/0 conn(0x7f00e006bc10 legacy=0x7f00e00c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.136+0000 7f00ddffb640 1 -- 172.21.15.120:0/3564159909 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 857414541 0 0) 0x7f00c8003520 con 0x7f00e006bc10 2024-09-10T10:29:14.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.136+0000 7f00df7fe640 1 -- 172.21.15.120:0/3564159909 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00c7ff0 con 0x7f00e00c9c10 2024-09-10T10:29:14.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.136+0000 7f00deffd640 1 --2- 172.21.15.120:0/3564159909 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 0x7f00e006cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:29:14.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00df7fe640 1 --2- 172.21.15.120:0/3564159909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00e00c9c10 0x7f00e00ca080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009d90 tx=0x7f00d802a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6204fe10e39f1a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00ddffb640 1 -- 172.21.15.120:0/3564159909 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4001670 con 0x7f00e00c9c10 2024-09-10T10:29:14.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00ddffb640 1 -- 172.21.15.120:0/3564159909 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d8007540 con 0x7f00e00c9c10 2024-09-10T10:29:14.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00ddffb640 1 -- 172.21.15.120:0/3564159909 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d80078f0 con 0x7f00e00c9c10 2024-09-10T10:29:14.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00e56ba640 1 -- 172.21.15.120:0/3564159909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00e00c9c10 msgr2=0x7f00e00ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00e56ba640 1 --2- 172.21.15.120:0/3564159909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00e00c9c10 0x7f00e00ca080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009d90 tx=0x7f00d802a560 comp rx=0 tx=0).stop 2024-09-10T10:29:14.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00e56ba640 1 -- 172.21.15.120:0/3564159909 shutdown_connections 2024-09-10T10:29:14.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00e56ba640 1 --2- 172.21.15.120:0/3564159909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00e00c9c10 0x7f00e00ca080 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00e56ba640 1 --2- 172.21.15.120:0/3564159909 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 0x7f00e006cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00e56ba640 1 -- 172.21.15.120:0/3564159909 >> 172.21.15.120:0/3564159909 conn(0x7f00e0068170 msgr2=0x7f00e0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:14.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00e56ba640 1 -- 172.21.15.120:0/3564159909 shutdown_connections 2024-09-10T10:29:14.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.137+0000 7f00e56ba640 1 -- 172.21.15.120:0/3564159909 wait complete. 2024-09-10T10:29:14.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.139+0000 7f00e56ba640 1 Processor -- start 2024-09-10T10:29:14.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.139+0000 7f00e56ba640 1 -- start start 2024-09-10T10:29:14.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.139+0000 7f00e56ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00e00c9c10 0x7f00e01f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.139+0000 7f00e56ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 0x7f00e0153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.139+0000 7f00e56ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e014e130 0x7f00e014b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.139+0000 7f00e56ba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00e015a8e0 con 0x7f00e014e130 2024-09-10T10:29:14.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.139+0000 7f00e56ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00e00c85b0 con 0x7f00e00c9c10 2024-09-10T10:29:14.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.139+0000 7f00e56ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f00e00c74f0 con 0x7f00e0154a30 2024-09-10T10:29:14.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e014e130 0x7f00e014b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e014e130 0x7f00e014b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42632/0 (socket says 172.21.15.120:42632) 2024-09-10T10:29:14.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00df7fe640 1 -- 172.21.15.120:0/799173204 learned_addr learned my addr 172.21.15.120:0/799173204 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00deffd640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00e00c9c10 0x7f00e01f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00de7fc640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 0x7f00e0153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00df7fe640 1 -- 172.21.15.120:0/799173204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 msgr2=0x7f00e0153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00df7fe640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 0x7f00e0153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00df7fe640 1 -- 172.21.15.120:0/799173204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00e00c9c10 msgr2=0x7f00e01f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00df7fe640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00e00c9c10 0x7f00e01f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00df7fe640 1 -- 172.21.15.120:0/799173204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00cdfb0 con 0x7f00e014e130 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00de7fc640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00e0154a30 0x7f00e0153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00df7fe640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e014e130 0x7f00e014b850 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f00d802ab50 tx=0x7f00d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00c77fe640 1 -- 172.21.15.120:0/799173204 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d803f070 con 0x7f00e014e130 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00c77fe640 1 -- 172.21.15.120:0/799173204 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d8031c40 con 0x7f00e014e130 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00c77fe640 1 -- 172.21.15.120:0/799173204 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8009450 con 0x7f00e014e130 2024-09-10T10:29:14.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e00ccfa0 con 0x7f00e014e130 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.140+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e00cd240 con 0x7f00e014e130 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.141+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f00e0154e10 con 0x7f00e014e130 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.142+0000 7f00c77fe640 1 -- 172.21.15.120:0/799173204 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00d8043410 con 0x7f00e014e130 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.142+0000 7f00c77fe640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f00cc06be60 0x7f00cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.142+0000 7f00c77fe640 1 -- 172.21.15.120:0/799173204 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f00d80ac9f0 con 0x7f00e014e130 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.142+0000 7f00deffd640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f00cc06be60 0x7f00cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.142+0000 7f00c77fe640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f00cc075c70 0x7f00cc078110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.142+0000 7f00c77fe640 1 -- 172.21.15.120:0/799173204 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f00d804ce50 con 0x7f00cc075c70 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.142+0000 7f00c77fe640 1 -- 172.21.15.120:0/799173204 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f00d80ace80 con 0x7f00e014e130 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.142+0000 7f00de7fc640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f00cc075c70 0x7f00cc078110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.143+0000 7f00deffd640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f00cc06be60 0x7f00cc06e320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009560 tx=0x7f00d0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.143+0000 7f00de7fc640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f00cc075c70 0x7f00cc078110 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.143+0000 7f00c77fe640 1 -- 172.21.15.120:0/799173204 <== osd.3 v2:172.21.15.120:6824/1224278950 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f00d804ce50 con 0x7f00cc075c70 2024-09-10T10:29:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.155+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 --> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f00e01e4690 con 0x7f00cc075c70 2024-09-10T10:29:14.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00c77fe640 1 -- 172.21.15.120:0/799173204 <== osd.3 v2:172.21.15.120:6824/1224278950 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f00e01e4690 con 0x7f00cc075c70 2024-09-10T10:29:14.157 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:29:14.157 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:29:14.157 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:29:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f00cc075c70 msgr2=0x7f00cc078110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00e56ba640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:6824/1224278950,v1:172.21.15.120:6825/1224278950] conn(0x7f00cc075c70 0x7f00cc078110 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f00cc06be60 msgr2=0x7f00cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00e56ba640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f00cc06be60 0x7f00cc06e320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009560 tx=0x7f00d0009190 comp rx=0 tx=0).stop 2024-09-10T10:29:14.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e014e130 msgr2=0x7f00e014b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00e56ba640 1 --2- 172.21.15.120:0/799173204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e014e130 0x7f00e014b850 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f00d802ab50 tx=0x7f00d8004a40 comp rx=0 tx=0).stop 2024-09-10T10:29:14.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00df7fe640 1 -- 172.21.15.120:0/799173204 reap_dead start 2024-09-10T10:29:14.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 shutdown_connections 2024-09-10T10:29:14.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 >> 172.21.15.120:0/799173204 conn(0x7f00e0068170 msgr2=0x7f00e00c8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:14.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 shutdown_connections 2024-09-10T10:29:14.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.156+0000 7f00e56ba640 1 -- 172.21.15.120:0/799173204 wait complete. 2024-09-10T10:29:14.167 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T10:29:14.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.265+0000 7f447b12c640 1 Processor -- start 2024-09-10T10:29:14.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.265+0000 7f447b12c640 1 -- start start 2024-09-10T10:29:14.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.266+0000 7f447b12c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f447406eaa0 0x7f44740ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.266+0000 7f447b12c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f447415aa00 0x7f44740ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.266+0000 7f447b12c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44740c9440 con 0x7f4474158140 2024-09-10T10:29:14.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.266+0000 7f447b12c640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f44740c8dc0 con 0x7f447415aa00 2024-09-10T10:29:14.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.266+0000 7f447b12c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4474158520 con 0x7f447406eaa0 2024-09-10T10:29:14.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.266+0000 7f44796c9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f447415aa00 0x7f44740ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.266+0000 7f44796c9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f447415aa00 0x7f44740ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42634/0 (socket says 172.21.15.120:42634) 2024-09-10T10:29:14.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.266+0000 7f44796c9640 1 -- 172.21.15.120:0/3300193790 learned_addr learned my addr 172.21.15.120:0/3300193790 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:14.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.266+0000 7f4473fff640 1 --2- 172.21.15.120:0/3300193790 >> v2:172.21.15.134:3300/0 conn(0x7f447406eaa0 0x7f44740ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.266+0000 7f44796c9640 1 -- 172.21.15.120:0/3300193790 >> v2:172.21.15.134:3300/0 conn(0x7f447406eaa0 msgr2=0x7f44740ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:14.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.267+0000 7f44796c9640 1 --2- 172.21.15.120:0/3300193790 >> v2:172.21.15.134:3300/0 conn(0x7f447406eaa0 0x7f44740ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.267+0000 7f44796c9640 1 -- 172.21.15.120:0/3300193790 >> v1:172.21.15.134:6789/0 conn(0x7f4474158140 legacy=0x7f447406cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.267+0000 7f44796c9640 1 -- 172.21.15.120:0/3300193790 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44740b0d40 con 0x7f447415aa00 2024-09-10T10:29:14.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.267+0000 7f44796c9640 1 --2- 172.21.15.120:0/3300193790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f447415aa00 0x7f44740ce6d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f447406fed0 tx=0x7f446c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50e4a13060cdf95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.267+0000 7f44737fe640 1 -- 172.21.15.120:0/3300193790 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c037070 con 0x7f447415aa00 2024-09-10T10:29:14.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.267+0000 7f44737fe640 1 -- 172.21.15.120:0/3300193790 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446c02ae50 con 0x7f447415aa00 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.267+0000 7f44737fe640 1 -- 172.21.15.120:0/3300193790 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c032860 con 0x7f447415aa00 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.268+0000 7f447b12c640 1 -- 172.21.15.120:0/3300193790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f447415aa00 msgr2=0x7f44740ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.268+0000 7f447b12c640 1 --2- 172.21.15.120:0/3300193790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f447415aa00 0x7f44740ce6d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f447406fed0 tx=0x7f446c02a300 comp rx=0 tx=0).stop 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.268+0000 7f447b12c640 1 -- 172.21.15.120:0/3300193790 shutdown_connections 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.268+0000 7f447b12c640 1 --2- 172.21.15.120:0/3300193790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f447415aa00 0x7f44740ce6d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.268+0000 7f447b12c640 1 --2- 172.21.15.120:0/3300193790 >> v2:172.21.15.134:3300/0 conn(0x7f447406eaa0 0x7f44740ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.268+0000 7f447b12c640 1 -- 172.21.15.120:0/3300193790 >> 172.21.15.120:0/3300193790 conn(0x7f4474067950 msgr2=0x7f4474067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.268+0000 7f447b12c640 1 -- 172.21.15.120:0/3300193790 shutdown_connections 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.268+0000 7f447b12c640 1 -- 172.21.15.120:0/3300193790 wait complete. 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.269+0000 7f447b12c640 1 Processor -- start 2024-09-10T10:29:14.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.269+0000 7f447b12c640 1 -- start start 2024-09-10T10:29:14.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f447b12c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f447406eaa0 0x7f44740b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f447b12c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4474158140 0x7f44741568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f447b12c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447415aa00 0x7f44741f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f447b12c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44740b1020 con 0x7f4474158140 2024-09-10T10:29:14.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f447b12c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44740c99c0 con 0x7f447415aa00 2024-09-10T10:29:14.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f447b12c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f44740cec10 con 0x7f447406eaa0 2024-09-10T10:29:14.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4473fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4474158140 0x7f44741568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4473fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4474158140 0x7f44741568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42640/0 (socket says 172.21.15.120:42640) 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4473fff640 1 -- 172.21.15.120:0/345656039 learned_addr learned my addr 172.21.15.120:0/345656039 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f44796c9640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447415aa00 0x7f44741f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4478ec8640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f447406eaa0 0x7f44740b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4473fff640 1 -- 172.21.15.120:0/345656039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f447406eaa0 msgr2=0x7f44740b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4473fff640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f447406eaa0 0x7f44740b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4473fff640 1 -- 172.21.15.120:0/345656039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447415aa00 msgr2=0x7f44741f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4473fff640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447415aa00 0x7f44741f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4473fff640 1 -- 172.21.15.120:0/345656039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44740af400 con 0x7f4474158140 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4478ec8640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f447406eaa0 0x7f44740b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.270+0000 7f4473fff640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4474158140 0x7f44741568b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f446800e9e0 tx=0x7f446800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.271+0000 7f44717fa640 1 -- 172.21.15.120:0/345656039 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468017070 con 0x7f4474158140 2024-09-10T10:29:14.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.271+0000 7f44717fa640 1 -- 172.21.15.120:0/345656039 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446800c9d0 con 0x7f4474158140 2024-09-10T10:29:14.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.271+0000 7f447b12c640 1 -- 172.21.15.120:0/345656039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44740af680 con 0x7f4474158140 2024-09-10T10:29:14.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.271+0000 7f44717fa640 1 -- 172.21.15.120:0/345656039 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446800cdc0 con 0x7f4474158140 2024-09-10T10:29:14.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.271+0000 7f447b12c640 1 -- 172.21.15.120:0/345656039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447415bfc0 con 0x7f4474158140 2024-09-10T10:29:14.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.272+0000 7f447b12c640 1 -- 172.21.15.120:0/345656039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4474157960 con 0x7f4474158140 2024-09-10T10:29:14.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.272+0000 7f44717fa640 1 -- 172.21.15.120:0/345656039 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f446801a070 con 0x7f4474158140 2024-09-10T10:29:14.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.272+0000 7f44717fa640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f446006be30 0x7f446006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.272+0000 7f44717fa640 1 -- 172.21.15.120:0/345656039 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f446808b5f0 con 0x7f4474158140 2024-09-10T10:29:14.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.272+0000 7f4478ec8640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f446006be30 0x7f446006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.273+0000 7f4478ec8640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f446006be30 0x7f446006e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4464001e00 tx=0x7f4464009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.277+0000 7f44717fa640 1 -- 172.21.15.120:0/345656039 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44740af400 con 0x7f4474158140 2024-09-10T10:29:14.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.440+0000 7f447b12c640 1 -- 172.21.15.120:0/345656039 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f447415c1c0 con 0x7f446006be30 2024-09-10T10:29:14.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.441+0000 7f44717fa640 1 -- 172.21.15.120:0/345656039 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f447415c1c0 con 0x7f446006be30 2024-09-10T10:29:14.442 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T10:29:14.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 -- 172.21.15.120:0/345656039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f446006be30 msgr2=0x7f446006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f446006be30 0x7f446006e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4464001e00 tx=0x7f4464009040 comp rx=0 tx=0).stop 2024-09-10T10:29:14.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 -- 172.21.15.120:0/345656039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4474158140 msgr2=0x7f44741568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4474158140 0x7f44741568b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f446800e9e0 tx=0x7f446800eea0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 -- 172.21.15.120:0/345656039 shutdown_connections 2024-09-10T10:29:14.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f446006be30 0x7f446006e2f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447415aa00 0x7f44741f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4474158140 0x7f44741568b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 --2- 172.21.15.120:0/345656039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f447406eaa0 0x7f44740b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 -- 172.21.15.120:0/345656039 >> 172.21.15.120:0/345656039 conn(0x7f4474067950 msgr2=0x7f44740cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:14.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 -- 172.21.15.120:0/345656039 shutdown_connections 2024-09-10T10:29:14.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.443+0000 7f447b12c640 1 -- 172.21.15.120:0/345656039 wait complete. 2024-09-10T10:29:14.455 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T10:29:14.455 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:29:14.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.555+0000 7fe93bfc0640 1 Processor -- start 2024-09-10T10:29:14.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.555+0000 7fe93bfc0640 1 -- start start 2024-09-10T10:29:14.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93bfc0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe934154df0 0x7fe9340c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93bfc0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe93415df90 0x7fe93415e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93bfc0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe934070390 con 0x7fe93406d2e0 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93bfc0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe93406ce50 con 0x7fe93415df90 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93bfc0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9341551d0 con 0x7fe934154df0 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93a55d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe93415df90 0x7fe93415e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93a55d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe93415df90 0x7fe93415e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42644/0 (socket says 172.21.15.120:42644) 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93a55d640 1 -- 172.21.15.120:0/4109931636 learned_addr learned my addr 172.21.15.120:0/4109931636 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe939d5c640 1 --2- 172.21.15.120:0/4109931636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe934154df0 0x7fe9340c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93a55d640 1 -- 172.21.15.120:0/4109931636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe934154df0 msgr2=0x7fe9340c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93a55d640 1 --2- 172.21.15.120:0/4109931636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe934154df0 0x7fe9340c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93a55d640 1 -- 172.21.15.120:0/4109931636 >> v1:172.21.15.134:6789/0 conn(0x7fe93406d2e0 legacy=0x7fe93406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.556+0000 7fe93a55d640 1 -- 172.21.15.120:0/4109931636 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9340b0f60 con 0x7fe93415df90 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.557+0000 7fe93a55d640 1 --2- 172.21.15.120:0/4109931636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93415df90 0x7fe93415e440 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009870 tx=0x7fe92c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4794b84ccb4dbba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.557+0000 7fe938d5a640 1 -- 172.21.15.120:0/4109931636 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c037070 con 0x7fe93415df90 2024-09-10T10:29:14.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.557+0000 7fe938d5a640 1 -- 172.21.15.120:0/4109931636 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92c032440 con 0x7fe93415df90 2024-09-10T10:29:14.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.557+0000 7fe938d5a640 1 -- 172.21.15.120:0/4109931636 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c032810 con 0x7fe93415df90 2024-09-10T10:29:14.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.558+0000 7fe93bfc0640 1 -- 172.21.15.120:0/4109931636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93415df90 msgr2=0x7fe93415e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.558+0000 7fe93bfc0640 1 --2- 172.21.15.120:0/4109931636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93415df90 0x7fe93415e440 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009870 tx=0x7fe92c02a270 comp rx=0 tx=0).stop 2024-09-10T10:29:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.558+0000 7fe93bfc0640 1 -- 172.21.15.120:0/4109931636 shutdown_connections 2024-09-10T10:29:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.558+0000 7fe93bfc0640 1 --2- 172.21.15.120:0/4109931636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93415df90 0x7fe93415e440 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.558+0000 7fe93bfc0640 1 --2- 172.21.15.120:0/4109931636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe934154df0 0x7fe9340c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.558+0000 7fe93bfc0640 1 -- 172.21.15.120:0/4109931636 >> 172.21.15.120:0/4109931636 conn(0x7fe934071720 msgr2=0x7fe934071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.558+0000 7fe93bfc0640 1 -- 172.21.15.120:0/4109931636 shutdown_connections 2024-09-10T10:29:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.558+0000 7fe93bfc0640 1 -- 172.21.15.120:0/4109931636 wait complete. 2024-09-10T10:29:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.559+0000 7fe93bfc0640 1 Processor -- start 2024-09-10T10:29:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.559+0000 7fe93bfc0640 1 -- start start 2024-09-10T10:29:14.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe93bfc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93406d2e0 0x7fe934150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe93bfc0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe934154df0 0x7fe93414a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe93bfc0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe93415df90 0x7fe9340b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe93bfc0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe934070910 con 0x7fe93406d2e0 2024-09-10T10:29:14.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe93bfc0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9340b1240 con 0x7fe934154df0 2024-09-10T10:29:14.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe93bfc0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9340c90c0 con 0x7fe93415df90 2024-09-10T10:29:14.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe939d5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93406d2e0 0x7fe934150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe939d5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93406d2e0 0x7fe934150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42660/0 (socket says 172.21.15.120:42660) 2024-09-10T10:29:14.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe939d5c640 1 -- 172.21.15.120:0/1567883764 learned_addr learned my addr 172.21.15.120:0/1567883764 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:14.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe93955b640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe934154df0 0x7fe93414a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe93a55d640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe93415df90 0x7fe9340b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe939d5c640 1 -- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe93415df90 msgr2=0x7fe9340b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe939d5c640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe93415df90 0x7fe9340b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe939d5c640 1 -- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe934154df0 msgr2=0x7fe93414a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe939d5c640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe934154df0 0x7fe93414a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe939d5c640 1 -- 172.21.15.120:0/1567883764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe934150ef0 con 0x7fe93406d2e0 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe93a55d640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe93415df90 0x7fe9340b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe939d5c640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93406d2e0 0x7fe934150840 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe924007ef0 tx=0x7fe92400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe9327fc640 1 -- 172.21.15.120:0/1567883764 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924014620 con 0x7fe93406d2e0 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe9327fc640 1 -- 172.21.15.120:0/1567883764 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe924019070 con 0x7fe93406d2e0 2024-09-10T10:29:14.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9341511d0 con 0x7fe93406d2e0 2024-09-10T10:29:14.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.560+0000 7fe9327fc640 1 -- 172.21.15.120:0/1567883764 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924014a00 con 0x7fe93406d2e0 2024-09-10T10:29:14.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.561+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93414ee60 con 0x7fe93406d2e0 2024-09-10T10:29:14.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.561+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9340c4ea0 con 0x7fe93406d2e0 2024-09-10T10:29:14.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.562+0000 7fe9327fc640 1 -- 172.21.15.120:0/1567883764 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe924014ba0 con 0x7fe93406d2e0 2024-09-10T10:29:14.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.562+0000 7fe9327fc640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe92006be30 0x7fe92006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.562+0000 7fe9327fc640 1 -- 172.21.15.120:0/1567883764 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fe92408bc50 con 0x7fe93406d2e0 2024-09-10T10:29:14.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.562+0000 7fe93955b640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe92006be30 0x7fe92006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.563+0000 7fe9327fc640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe920075c40 0x7fe9200780e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.563+0000 7fe9327fc640 1 -- 172.21.15.120:0/1567883764 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe924006b90 con 0x7fe920075c40 2024-09-10T10:29:14.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.563+0000 7fe9327fc640 1 -- 172.21.15.120:0/1567883764 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fe92408e350 con 0x7fe93406d2e0 2024-09-10T10:29:14.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.563+0000 7fe93955b640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe92006be30 0x7fe92006e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe93414adf0 tx=0x7fe928009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.563+0000 7fe93a55d640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe920075c40 0x7fe9200780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.563+0000 7fe93a55d640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe920075c40 0x7fe9200780e0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.564+0000 7fe9327fc640 1 -- 172.21.15.120:0/1567883764 <== osd.4 v2:172.21.15.134:6816/1874635282 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe924006b90 con 0x7fe920075c40 2024-09-10T10:29:14.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.576+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 --> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe9340c4bc0 con 0x7fe920075c40 2024-09-10T10:29:14.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe9327fc640 1 -- 172.21.15.120:0/1567883764 <== osd.4 v2:172.21.15.134:6816/1874635282 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe9340c4bc0 con 0x7fe920075c40 2024-09-10T10:29:14.578 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:29:14.578 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:29:14.578 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:29:14.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe920075c40 msgr2=0x7fe9200780e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93bfc0640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.134:6816/1874635282,v1:172.21.15.134:6817/1874635282] conn(0x7fe920075c40 0x7fe9200780e0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe92006be30 msgr2=0x7fe92006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93bfc0640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe92006be30 0x7fe92006e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe93414adf0 tx=0x7fe928009040 comp rx=0 tx=0).stop 2024-09-10T10:29:14.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93406d2e0 msgr2=0x7fe934150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93bfc0640 1 --2- 172.21.15.120:0/1567883764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93406d2e0 0x7fe934150840 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe924007ef0 tx=0x7fe92400e4b0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93a55d640 1 -- 172.21.15.120:0/1567883764 reap_dead start 2024-09-10T10:29:14.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 shutdown_connections 2024-09-10T10:29:14.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 >> 172.21.15.120:0/1567883764 conn(0x7fe934071720 msgr2=0x7fe93406f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:14.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 shutdown_connections 2024-09-10T10:29:14.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.577+0000 7fe93bfc0640 1 -- 172.21.15.120:0/1567883764 wait complete. 2024-09-10T10:29:14.588 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T10:29:14.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.687+0000 7fe4d8f9d640 1 Processor -- start 2024-09-10T10:29:14.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.687+0000 7fe4d8f9d640 1 -- start start 2024-09-10T10:29:14.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d8f9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe4d41515f0 0x7fe4d40c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d8f9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe4d41519d0 0x7fe4d4072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d8f9d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d40c1150 con 0x7fe4d40c8a20 2024-09-10T10:29:14.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d8f9d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d4073630 con 0x7fe4d41519d0 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d8f9d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d4073130 con 0x7fe4d41515f0 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d2ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe4d41519d0 0x7fe4d4072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d3fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fe4d40c8a20 0x7fe4d4157630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41620/0 (socket says 172.21.15.120:41620) 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d2ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe4d41519d0 0x7fe4d4072bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42676/0 (socket says 172.21.15.120:42676) 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d3fff640 1 -- 172.21.15.120:0/2223856051 learned_addr learned my addr 172.21.15.120:0/2223856051 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d37fe640 1 --2- 172.21.15.120:0/2223856051 >> v2:172.21.15.134:3300/0 conn(0x7fe4d41515f0 0x7fe4d40c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d2ffd640 1 -- 172.21.15.120:0/2223856051 >> v2:172.21.15.134:3300/0 conn(0x7fe4d41515f0 msgr2=0x7fe4d40c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d2ffd640 1 --2- 172.21.15.120:0/2223856051 >> v2:172.21.15.134:3300/0 conn(0x7fe4d41515f0 0x7fe4d40c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d2ffd640 1 -- 172.21.15.120:0/2223856051 >> v1:172.21.15.120:6789/0 conn(0x7fe4d40c8a20 legacy=0x7fe4d4157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d2ffd640 1 -- 172.21.15.120:0/2223856051 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d4073350 con 0x7fe4d41519d0 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.688+0000 7fe4d37fe640 1 --2- 172.21.15.120:0/2223856051 >> v2:172.21.15.134:3300/0 conn(0x7fe4d41515f0 0x7fe4d40c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d2ffd640 1 --2- 172.21.15.120:0/2223856051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41519d0 0x7fe4d4072bf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe4d406ccb0 tx=0x7fe4bc02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210b8e047703be59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d27fc640 1 -- 172.21.15.120:0/2223856051 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc037070 con 0x7fe4d41519d0 2024-09-10T10:29:14.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d27fc640 1 -- 172.21.15.120:0/2223856051 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4bc02adc0 con 0x7fe4d41519d0 2024-09-10T10:29:14.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d27fc640 1 -- 172.21.15.120:0/2223856051 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc032660 con 0x7fe4d41519d0 2024-09-10T10:29:14.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/2223856051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41519d0 msgr2=0x7fe4d4072bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d8f9d640 1 --2- 172.21.15.120:0/2223856051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41519d0 0x7fe4d4072bf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe4d406ccb0 tx=0x7fe4bc02a130 comp rx=0 tx=0).stop 2024-09-10T10:29:14.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/2223856051 shutdown_connections 2024-09-10T10:29:14.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d8f9d640 1 --2- 172.21.15.120:0/2223856051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41519d0 0x7fe4d4072bf0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d8f9d640 1 --2- 172.21.15.120:0/2223856051 >> v2:172.21.15.134:3300/0 conn(0x7fe4d41515f0 0x7fe4d40c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/2223856051 >> 172.21.15.120:0/2223856051 conn(0x7fe4d4067950 msgr2=0x7fe4d4067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:14.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/2223856051 shutdown_connections 2024-09-10T10:29:14.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.689+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/2223856051 wait complete. 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.691+0000 7fe4d8f9d640 1 Processor -- start 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.691+0000 7fe4d8f9d640 1 -- start start 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.691+0000 7fe4d8f9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d40c8a20 0x7fe4d41504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.691+0000 7fe4d8f9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d41515f0 0x7fe4d41ec740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.691+0000 7fe4d8f9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4d41519d0 0x7fe4d41ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.691+0000 7fe4d8f9d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40c19b0 con 0x7fe4d40c8a20 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.691+0000 7fe4d8f9d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40c16d0 con 0x7fe4d41515f0 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.691+0000 7fe4d8f9d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d4074060 con 0x7fe4d41519d0 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.691+0000 7fe4d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d40c8a20 0x7fe4d41504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d40c8a20 0x7fe4d41504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42686/0 (socket says 172.21.15.120:42686) 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d37fe640 1 -- 172.21.15.120:0/3496257211 learned_addr learned my addr 172.21.15.120:0/3496257211 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d3fff640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4d41519d0 0x7fe4d41ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d2ffd640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d41515f0 0x7fe4d41ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d37fe640 1 -- 172.21.15.120:0/3496257211 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4d41519d0 msgr2=0x7fe4d41ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d37fe640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4d41519d0 0x7fe4d41ef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d37fe640 1 -- 172.21.15.120:0/3496257211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d41515f0 msgr2=0x7fe4d41ec740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d37fe640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d41515f0 0x7fe4d41ec740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d37fe640 1 -- 172.21.15.120:0/3496257211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d4152af0 con 0x7fe4d40c8a20 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d2ffd640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d41515f0 0x7fe4d41ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d37fe640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d40c8a20 0x7fe4d41504f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400e7e0 tx=0x7fe4c400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4cbfff640 1 -- 172.21.15.120:0/3496257211 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c400f040 con 0x7fe4d40c8a20 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4cbfff640 1 -- 172.21.15.120:0/3496257211 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c4004030 con 0x7fe4d40c8a20 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4cbfff640 1 -- 172.21.15.120:0/3496257211 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4004400 con 0x7fe4d40c8a20 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/3496257211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d41564d0 con 0x7fe4d40c8a20 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.692+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/3496257211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d4156a70 con 0x7fe4d40c8a20 2024-09-10T10:29:14.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.693+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/3496257211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d4072bf0 con 0x7fe4d40c8a20 2024-09-10T10:29:14.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.694+0000 7fe4cbfff640 1 -- 172.21.15.120:0/3496257211 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4c400c6d0 con 0x7fe4d40c8a20 2024-09-10T10:29:14.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.694+0000 7fe4cbfff640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe4b406be30 0x7fe4b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.694+0000 7fe4cbfff640 1 -- 172.21.15.120:0/3496257211 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fe4c408b0c0 con 0x7fe4d40c8a20 2024-09-10T10:29:14.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.694+0000 7fe4d2ffd640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe4b406be30 0x7fe4b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.694+0000 7fe4d2ffd640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe4b406be30 0x7fe4b406e2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc034000 tx=0x7fe4bc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.698+0000 7fe4cbfff640 1 -- 172.21.15.120:0/3496257211 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe4c4017020 con 0x7fe4d40c8a20 2024-09-10T10:29:14.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.866+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/3496257211 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fe4d40c7e40 con 0x7fe4b406be30 2024-09-10T10:29:14.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.867+0000 7fe4cbfff640 1 -- 172.21.15.120:0/3496257211 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe4d40c7e40 con 0x7fe4b406be30 2024-09-10T10:29:14.868 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T10:29:14.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/3496257211 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe4b406be30 msgr2=0x7fe4b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe4b406be30 0x7fe4b406e2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc034000 tx=0x7fe4bc02e000 comp rx=0 tx=0).stop 2024-09-10T10:29:14.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/3496257211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d40c8a20 msgr2=0x7fe4d41504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d40c8a20 0x7fe4d41504f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400e7e0 tx=0x7fe4c400eca0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/3496257211 shutdown_connections 2024-09-10T10:29:14.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe4b406be30 0x7fe4b406e2f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4d41519d0 0x7fe4d41ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d41515f0 0x7fe4d41ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 --2- 172.21.15.120:0/3496257211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d40c8a20 0x7fe4d41504f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/3496257211 >> 172.21.15.120:0/3496257211 conn(0x7fe4d4067950 msgr2=0x7fe4d4157ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:14.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/3496257211 shutdown_connections 2024-09-10T10:29:14.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.869+0000 7fe4d8f9d640 1 -- 172.21.15.120:0/3496257211 wait complete. 2024-09-10T10:29:14.881 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T10:29:14.881 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:29:14.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.984+0000 7efd8000e640 1 Processor -- start 2024-09-10T10:29:14.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.984+0000 7efd8000e640 1 -- start start 2024-09-10T10:29:14.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd8000e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efd780c6220 0x7efd780c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd8000e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd781530a0 con 0x7efd7806f0f0 2024-09-10T10:29:14.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd8000e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd780c2d70 con 0x7efd78152cc0 2024-09-10T10:29:14.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd8000e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7efd78159960 con 0x7efd780c6220 2024-09-10T10:29:14.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7d5a9640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7efd7806f0f0 0x7efd780c3da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41624/0 (socket says 172.21.15.120:41624) 2024-09-10T10:29:14.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7d5a9640 1 -- 172.21.15.120:0/1357222894 learned_addr learned my addr 172.21.15.120:0/1357222894 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:14.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7e5ab640 1 --2- 172.21.15.120:0/1357222894 >> v2:172.21.15.134:3300/0 conn(0x7efd780c6220 0x7efd780c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7cda8640 1 -- 172.21.15.120:0/1357222894 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447989465 0 0) 0x7efd781530a0 con 0x7efd7806f0f0 2024-09-10T10:29:14.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7cda8640 1 -- 172.21.15.120:0/1357222894 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd64003520 con 0x7efd7806f0f0 2024-09-10T10:29:14.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7cda8640 1 -- 172.21.15.120:0/1357222894 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 565935530 0 0) 0x7efd780c2d70 con 0x7efd78152cc0 2024-09-10T10:29:14.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7cda8640 1 -- 172.21.15.120:0/1357222894 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd781530a0 con 0x7efd78152cc0 2024-09-10T10:29:14.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7cda8640 1 -- 172.21.15.120:0/1357222894 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1880361795 0 0) 0x7efd64003520 con 0x7efd7806f0f0 2024-09-10T10:29:14.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7e5ab640 1 -- 172.21.15.120:0/1357222894 >> v1:172.21.15.134:6789/0 conn(0x7efd78152cc0 legacy=0x7efd7806e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7e5ab640 1 -- 172.21.15.120:0/1357222894 >> v1:172.21.15.120:6789/0 conn(0x7efd7806f0f0 legacy=0x7efd780c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.985+0000 7efd7e5ab640 1 -- 172.21.15.120:0/1357222894 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780c8550 con 0x7efd780c6220 2024-09-10T10:29:14.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.986+0000 7efd7e5ab640 1 --2- 172.21.15.120:0/1357222894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd780c6220 0x7efd780c44b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd70009d90 tx=0x7efd7002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5f5febc1a7931e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.986+0000 7efd7cda8640 1 -- 172.21.15.120:0/1357222894 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68001670 con 0x7efd780c6220 2024-09-10T10:29:14.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.986+0000 7efd7cda8640 1 -- 172.21.15.120:0/1357222894 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd70007540 con 0x7efd780c6220 2024-09-10T10:29:14.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.986+0000 7efd7cda8640 1 -- 172.21.15.120:0/1357222894 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd700078f0 con 0x7efd780c6220 2024-09-10T10:29:14.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.987+0000 7efd8000e640 1 -- 172.21.15.120:0/1357222894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd780c6220 msgr2=0x7efd780c44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.987+0000 7efd8000e640 1 --2- 172.21.15.120:0/1357222894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd780c6220 0x7efd780c44b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd70009d90 tx=0x7efd7002a710 comp rx=0 tx=0).stop 2024-09-10T10:29:14.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.987+0000 7efd8000e640 1 -- 172.21.15.120:0/1357222894 shutdown_connections 2024-09-10T10:29:14.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.987+0000 7efd8000e640 1 --2- 172.21.15.120:0/1357222894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd780c6220 0x7efd780c44b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.987+0000 7efd8000e640 1 -- 172.21.15.120:0/1357222894 >> 172.21.15.120:0/1357222894 conn(0x7efd780cc0b0 msgr2=0x7efd780cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:14.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.987+0000 7efd8000e640 1 -- 172.21.15.120:0/1357222894 shutdown_connections 2024-09-10T10:29:14.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.987+0000 7efd8000e640 1 -- 172.21.15.120:0/1357222894 wait complete. 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.988+0000 7efd8000e640 1 Processor -- start 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.988+0000 7efd8000e640 1 -- start start 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd8000e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd7806f0f0 0x7efd781e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd8000e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd780c6220 0x7efd781e3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd8000e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd78152cc0 0x7efd781450f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd8000e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd78070320 con 0x7efd78152cc0 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd8000e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd78159680 con 0x7efd780c6220 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd8000e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efd780c8210 con 0x7efd7806f0f0 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7e5ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd78152cc0 0x7efd781450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7e5ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd78152cc0 0x7efd781450f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42702/0 (socket says 172.21.15.120:42702) 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7e5ab640 1 -- 172.21.15.120:0/1240630296 learned_addr learned my addr 172.21.15.120:0/1240630296 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7d5a9640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd780c6220 0x7efd781e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7ddaa640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd7806f0f0 0x7efd781e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7e5ab640 1 -- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd7806f0f0 msgr2=0x7efd781e32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7e5ab640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd7806f0f0 0x7efd781e32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7e5ab640 1 -- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd780c6220 msgr2=0x7efd781e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7e5ab640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd780c6220 0x7efd781e3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7e5ab640 1 -- 172.21.15.120:0/1240630296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd78148ce0 con 0x7efd78152cc0 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7d5a9640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd780c6220 0x7efd781e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.989+0000 7efd7e5ab640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd78152cc0 0x7efd781450f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efd70009eb0 tx=0x7efd70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.990+0000 7efd6e7fc640 1 -- 172.21.15.120:0/1240630296 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70041030 con 0x7efd78152cc0 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.990+0000 7efd6e7fc640 1 -- 172.21.15.120:0/1240630296 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd70007e70 con 0x7efd78152cc0 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.990+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd780c7370 con 0x7efd78152cc0 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.990+0000 7efd6e7fc640 1 -- 172.21.15.120:0/1240630296 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70009450 con 0x7efd78152cc0 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.990+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd780c7930 con 0x7efd78152cc0 2024-09-10T10:29:14.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.990+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd7806e9e0 con 0x7efd78152cc0 2024-09-10T10:29:14.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.991+0000 7efd6e7fc640 1 -- 172.21.15.120:0/1240630296 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd70039040 con 0x7efd78152cc0 2024-09-10T10:29:14.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.991+0000 7efd6e7fc640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efd5c06be30 0x7efd5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.992+0000 7efd7ddaa640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efd5c06be30 0x7efd5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.992+0000 7efd6e7fc640 1 -- 172.21.15.120:0/1240630296 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7efd700ac270 con 0x7efd78152cc0 2024-09-10T10:29:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.992+0000 7efd6e7fc640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7efd5c075c40 0x7efd5c0780e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.992+0000 7efd6e7fc640 1 -- 172.21.15.120:0/1240630296 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd700091a0 con 0x7efd5c075c40 2024-09-10T10:29:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.992+0000 7efd6e7fc640 1 -- 172.21.15.120:0/1240630296 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7efd700b3050 con 0x7efd78152cc0 2024-09-10T10:29:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.992+0000 7efd7ddaa640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efd5c06be30 0x7efd5c06e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efd740010e0 tx=0x7efd74009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.992+0000 7efd7d5a9640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7efd5c075c40 0x7efd5c0780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.992+0000 7efd7d5a9640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7efd5c075c40 0x7efd5c0780e0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:14.993+0000 7efd6e7fc640 1 -- 172.21.15.120:0/1240630296 <== osd.5 v2:172.21.15.134:6808/457724087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd700091a0 con 0x7efd5c075c40 2024-09-10T10:29:15.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 --> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efd780c3970 con 0x7efd5c075c40 2024-09-10T10:29:15.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd6e7fc640 1 -- 172.21.15.120:0/1240630296 <== osd.5 v2:172.21.15.134:6808/457724087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efd780c3970 con 0x7efd5c075c40 2024-09-10T10:29:15.007 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:29:15.007 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:29:15.008 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7efd5c075c40 msgr2=0x7efd5c0780e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd8000e640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.134:6808/457724087,v1:172.21.15.134:6809/457724087] conn(0x7efd5c075c40 0x7efd5c0780e0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efd5c06be30 msgr2=0x7efd5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd8000e640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7efd5c06be30 0x7efd5c06e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efd740010e0 tx=0x7efd74009040 comp rx=0 tx=0).stop 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd78152cc0 msgr2=0x7efd781450f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd8000e640 1 --2- 172.21.15.120:0/1240630296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd78152cc0 0x7efd781450f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efd70009eb0 tx=0x7efd70004a40 comp rx=0 tx=0).stop 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd7e5ab640 1 -- 172.21.15.120:0/1240630296 reap_dead start 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 shutdown_connections 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 >> 172.21.15.120:0/1240630296 conn(0x7efd780cc0b0 msgr2=0x7efd78158920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.006+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 shutdown_connections 2024-09-10T10:29:15.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.007+0000 7efd8000e640 1 -- 172.21.15.120:0/1240630296 wait complete. 2024-09-10T10:29:15.018 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T10:29:15.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.116+0000 7fc8ece53640 1 Processor -- start 2024-09-10T10:29:15.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.116+0000 7fc8ece53640 1 -- start start 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8ece53640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc8e80c7770 0x7fc8e80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8ece53640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 0x7fc8e80cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8ece53640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e815b1a0 con 0x7fc8e80c6980 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8ece53640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e815b9e0 con 0x7fc8e80c7770 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8ece53640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e815b420 con 0x7fc8e80cb410 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8e5d9b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc8e80c7770 0x7fc8e80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8e659c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fc8e80c6980 0x7fc8e815aa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41634/0 (socket says 172.21.15.120:41634) 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8e5d9b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc8e80c7770 0x7fc8e80cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42706/0 (socket says 172.21.15.120:42706) 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8e659c640 1 -- 172.21.15.120:0/2692143487 learned_addr learned my addr 172.21.15.120:0/2692143487 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8e5d9b640 1 -- 172.21.15.120:0/2692143487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 msgr2=0x7fc8e80cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:15.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8e6d9d640 1 --2- 172.21.15.120:0/2692143487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 0x7fc8e80cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8e5d9b640 1 --2- 172.21.15.120:0/2692143487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 0x7fc8e80cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8e559a640 1 -- 172.21.15.120:0/2692143487 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2816617879 0 0) 0x7fc8d8001490 con 0x7fc8e80c6980 2024-09-10T10:29:15.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8e5d9b640 1 -- 172.21.15.120:0/2692143487 >> v1:172.21.15.120:6789/0 conn(0x7fc8e80c6980 legacy=0x7fc8e815aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.117+0000 7fc8e5d9b640 1 -- 172.21.15.120:0/2692143487 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e815b700 con 0x7fc8e80c7770 2024-09-10T10:29:15.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.118+0000 7fc8e5d9b640 1 --2- 172.21.15.120:0/2692143487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c7770 0x7fc8e80cd730 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0097a0 tx=0x7fc8dc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78af52a69a30b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.118+0000 7fc8e559a640 1 -- 172.21.15.120:0/2692143487 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc037070 con 0x7fc8e80c7770 2024-09-10T10:29:15.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.118+0000 7fc8e559a640 1 -- 172.21.15.120:0/2692143487 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8dc02ae50 con 0x7fc8e80c7770 2024-09-10T10:29:15.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.118+0000 7fc8e559a640 1 -- 172.21.15.120:0/2692143487 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc032660 con 0x7fc8e80c7770 2024-09-10T10:29:15.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.119+0000 7fc8ece53640 1 -- 172.21.15.120:0/2692143487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c7770 msgr2=0x7fc8e80cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.119+0000 7fc8ece53640 1 --2- 172.21.15.120:0/2692143487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c7770 0x7fc8e80cd730 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0097a0 tx=0x7fc8dc02a100 comp rx=0 tx=0).stop 2024-09-10T10:29:15.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.119+0000 7fc8ece53640 1 -- 172.21.15.120:0/2692143487 shutdown_connections 2024-09-10T10:29:15.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.119+0000 7fc8ece53640 1 --2- 172.21.15.120:0/2692143487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 0x7fc8e80cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.119+0000 7fc8ece53640 1 --2- 172.21.15.120:0/2692143487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c7770 0x7fc8e80cd730 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.119+0000 7fc8ece53640 1 -- 172.21.15.120:0/2692143487 >> 172.21.15.120:0/2692143487 conn(0x7fc8e806cb70 msgr2=0x7fc8e806e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:15.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.119+0000 7fc8ece53640 1 -- 172.21.15.120:0/2692143487 shutdown_connections 2024-09-10T10:29:15.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.119+0000 7fc8ece53640 1 -- 172.21.15.120:0/2692143487 wait complete. 2024-09-10T10:29:15.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.120+0000 7fc8ece53640 1 Processor -- start 2024-09-10T10:29:15.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.120+0000 7fc8ece53640 1 -- start start 2024-09-10T10:29:15.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8ece53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c6980 0x7fc8e81569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8ece53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8e80c7770 0x7fc8e81e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8ece53640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 0x7fc8e81e3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8ece53640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e80cd210 con 0x7fc8e80c6980 2024-09-10T10:29:15.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8ece53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e80ccf30 con 0x7fc8e80c7770 2024-09-10T10:29:15.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8ece53640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e80d0fe0 con 0x7fc8e80cb410 2024-09-10T10:29:15.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c6980 0x7fc8e81569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c6980 0x7fc8e81569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42718/0 (socket says 172.21.15.120:42718) 2024-09-10T10:29:15.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e659c640 1 -- 172.21.15.120:0/3061698564 learned_addr learned my addr 172.21.15.120:0/3061698564 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:15.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e5d9b640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8e80c7770 0x7fc8e81e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e6d9d640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 0x7fc8e81e3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e659c640 1 -- 172.21.15.120:0/3061698564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 msgr2=0x7fc8e81e3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e659c640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 0x7fc8e81e3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e659c640 1 -- 172.21.15.120:0/3061698564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8e80c7770 msgr2=0x7fc8e81e2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e659c640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8e80c7770 0x7fc8e81e2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e659c640 1 -- 172.21.15.120:0/3061698564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e806e760 con 0x7fc8e80c6980 2024-09-10T10:29:15.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.121+0000 7fc8e6d9d640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 0x7fc8e81e3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:15.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.122+0000 7fc8e659c640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c6980 0x7fc8e81569b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c7c0 tx=0x7fc8d800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.122+0000 7fc8d6ffd640 1 -- 172.21.15.120:0/3061698564 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8018070 con 0x7fc8e80c6980 2024-09-10T10:29:15.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.122+0000 7fc8d6ffd640 1 -- 172.21.15.120:0/3061698564 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d800a640 con 0x7fc8e80c6980 2024-09-10T10:29:15.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.122+0000 7fc8ece53640 1 -- 172.21.15.120:0/3061698564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e80c5660 con 0x7fc8e80c6980 2024-09-10T10:29:15.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.122+0000 7fc8d6ffd640 1 -- 172.21.15.120:0/3061698564 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800aa30 con 0x7fc8e80c6980 2024-09-10T10:29:15.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.122+0000 7fc8ece53640 1 -- 172.21.15.120:0/3061698564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e8157780 con 0x7fc8e80c6980 2024-09-10T10:29:15.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.123+0000 7fc8ece53640 1 -- 172.21.15.120:0/3061698564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e80cd730 con 0x7fc8e80c6980 2024-09-10T10:29:15.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.123+0000 7fc8d6ffd640 1 -- 172.21.15.120:0/3061698564 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8d801b070 con 0x7fc8e80c6980 2024-09-10T10:29:15.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.124+0000 7fc8d6ffd640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc8c806be30 0x7fc8c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.124+0000 7fc8d6ffd640 1 -- 172.21.15.120:0/3061698564 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fc8d808a920 con 0x7fc8e80c6980 2024-09-10T10:29:15.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.124+0000 7fc8e5d9b640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc8c806be30 0x7fc8c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.124+0000 7fc8e5d9b640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc8c806be30 0x7fc8c806e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc034000 tx=0x7fc8dc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.128+0000 7fc8d6ffd640 1 -- 172.21.15.120:0/3061698564 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc8e815b1a0 con 0x7fc8e80c6980 2024-09-10T10:29:15.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.293+0000 7fc8ece53640 1 -- 172.21.15.120:0/3061698564 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fc8e80cbdc0 con 0x7fc8c806be30 2024-09-10T10:29:15.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.293+0000 7fc8d6ffd640 1 -- 172.21.15.120:0/3061698564 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc8e80cbdc0 con 0x7fc8c806be30 2024-09-10T10:29:15.294 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T10:29:15.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 -- 172.21.15.120:0/3061698564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc8c806be30 msgr2=0x7fc8c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc8c806be30 0x7fc8c806e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc034000 tx=0x7fc8dc02e000 comp rx=0 tx=0).stop 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 -- 172.21.15.120:0/3061698564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c6980 msgr2=0x7fc8e81569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c6980 0x7fc8e81569b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c7c0 tx=0x7fc8d800cc80 comp rx=0 tx=0).stop 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 -- 172.21.15.120:0/3061698564 shutdown_connections 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fc8c806be30 0x7fc8c806e2f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8e80cb410 0x7fc8e81e3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8e80c7770 0x7fc8e81e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 --2- 172.21.15.120:0/3061698564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8e80c6980 0x7fc8e81569b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 -- 172.21.15.120:0/3061698564 >> 172.21.15.120:0/3061698564 conn(0x7fc8e806cb70 msgr2=0x7fc8e80c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 -- 172.21.15.120:0/3061698564 shutdown_connections 2024-09-10T10:29:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.295+0000 7fc8ece53640 1 -- 172.21.15.120:0/3061698564 wait complete. 2024-09-10T10:29:15.307 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T10:29:15.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:29:15.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.409+0000 7ff46b623640 1 Processor -- start 2024-09-10T10:29:15.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.409+0000 7ff46b623640 1 -- start start 2024-09-10T10:29:15.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.410+0000 7ff46b623640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff4640c8c80 0x7ff46406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.410+0000 7ff46b623640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4640cad10 con 0x7ff464154e90 2024-09-10T10:29:15.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.410+0000 7ff46b623640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4640c91c0 con 0x7ff46406d260 2024-09-10T10:29:15.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.410+0000 7ff46b623640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff4640715f0 con 0x7ff4640c8c80 2024-09-10T10:29:15.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.410+0000 7ff4693bf640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff464154e90 0x7ff4640ca600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41650/0 (socket says 172.21.15.120:41650) 2024-09-10T10:29:15.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.410+0000 7ff4693bf640 1 -- 172.21.15.120:0/2899184595 learned_addr learned my addr 172.21.15.120:0/2899184595 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:15.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.410+0000 7ff468bbe640 1 --2- 172.21.15.120:0/2899184595 >> v2:172.21.15.120:3300/0 conn(0x7ff4640c8c80 0x7ff46406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.411+0000 7ff468bbe640 1 -- 172.21.15.120:0/2899184595 >> v1:172.21.15.134:6789/0 conn(0x7ff46406d260 legacy=0x7ff4640c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:15.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.411+0000 7ff468bbe640 1 -- 172.21.15.120:0/2899184595 >> v1:172.21.15.120:6789/0 conn(0x7ff464154e90 legacy=0x7ff4640ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.411+0000 7ff468bbe640 1 -- 172.21.15.120:0/2899184595 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff464071840 con 0x7ff4640c8c80 2024-09-10T10:29:15.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.411+0000 7ff468bbe640 1 --2- 172.21.15.120:0/2899184595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4640c8c80 0x7ff46406cd20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff4540097a0 tx=0x7ff45402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0543d275523a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.411+0000 7ff45bfff640 1 -- 172.21.15.120:0/2899184595 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454037070 con 0x7ff4640c8c80 2024-09-10T10:29:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.411+0000 7ff45bfff640 1 -- 172.21.15.120:0/2899184595 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff454032440 con 0x7ff4640c8c80 2024-09-10T10:29:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.411+0000 7ff45bfff640 1 -- 172.21.15.120:0/2899184595 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454032810 con 0x7ff4640c8c80 2024-09-10T10:29:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.412+0000 7ff46b623640 1 -- 172.21.15.120:0/2899184595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4640c8c80 msgr2=0x7ff46406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.412+0000 7ff46b623640 1 --2- 172.21.15.120:0/2899184595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4640c8c80 0x7ff46406cd20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff4540097a0 tx=0x7ff45402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.412+0000 7ff46b623640 1 -- 172.21.15.120:0/2899184595 shutdown_connections 2024-09-10T10:29:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.412+0000 7ff46b623640 1 --2- 172.21.15.120:0/2899184595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4640c8c80 0x7ff46406cd20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.412+0000 7ff46b623640 1 -- 172.21.15.120:0/2899184595 >> 172.21.15.120:0/2899184595 conn(0x7ff46406bc10 msgr2=0x7ff46406fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.412+0000 7ff46b623640 1 -- 172.21.15.120:0/2899184595 shutdown_connections 2024-09-10T10:29:15.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.412+0000 7ff46b623640 1 -- 172.21.15.120:0/2899184595 wait complete. 2024-09-10T10:29:15.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.413+0000 7ff46b623640 1 Processor -- start 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.413+0000 7ff46b623640 1 -- start start 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff46b623640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff46406d260 0x7ff4641f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff46b623640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4640c8c80 0x7ff4641f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff46b623640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff464154e90 0x7ff464157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff46b623640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff464071e00 con 0x7ff4640c8c80 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff46b623640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4640ca170 con 0x7ff464154e90 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff46b623640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff46415df80 con 0x7ff46406d260 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff468bbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4640c8c80 0x7ff4641f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff468bbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4640c8c80 0x7ff4641f3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42728/0 (socket says 172.21.15.120:42728) 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff468bbe640 1 -- 172.21.15.120:0/3362162812 learned_addr learned my addr 172.21.15.120:0/3362162812 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff4693bf640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff46406d260 0x7ff4641f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff469bc0640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff464154e90 0x7ff464157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff468bbe640 1 -- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff46406d260 msgr2=0x7ff4641f08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff468bbe640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff46406d260 0x7ff4641f08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff468bbe640 1 -- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff464154e90 msgr2=0x7ff464157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff468bbe640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff464154e90 0x7ff464157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff468bbe640 1 -- 172.21.15.120:0/3362162812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4641f20c0 con 0x7ff4640c8c80 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff468bbe640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4640c8c80 0x7ff4641f3440 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff4540098c0 tx=0x7ff4540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.414+0000 7ff459ffb640 1 -- 172.21.15.120:0/3362162812 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454037040 con 0x7ff4640c8c80 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.415+0000 7ff459ffb640 1 -- 172.21.15.120:0/3362162812 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff454042070 con 0x7ff4640c8c80 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.415+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4641f2680 con 0x7ff4640c8c80 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.415+0000 7ff459ffb640 1 -- 172.21.15.120:0/3362162812 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454030a70 con 0x7ff4640c8c80 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.415+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641f2c40 con 0x7ff4640c8c80 2024-09-10T10:29:15.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.416+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff46406cd20 con 0x7ff4640c8c80 2024-09-10T10:29:15.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.416+0000 7ff459ffb640 1 -- 172.21.15.120:0/3362162812 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff454030c10 con 0x7ff4640c8c80 2024-09-10T10:29:15.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.417+0000 7ff459ffb640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff44806be30 0x7ff44806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.417+0000 7ff459ffb640 1 -- 172.21.15.120:0/3362162812 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7ff4540ac740 con 0x7ff4640c8c80 2024-09-10T10:29:15.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.417+0000 7ff4693bf640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff44806be30 0x7ff44806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.417+0000 7ff459ffb640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff448075c40 0x7ff4480780e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.417+0000 7ff459ffb640 1 -- 172.21.15.120:0/3362162812 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff454001d30 con 0x7ff448075c40 2024-09-10T10:29:15.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.417+0000 7ff459ffb640 1 -- 172.21.15.120:0/3362162812 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7ff4540acbd0 con 0x7ff4640c8c80 2024-09-10T10:29:15.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.417+0000 7ff469bc0640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff448075c40 0x7ff4480780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.417+0000 7ff4693bf640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff44806be30 0x7ff44806e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff460007fd0 tx=0x7ff460007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.418+0000 7ff469bc0640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff448075c40 0x7ff4480780e0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.418+0000 7ff459ffb640 1 -- 172.21.15.120:0/3362162812 <== osd.6 v2:172.21.15.134:6824/50133702 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff454001d30 con 0x7ff448075c40 2024-09-10T10:29:15.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.431+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 --> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff4641e7290 con 0x7ff448075c40 2024-09-10T10:29:15.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.431+0000 7ff459ffb640 1 -- 172.21.15.120:0/3362162812 <== osd.6 v2:172.21.15.134:6824/50133702 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff4641e7290 con 0x7ff448075c40 2024-09-10T10:29:15.432 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:29:15.433 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:29:15.433 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff448075c40 msgr2=0x7ff4480780e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff46b623640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.134:6824/50133702,v1:172.21.15.134:6825/50133702] conn(0x7ff448075c40 0x7ff4480780e0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff44806be30 msgr2=0x7ff44806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff46b623640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff44806be30 0x7ff44806e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff460007fd0 tx=0x7ff460007e80 comp rx=0 tx=0).stop 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4640c8c80 msgr2=0x7ff4641f3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff46b623640 1 --2- 172.21.15.120:0/3362162812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4640c8c80 0x7ff4641f3440 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff4540098c0 tx=0x7ff4540046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff469bc0640 1 -- 172.21.15.120:0/3362162812 reap_dead start 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 shutdown_connections 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 >> 172.21.15.120:0/3362162812 conn(0x7ff46406bc10 msgr2=0x7ff4640c3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 shutdown_connections 2024-09-10T10:29:15.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.432+0000 7ff46b623640 1 -- 172.21.15.120:0/3362162812 wait complete. 2024-09-10T10:29:15.443 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T10:29:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.540+0000 7effaca53640 1 Processor -- start 2024-09-10T10:29:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.540+0000 7effaca53640 1 -- start start 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effaca53640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7effa80c9580 0x7effa8159170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effaca53640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c5140 0x7effa8163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effaca53640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa815cc60 con 0x7effa8152ae0 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effaca53640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7effa81596b0 con 0x7effa80c9580 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effaca53640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7effa8152ec0 con 0x7effa80c5140 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa659c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7effa8152ae0 0x7effa80c3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41658/0 (socket says 172.21.15.120:41658) 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa659c640 1 -- 172.21.15.120:0/2471302998 learned_addr learned my addr 172.21.15.120:0/2471302998 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa5d9b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7effa80c9580 0x7effa8159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa559a640 1 -- 172.21.15.120:0/2471302998 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945266892 0 0) 0x7effa815cc60 con 0x7effa8152ae0 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa6d9d640 1 --2- 172.21.15.120:0/2471302998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c5140 0x7effa8163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa559a640 1 -- 172.21.15.120:0/2471302998 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff90003520 con 0x7effa8152ae0 2024-09-10T10:29:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa5d9b640 1 -- 172.21.15.120:0/2471302998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c5140 msgr2=0x7effa8163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa5d9b640 1 --2- 172.21.15.120:0/2471302998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c5140 0x7effa8163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa5d9b640 1 -- 172.21.15.120:0/2471302998 >> v1:172.21.15.120:6789/0 conn(0x7effa8152ae0 legacy=0x7effa80c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa5d9b640 1 -- 172.21.15.120:0/2471302998 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa80c9a70 con 0x7effa80c9580 2024-09-10T10:29:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.541+0000 7effa6d9d640 1 --2- 172.21.15.120:0/2471302998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c5140 0x7effa8163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:15.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.542+0000 7effa5d9b640 1 --2- 172.21.15.120:0/2471302998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa80c9580 0x7effa8159170 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0097a0 tx=0x7eff9c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2b94fc91d90448 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.542+0000 7effa559a640 1 -- 172.21.15.120:0/2471302998 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c037070 con 0x7effa80c9580 2024-09-10T10:29:15.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.542+0000 7effa559a640 1 -- 172.21.15.120:0/2471302998 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff9c032440 con 0x7effa80c9580 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.542+0000 7effa559a640 1 -- 172.21.15.120:0/2471302998 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c032810 con 0x7effa80c9580 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.543+0000 7effaca53640 1 -- 172.21.15.120:0/2471302998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa80c9580 msgr2=0x7effa8159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.543+0000 7effaca53640 1 --2- 172.21.15.120:0/2471302998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa80c9580 0x7effa8159170 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0097a0 tx=0x7eff9c02a180 comp rx=0 tx=0).stop 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.543+0000 7effaca53640 1 -- 172.21.15.120:0/2471302998 shutdown_connections 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.543+0000 7effaca53640 1 --2- 172.21.15.120:0/2471302998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c5140 0x7effa8163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.543+0000 7effaca53640 1 --2- 172.21.15.120:0/2471302998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa80c9580 0x7effa8159170 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.543+0000 7effaca53640 1 -- 172.21.15.120:0/2471302998 >> 172.21.15.120:0/2471302998 conn(0x7effa806bc10 msgr2=0x7effa806fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.543+0000 7effaca53640 1 -- 172.21.15.120:0/2471302998 shutdown_connections 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.543+0000 7effaca53640 1 -- 172.21.15.120:0/2471302998 wait complete. 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.544+0000 7effaca53640 1 Processor -- start 2024-09-10T10:29:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.544+0000 7effaca53640 1 -- start start 2024-09-10T10:29:15.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effaca53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effa80c5140 0x7effa81f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effaca53640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c9580 0x7effa8154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effaca53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa8152ae0 0x7effa8156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effaca53640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7effa8158970 con 0x7effa8152ae0 2024-09-10T10:29:15.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effaca53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7effa80ca030 con 0x7effa80c5140 2024-09-10T10:29:15.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effaca53640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7effa8158c50 con 0x7effa80c9580 2024-09-10T10:29:15.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa8152ae0 0x7effa8156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa8152ae0 0x7effa8156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42754/0 (socket says 172.21.15.120:42754) 2024-09-10T10:29:15.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa6d9d640 1 -- 172.21.15.120:0/3674906567 learned_addr learned my addr 172.21.15.120:0/3674906567 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa659c640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effa80c5140 0x7effa81f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa5d9b640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c9580 0x7effa8154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa6d9d640 1 -- 172.21.15.120:0/3674906567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c9580 msgr2=0x7effa8154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa6d9d640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c9580 0x7effa8154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa6d9d640 1 -- 172.21.15.120:0/3674906567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effa80c5140 msgr2=0x7effa81f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa6d9d640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effa80c5140 0x7effa81f1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa6d9d640 1 -- 172.21.15.120:0/3674906567 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa8156160 con 0x7effa8152ae0 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.545+0000 7effa659c640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effa80c5140 0x7effa81f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.546+0000 7effa6d9d640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa8152ae0 0x7effa8156bf0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7effa000cca0 tx=0x7effa000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.546+0000 7eff96ffd640 1 -- 172.21.15.120:0/3674906567 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa000f680 con 0x7effa8152ae0 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.546+0000 7eff96ffd640 1 -- 172.21.15.120:0/3674906567 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa000f840 con 0x7effa8152ae0 2024-09-10T10:29:15.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.546+0000 7effaca53640 1 -- 172.21.15.120:0/3674906567 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa8155ba0 con 0x7effa8152ae0 2024-09-10T10:29:15.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.546+0000 7eff96ffd640 1 -- 172.21.15.120:0/3674906567 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa000fc30 con 0x7effa8152ae0 2024-09-10T10:29:15.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.546+0000 7effaca53640 1 -- 172.21.15.120:0/3674906567 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa81ee700 con 0x7effa8152ae0 2024-09-10T10:29:15.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.547+0000 7effaca53640 1 -- 172.21.15.120:0/3674906567 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa80c8640 con 0x7effa8152ae0 2024-09-10T10:29:15.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.547+0000 7eff96ffd640 1 -- 172.21.15.120:0/3674906567 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7effa00213f0 con 0x7effa8152ae0 2024-09-10T10:29:15.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.548+0000 7eff96ffd640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7eff8806be30 0x7eff8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.548+0000 7eff96ffd640 1 -- 172.21.15.120:0/3674906567 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7effa008c0d0 con 0x7effa8152ae0 2024-09-10T10:29:15.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.548+0000 7effa659c640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7eff8806be30 0x7eff8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.548+0000 7effa659c640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7eff8806be30 0x7eff8806e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7eff98007fd0 tx=0x7eff98009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.552+0000 7eff96ffd640 1 -- 172.21.15.120:0/3674906567 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7effa0014030 con 0x7effa8152ae0 2024-09-10T10:29:15.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.719+0000 7effaca53640 1 -- 172.21.15.120:0/3674906567 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7effa8151a50 con 0x7eff8806be30 2024-09-10T10:29:15.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.720+0000 7eff96ffd640 1 -- 172.21.15.120:0/3674906567 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7effa8151a50 con 0x7eff8806be30 2024-09-10T10:29:15.721 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T10:29:15.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 -- 172.21.15.120:0/3674906567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7eff8806be30 msgr2=0x7eff8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7eff8806be30 0x7eff8806e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7eff98007fd0 tx=0x7eff98009040 comp rx=0 tx=0).stop 2024-09-10T10:29:15.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 -- 172.21.15.120:0/3674906567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa8152ae0 msgr2=0x7effa8156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa8152ae0 0x7effa8156bf0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7effa000cca0 tx=0x7effa000a650 comp rx=0 tx=0).stop 2024-09-10T10:29:15.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 -- 172.21.15.120:0/3674906567 shutdown_connections 2024-09-10T10:29:15.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7eff8806be30 0x7eff8806e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effa8152ae0 0x7effa8156bf0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa80c9580 0x7effa8154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 --2- 172.21.15.120:0/3674906567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effa80c5140 0x7effa81f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 -- 172.21.15.120:0/3674906567 >> 172.21.15.120:0/3674906567 conn(0x7effa806bc10 msgr2=0x7effa80cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:15.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 -- 172.21.15.120:0/3674906567 shutdown_connections 2024-09-10T10:29:15.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.722+0000 7effaca53640 1 -- 172.21.15.120:0/3674906567 wait complete. 2024-09-10T10:29:15.734 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T10:29:15.734 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:29:15.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.836+0000 7f9bdb591640 1 Processor -- start 2024-09-10T10:29:15.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.836+0000 7f9bdb591640 1 -- start start 2024-09-10T10:29:15.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bdb591640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9bd4154e90 0x7f9bd40ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bdb591640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9bd40cab40 0x7f9bd406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bdb591640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bd40c8c80 0x7f9bd406d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bdb591640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9bd40715f0 con 0x7f9bd40cab40 2024-09-10T10:29:15.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bdb591640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9bd40ca170 con 0x7f9bd4154e90 2024-09-10T10:29:15.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bdb591640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd40c91c0 con 0x7f9bd40c8c80 2024-09-10T10:29:15.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd8b2c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9bd40cab40 0x7f9bd406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd8b2c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9bd40cab40 0x7f9bd406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42762/0 (socket says 172.21.15.120:42762) 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd8b2c640 1 -- 172.21.15.120:0/3744982199 learned_addr learned my addr 172.21.15.120:0/3744982199 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd932d640 1 --2- 172.21.15.120:0/3744982199 >> v2:172.21.15.134:3300/0 conn(0x7f9bd4154e90 0x7f9bd40ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd9b2e640 1 --2- 172.21.15.120:0/3744982199 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bd40c8c80 0x7f9bd406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd8b2c640 1 -- 172.21.15.120:0/3744982199 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bd40c8c80 msgr2=0x7f9bd406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd8b2c640 1 --2- 172.21.15.120:0/3744982199 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bd40c8c80 0x7f9bd406d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd8b2c640 1 -- 172.21.15.120:0/3744982199 >> v2:172.21.15.134:3300/0 conn(0x7f9bd4154e90 msgr2=0x7f9bd40ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd8b2c640 1 --2- 172.21.15.120:0/3744982199 >> v2:172.21.15.134:3300/0 conn(0x7f9bd4154e90 0x7f9bd40ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd8b2c640 1 -- 172.21.15.120:0/3744982199 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd4071840 con 0x7f9bd40cab40 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd932d640 1 --2- 172.21.15.120:0/3744982199 >> v2:172.21.15.134:3300/0 conn(0x7f9bd4154e90 0x7f9bd40ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:15.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.837+0000 7f9bd9b2e640 1 --2- 172.21.15.120:0/3744982199 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bd40c8c80 0x7f9bd406d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:15.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.838+0000 7f9bd8b2c640 1 --2- 172.21.15.120:0/3744982199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bd40cab40 0x7f9bd406cd20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40cf320 tx=0x7f9bc402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5510d59634c1a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.838+0000 7f9bcbfff640 1 -- 172.21.15.120:0/3744982199 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4037070 con 0x7f9bd40cab40 2024-09-10T10:29:15.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.838+0000 7f9bcbfff640 1 -- 172.21.15.120:0/3744982199 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc4032440 con 0x7f9bd40cab40 2024-09-10T10:29:15.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.838+0000 7f9bcbfff640 1 -- 172.21.15.120:0/3744982199 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4032810 con 0x7f9bd40cab40 2024-09-10T10:29:15.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.839+0000 7f9bdb591640 1 -- 172.21.15.120:0/3744982199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bd40cab40 msgr2=0x7f9bd406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.839+0000 7f9bdb591640 1 --2- 172.21.15.120:0/3744982199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bd40cab40 0x7f9bd406cd20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40cf320 tx=0x7f9bc402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.839+0000 7f9bdb591640 1 -- 172.21.15.120:0/3744982199 shutdown_connections 2024-09-10T10:29:15.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.839+0000 7f9bdb591640 1 --2- 172.21.15.120:0/3744982199 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bd40c8c80 0x7f9bd406d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.839+0000 7f9bdb591640 1 --2- 172.21.15.120:0/3744982199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bd40cab40 0x7f9bd406cd20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.839+0000 7f9bdb591640 1 --2- 172.21.15.120:0/3744982199 >> v2:172.21.15.134:3300/0 conn(0x7f9bd4154e90 0x7f9bd40ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.839+0000 7f9bdb591640 1 -- 172.21.15.120:0/3744982199 >> 172.21.15.120:0/3744982199 conn(0x7f9bd406bc10 msgr2=0x7f9bd406fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:15.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.839+0000 7f9bdb591640 1 -- 172.21.15.120:0/3744982199 shutdown_connections 2024-09-10T10:29:15.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.839+0000 7f9bdb591640 1 -- 172.21.15.120:0/3744982199 wait complete. 2024-09-10T10:29:15.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.840+0000 7f9bdb591640 1 Processor -- start 2024-09-10T10:29:15.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.840+0000 7f9bdb591640 1 -- start start 2024-09-10T10:29:15.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bdb591640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bd40c8c80 0x7f9bd40c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bdb591640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bd40cab40 0x7f9bd414e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bdb591640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bd4154e90 0x7f9bd41f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bdb591640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd415df80 con 0x7f9bd4154e90 2024-09-10T10:29:15.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bdb591640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4071e00 con 0x7f9bd40c8c80 2024-09-10T10:29:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bdb591640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd40c9620 con 0x7f9bd40cab40 2024-09-10T10:29:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd9b2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bd4154e90 0x7f9bd41f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd9b2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bd4154e90 0x7f9bd41f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42774/0 (socket says 172.21.15.120:42774) 2024-09-10T10:29:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd9b2e640 1 -- 172.21.15.120:0/802926 learned_addr learned my addr 172.21.15.120:0/802926 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd8b2c640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bd40cab40 0x7f9bd414e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd932d640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bd40c8c80 0x7f9bd40c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd9b2e640 1 -- 172.21.15.120:0/802926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bd40cab40 msgr2=0x7f9bd414e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd9b2e640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bd40cab40 0x7f9bd414e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd9b2e640 1 -- 172.21.15.120:0/802926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bd40c8c80 msgr2=0x7f9bd40c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd9b2e640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bd40c8c80 0x7f9bd40c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd9b2e640 1 -- 172.21.15.120:0/802926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd414ecb0 con 0x7f9bd4154e90 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bd9b2e640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bd4154e90 0x7f9bd41f4290 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00cf20 tx=0x7f9bcc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.841+0000 7f9bc9ffb640 1 -- 172.21.15.120:0/802926 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc01a070 con 0x7f9bd4154e90 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.842+0000 7f9bc9ffb640 1 -- 172.21.15.120:0/802926 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bcc00d840 con 0x7f9bd4154e90 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.842+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd41e3e40 con 0x7f9bd4154e90 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.842+0000 7f9bc9ffb640 1 -- 172.21.15.120:0/802926 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc00dc30 con 0x7f9bd4154e90 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.842+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd41e4400 con 0x7f9bd4154e90 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.842+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9bd40ca600 con 0x7f9bd4154e90 2024-09-10T10:29:15.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.843+0000 7f9bc9ffb640 1 -- 172.21.15.120:0/802926 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bcc01d070 con 0x7f9bd4154e90 2024-09-10T10:29:15.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.844+0000 7f9bc9ffb640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9bb806be60 0x7f9bb806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.844+0000 7f9bc9ffb640 1 -- 172.21.15.120:0/802926 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f9bcc08c260 con 0x7f9bd4154e90 2024-09-10T10:29:15.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.844+0000 7f9bd932d640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9bb806be60 0x7f9bb806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.844+0000 7f9bc9ffb640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f9bb8075c70 0x7f9bb8078110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.844+0000 7f9bc9ffb640 1 -- 172.21.15.120:0/802926 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9bcc006b90 con 0x7f9bb8075c70 2024-09-10T10:29:15.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.844+0000 7f9bc9ffb640 1 -- 172.21.15.120:0/802926 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f9bcc08c6f0 con 0x7f9bd4154e90 2024-09-10T10:29:15.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.844+0000 7f9bd8b2c640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f9bb8075c70 0x7f9bb8078110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.844+0000 7f9bd932d640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9bb806be60 0x7f9bb806e320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000aac0 tx=0x7f9bd0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.844+0000 7f9bd8b2c640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f9bb8075c70 0x7f9bb8078110 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.845+0000 7f9bc9ffb640 1 -- 172.21.15.120:0/802926 <== osd.7 v2:172.21.15.134:6800/251960875 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9bcc006b90 con 0x7f9bb8075c70 2024-09-10T10:29:15.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.858+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 --> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9bd40c5070 con 0x7f9bb8075c70 2024-09-10T10:29:15.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.858+0000 7f9bc9ffb640 1 -- 172.21.15.120:0/802926 <== osd.7 v2:172.21.15.134:6800/251960875 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9bd40c5070 con 0x7f9bb8075c70 2024-09-10T10:29:15.860 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T10:29:15.860 INFO:teuthology.orchestra.run.smithi120.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:29:15.860 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T10:29:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.858+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f9bb8075c70 msgr2=0x7f9bb8078110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.858+0000 7f9bdb591640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.134:6800/251960875,v1:172.21.15.134:6801/251960875] conn(0x7f9bb8075c70 0x7f9bb8078110 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.858+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9bb806be60 msgr2=0x7f9bb806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.858+0000 7f9bdb591640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9bb806be60 0x7f9bb806e320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000aac0 tx=0x7f9bd0008040 comp rx=0 tx=0).stop 2024-09-10T10:29:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.858+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bd4154e90 msgr2=0x7f9bd41f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.858+0000 7f9bdb591640 1 --2- 172.21.15.120:0/802926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9bd4154e90 0x7f9bd41f4290 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00cf20 tx=0x7f9bcc00a710 comp rx=0 tx=0).stop 2024-09-10T10:29:15.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.858+0000 7f9bd9b2e640 1 -- 172.21.15.120:0/802926 reap_dead start 2024-09-10T10:29:15.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.859+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 shutdown_connections 2024-09-10T10:29:15.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.859+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 >> 172.21.15.120:0/802926 conn(0x7f9bd406bc10 msgr2=0x7f9bd415d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:15.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.859+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 shutdown_connections 2024-09-10T10:29:15.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.859+0000 7f9bdb591640 1 -- 172.21.15.120:0/802926 wait complete. 2024-09-10T10:29:15.870 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T10:29:15.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.970+0000 7f0f652dd640 1 Processor -- start 2024-09-10T10:29:15.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.970+0000 7f0f652dd640 1 -- start start 2024-09-10T10:29:15.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f652dd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f600c6530 0x7f0f60152570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f652dd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0f600c6910 0x7f0f60152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f652dd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f6006beb0 con 0x7f0f601591e0 2024-09-10T10:29:15.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f652dd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f0f60067df0 con 0x7f0f600c6910 2024-09-10T10:29:15.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f652dd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0f601596d0 con 0x7f0f600c6530 2024-09-10T10:29:15.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5f7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f0f601591e0 0x7f0f6015be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:41672/0 (socket says 172.21.15.120:41672) 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5f7fe640 1 -- 172.21.15.120:0/1077157396 learned_addr learned my addr 172.21.15.120:0/1077157396 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f567fc640 1 --2- 172.21.15.120:0/1077157396 >> v2:172.21.15.134:3300/0 conn(0x7f0f600c6910 0x7f0f60152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5effd640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f600c6530 0x7f0f60152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2873663110 0 0) 0x7f0f6006beb0 con 0x7f0f601591e0 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f4c003520 con 0x7f0f601591e0 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 <== mon.0 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 408234382 0 0) 0x7f0f4c003520 con 0x7f0f601591e0 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f600c6530 msgr2=0x7f0f60152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f600c6530 0x7f0f60152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 >> v2:172.21.15.134:3300/0 conn(0x7f0f600c6910 msgr2=0x7f0f60152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 --2- 172.21.15.120:0/1077157396 >> v2:172.21.15.134:3300/0 conn(0x7f0f600c6910 0x7f0f60152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f600c7650 con 0x7f0f601591e0 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 <== mon.0 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2404224011 0 0) 0x7f0f580019f0 con 0x7f0f601591e0 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5effd640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f600c6530 0x7f0f60152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 >> v1:172.21.15.120:6789/0 conn(0x7f0f601591e0 legacy=0x7f0f6015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f4c0039b0 0x7f0f4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c004800 0x7f0f4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f600c6530 0x7f0f4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f60067df0 con 0x7f0f4c0039b0 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0f601596d0 con 0x7f0f4c004800 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4c00b9c0 con 0x7f0f600c6530 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.971+0000 7f0f567fc640 1 --2- 172.21.15.120:0/1077157396 >> v2:172.21.15.134:3300/0 conn(0x7f0f600c6910 0x7f0f60152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.972+0000 7f0f5effd640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f4c0039b0 0x7f0f4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.972+0000 7f0f5f7fe640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f600c6530 0x7f0f4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.972+0000 7f0f567fc640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c004800 0x7f0f4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.972+0000 7f0f5effd640 1 -- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f600c6530 msgr2=0x7f0f4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.972+0000 7f0f5effd640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f600c6530 0x7f0f4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.972+0000 7f0f5effd640 1 -- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c004800 msgr2=0x7f0f4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.972+0000 7f0f5effd640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c004800 0x7f0f4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.972+0000 7f0f5effd640 1 -- 172.21.15.120:0/1077157396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c003520 con 0x7f0f4c0039b0 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.972+0000 7f0f5f7fe640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f600c6530 0x7f0f4c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.972+0000 7f0f5effd640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f4c0039b0 0x7f0f4c003dd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009c90 tx=0x7f0f4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca42f4d2baa61e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.973+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50036070 con 0x7f0f4c0039b0 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.973+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f50031400 con 0x7f0f4c0039b0 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.973+0000 7f0f5e7fc640 1 -- 172.21.15.120:0/1077157396 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50009860 con 0x7f0f4c0039b0 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.973+0000 7f0f652dd640 1 -- 172.21.15.120:0/1077157396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f4c0039b0 msgr2=0x7f0f4c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.973+0000 7f0f652dd640 1 --2- 172.21.15.120:0/1077157396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f4c0039b0 0x7f0f4c003dd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009c90 tx=0x7f0f4c005830 comp rx=0 tx=0).stop 2024-09-10T10:29:15.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.973+0000 7f0f5f7fe640 1 -- 172.21.15.120:0/1077157396 reap_dead start 2024-09-10T10:29:15.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.973+0000 7f0f652dd640 1 -- 172.21.15.120:0/1077157396 shutdown_connections 2024-09-10T10:29:15.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.973+0000 7f0f652dd640 1 -- 172.21.15.120:0/1077157396 >> 172.21.15.120:0/1077157396 conn(0x7f0f600ccbb0 msgr2=0x7f0f600ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:15.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.973+0000 7f0f652dd640 1 -- 172.21.15.120:0/1077157396 shutdown_connections 2024-09-10T10:29:15.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.973+0000 7f0f652dd640 1 -- 172.21.15.120:0/1077157396 wait complete. 2024-09-10T10:29:15.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.974+0000 7f0f652dd640 1 Processor -- start 2024-09-10T10:29:15.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.975+0000 7f0f652dd640 1 -- start start 2024-09-10T10:29:15.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.975+0000 7f0f652dd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0039b0 0x7f0f600c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.975+0000 7f0f652dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f60155e40 0x7f0f601e5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.975+0000 7f0f652dd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f601e5860 0x7f0f601f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.975+0000 7f0f652dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f600c7c10 con 0x7f0f60155e40 2024-09-10T10:29:15.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.975+0000 7f0f652dd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0f60067a80 con 0x7f0f4c0039b0 2024-09-10T10:29:15.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.975+0000 7f0f652dd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6015c560 con 0x7f0f601e5860 2024-09-10T10:29:15.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.975+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f60155e40 0x7f0f601e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.975+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f60155e40 0x7f0f601e5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42800/0 (socket says 172.21.15.120:42800) 2024-09-10T10:29:15.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.975+0000 7f0f567fc640 1 -- 172.21.15.120:0/3657786949 learned_addr learned my addr 172.21.15.120:0/3657786949 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:15.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f5effd640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0039b0 0x7f0f600c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f5f7fe640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f601e5860 0x7f0f601f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f567fc640 1 -- 172.21.15.120:0/3657786949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f601e5860 msgr2=0x7f0f601f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f567fc640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f601e5860 0x7f0f601f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f567fc640 1 -- 172.21.15.120:0/3657786949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0039b0 msgr2=0x7f0f600c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:15.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f567fc640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0039b0 0x7f0f600c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:15.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f567fc640 1 -- 172.21.15.120:0/3657786949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6015bbf0 con 0x7f0f60155e40 2024-09-10T10:29:15.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f5effd640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0039b0 0x7f0f600c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:15.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f567fc640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f60155e40 0x7f0f601e5320 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0f48011930 tx=0x7f0f48011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f57fff640 1 -- 172.21.15.120:0/3657786949 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48004220 con 0x7f0f60155e40 2024-09-10T10:29:15.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f57fff640 1 -- 172.21.15.120:0/3657786949 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f480043c0 con 0x7f0f60155e40 2024-09-10T10:29:15.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f652dd640 1 -- 172.21.15.120:0/3657786949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f600c6070 con 0x7f0f60155e40 2024-09-10T10:29:15.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f57fff640 1 -- 172.21.15.120:0/3657786949 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800fbd0 con 0x7f0f60155e40 2024-09-10T10:29:15.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.976+0000 7f0f652dd640 1 -- 172.21.15.120:0/3657786949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6015b910 con 0x7f0f60155e40 2024-09-10T10:29:15.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.977+0000 7f0f652dd640 1 -- 172.21.15.120:0/3657786949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f600c91b0 con 0x7f0f60155e40 2024-09-10T10:29:15.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.978+0000 7f0f57fff640 1 -- 172.21.15.120:0/3657786949 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f48003610 con 0x7f0f60155e40 2024-09-10T10:29:15.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.978+0000 7f0f57fff640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0f4006be60 0x7f0f4006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:15.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.978+0000 7f0f57fff640 1 -- 172.21.15.120:0/3657786949 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f0f4808b7c0 con 0x7f0f60155e40 2024-09-10T10:29:15.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.978+0000 7f0f5effd640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0f4006be60 0x7f0f4006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:15.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.979+0000 7f0f5effd640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0f4006be60 0x7f0f4006e320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0f50033000 tx=0x7f0f50008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:15.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:15.983+0000 7f0f57fff640 1 -- 172.21.15.120:0/3657786949 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0f48019030 con 0x7f0f60155e40 2024-09-10T10:29:16.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.152+0000 7f0f652dd640 1 -- 172.21.15.120:0/3657786949 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f0f600cae30 con 0x7f0f4006be60 2024-09-10T10:29:16.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.153+0000 7f0f57fff640 1 -- 172.21.15.120:0/3657786949 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0f600cae30 con 0x7f0f4006be60 2024-09-10T10:29:16.154 INFO:teuthology.orchestra.run.smithi120.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T10:29:16.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 -- 172.21.15.120:0/3657786949 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0f4006be60 msgr2=0x7f0f4006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:16.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0f4006be60 0x7f0f4006e320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0f50033000 tx=0x7f0f50008030 comp rx=0 tx=0).stop 2024-09-10T10:29:16.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 -- 172.21.15.120:0/3657786949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f60155e40 msgr2=0x7f0f601e5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:16.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f60155e40 0x7f0f601e5320 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0f48011930 tx=0x7f0f48011df0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 -- 172.21.15.120:0/3657786949 shutdown_connections 2024-09-10T10:29:16.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f0f4006be60 0x7f0f4006e320 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f601e5860 0x7f0f601f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f60155e40 0x7f0f601e5320 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 --2- 172.21.15.120:0/3657786949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0039b0 0x7f0f600c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 -- 172.21.15.120:0/3657786949 >> 172.21.15.120:0/3657786949 conn(0x7f0f600ccbb0 msgr2=0x7f0f600cb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:16.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 -- 172.21.15.120:0/3657786949 shutdown_connections 2024-09-10T10:29:16.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.155+0000 7f0f652dd640 1 -- 172.21.15.120:0/3657786949 wait complete. 2024-09-10T10:29:16.167 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:29:16.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.263+0000 7fe6761d6640 1 Processor -- start 2024-09-10T10:29:16.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.263+0000 7fe6761d6640 1 -- start start 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.264+0000 7fe6761d6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe670071b30 0x7fe6700c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.264+0000 7fe6761d6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6700c98d0 0x7fe670071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.264+0000 7fe6761d6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6701578a0 con 0x7fe670072450 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.264+0000 7fe6761d6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe6700c7a80 con 0x7fe670071b30 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.264+0000 7fe6761d6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe670072f30 con 0x7fe6700c98d0 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.264+0000 7fe66f7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe670071b30 0x7fe6700c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.264+0000 7fe66f7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe670071b30 0x7fe6700c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42802/0 (socket says 172.21.15.120:42802) 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.264+0000 7fe66f7fe640 1 -- 172.21.15.120:0/4276185720 learned_addr learned my addr 172.21.15.120:0/4276185720 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.265+0000 7fe667fff640 1 --2- 172.21.15.120:0/4276185720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6700c98d0 0x7fe670071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.265+0000 7fe66f7fe640 1 -- 172.21.15.120:0/4276185720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6700c98d0 msgr2=0x7fe670071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.265+0000 7fe66f7fe640 1 --2- 172.21.15.120:0/4276185720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6700c98d0 0x7fe670071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.265+0000 7fe66f7fe640 1 -- 172.21.15.120:0/4276185720 >> v1:172.21.15.134:6789/0 conn(0x7fe670072450 legacy=0x7fe670150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:16.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.265+0000 7fe66f7fe640 1 -- 172.21.15.120:0/4276185720 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe670157b40 con 0x7fe670071b30 2024-09-10T10:29:16.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.265+0000 7fe667fff640 1 --2- 172.21.15.120:0/4276185720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6700c98d0 0x7fe670071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:16.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.265+0000 7fe66f7fe640 1 --2- 172.21.15.120:0/4276185720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 0x7fe6700c6dc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe660009870 tx=0x7fe66002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed4edfcff590940 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:16.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.265+0000 7fe66effd640 1 -- 172.21.15.120:0/4276185720 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660037070 con 0x7fe670071b30 2024-09-10T10:29:16.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.265+0000 7fe66effd640 1 -- 172.21.15.120:0/4276185720 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe660032440 con 0x7fe670071b30 2024-09-10T10:29:16.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.266+0000 7fe66effd640 1 -- 172.21.15.120:0/4276185720 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660032810 con 0x7fe670071b30 2024-09-10T10:29:16.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.266+0000 7fe6761d6640 1 -- 172.21.15.120:0/4276185720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 msgr2=0x7fe6700c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:16.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.266+0000 7fe6761d6640 1 --2- 172.21.15.120:0/4276185720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 0x7fe6700c6dc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe660009870 tx=0x7fe66002a2a0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.266+0000 7fe6761d6640 1 -- 172.21.15.120:0/4276185720 shutdown_connections 2024-09-10T10:29:16.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.266+0000 7fe6761d6640 1 --2- 172.21.15.120:0/4276185720 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6700c98d0 0x7fe670071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.266+0000 7fe6761d6640 1 --2- 172.21.15.120:0/4276185720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 0x7fe6700c6dc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.266+0000 7fe6761d6640 1 -- 172.21.15.120:0/4276185720 >> 172.21.15.120:0/4276185720 conn(0x7fe67006f3c0 msgr2=0x7fe67006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:16.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.266+0000 7fe6761d6640 1 -- 172.21.15.120:0/4276185720 shutdown_connections 2024-09-10T10:29:16.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.266+0000 7fe6761d6640 1 -- 172.21.15.120:0/4276185720 wait complete. 2024-09-10T10:29:16.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.267+0000 7fe6761d6640 1 Processor -- start 2024-09-10T10:29:16.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.267+0000 7fe6761d6640 1 -- start start 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe6761d6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 0x7fe67014d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe6761d6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe670072450 0x7fe67014a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe6761d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6700c98d0 0x7fe670153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe6761d6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe670151480 con 0x7fe670071b30 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe6761d6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe670151300 con 0x7fe6700c98d0 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe6761d6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe6700c8cb0 con 0x7fe670072450 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe66f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 0x7fe67014d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe66f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 0x7fe67014d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42806/0 (socket says 172.21.15.120:42806) 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe66f7fe640 1 -- 172.21.15.120:0/3611117384 learned_addr learned my addr 172.21.15.120:0/3611117384 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe667fff640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe670072450 0x7fe67014a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe66ffff640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6700c98d0 0x7fe670153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe66f7fe640 1 -- 172.21.15.120:0/3611117384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe670072450 msgr2=0x7fe67014a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:16.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe66f7fe640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe670072450 0x7fe67014a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe66f7fe640 1 -- 172.21.15.120:0/3611117384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6700c98d0 msgr2=0x7fe670153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:16.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe66f7fe640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6700c98d0 0x7fe670153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe66f7fe640 1 -- 172.21.15.120:0/3611117384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67014c8b0 con 0x7fe670071b30 2024-09-10T10:29:16.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.268+0000 7fe66f7fe640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 0x7fe67014d320 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe660009870 tx=0x7fe6600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:16.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.269+0000 7fe66cff9640 1 -- 172.21.15.120:0/3611117384 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660037040 con 0x7fe670071b30 2024-09-10T10:29:16.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.269+0000 7fe66cff9640 1 -- 172.21.15.120:0/3611117384 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe660042070 con 0x7fe670071b30 2024-09-10T10:29:16.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.269+0000 7fe6761d6640 1 -- 172.21.15.120:0/3611117384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67014df80 con 0x7fe670071b30 2024-09-10T10:29:16.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.269+0000 7fe66cff9640 1 -- 172.21.15.120:0/3611117384 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660030bb0 con 0x7fe670071b30 2024-09-10T10:29:16.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.269+0000 7fe6761d6640 1 -- 172.21.15.120:0/3611117384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67014cb20 con 0x7fe670071b30 2024-09-10T10:29:16.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.270+0000 7fe6761d6640 1 -- 172.21.15.120:0/3611117384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6700c90f0 con 0x7fe670071b30 2024-09-10T10:29:16.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.270+0000 7fe66cff9640 1 -- 172.21.15.120:0/3611117384 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe660047460 con 0x7fe670071b30 2024-09-10T10:29:16.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.271+0000 7fe66cff9640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe65006be30 0x7fe65006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:16.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.271+0000 7fe66cff9640 1 -- 172.21.15.120:0/3611117384 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fe6600abdc0 con 0x7fe670071b30 2024-09-10T10:29:16.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.271+0000 7fe667fff640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe65006be30 0x7fe65006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:16.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.271+0000 7fe667fff640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe65006be30 0x7fe65006e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe67014f1c0 tx=0x7fe658008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:16.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.275+0000 7fe66cff9640 1 -- 172.21.15.120:0/3611117384 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe660079cb0 con 0x7fe670071b30 2024-09-10T10:29:16.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.436+0000 7fe6761d6640 1 -- 172.21.15.120:0/3611117384 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe670156f00 con 0x7fe65006be30 2024-09-10T10:29:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.444+0000 7fe66cff9640 1 -- 172.21.15.120:0/3611117384 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+331569 (secure 0 0 0) 0x7fe670156f00 con 0x7fe65006be30 2024-09-10T10:29:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:16.450 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:29:16.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 -- 172.21.15.120:0/3611117384 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe65006be30 msgr2=0x7fe65006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:16.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe65006be30 0x7fe65006e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe67014f1c0 tx=0x7fe658008040 comp rx=0 tx=0).stop 2024-09-10T10:29:16.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 -- 172.21.15.120:0/3611117384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 msgr2=0x7fe67014d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:16.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 0x7fe67014d320 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe660009870 tx=0x7fe6600046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 -- 172.21.15.120:0/3611117384 shutdown_connections 2024-09-10T10:29:16.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe65006be30 0x7fe65006e2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6700c98d0 0x7fe670153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe670072450 0x7fe67014a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 --2- 172.21.15.120:0/3611117384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe670071b30 0x7fe67014d320 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:16.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 -- 172.21.15.120:0/3611117384 >> 172.21.15.120:0/3611117384 conn(0x7fe67006f3c0 msgr2=0x7fe6700c56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:16.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 -- 172.21.15.120:0/3611117384 shutdown_connections 2024-09-10T10:29:16.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:16.447+0000 7fe6761d6640 1 -- 172.21.15.120:0/3611117384 wait complete. 2024-09-10T10:29:16.459 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-10T10:29:16.121950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634862,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":119,"num_read_kb":103,"num_write":417,"num_write_kb":2544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":848,"ondisk_log_size":848,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227968,"kb_used_data":7616,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754746752,"statfs":{"total":773094113280,"available":772860674048,"internally_reserved":0,"allocated":7798784,"data_stored":5122900,"data_compressed":68176,"data_compressed_allocated":98304,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.641004"},"pg_stats":[{"pgid":"2.1f","version":"28'13","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893443+0000","last_change":"2024-09-10T10:26:40.784984+0000","last_active":"2024-09-10T10:29:05.893443+0000","last_peered":"2024-09-10T10:29:05.893443+0000","last_clean":"2024-09-10T10:29:05.893443+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:29:05.893443+0000","last_undegraded":"2024-09-10T10:29:05.893443+0000","last_fullsized":"2024-09-10T10:29:05.893443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"78'5","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.209362+0000","last_change":"2024-09-10T10:26:41.802325+0000","last_active":"2024-09-10T10:29:10.209362+0000","last_peered":"2024-09-10T10:29:10.209362+0000","last_clean":"2024-09-10T10:29:10.209362+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:29:10.209362+0000","last_undegraded":"2024-09-10T10:29:10.209362+0000","last_fullsized":"2024-09-10T10:29:10.209362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831790+0000","last_change":"2024-09-10T10:26:46.192408+0000","last_active":"2024-09-10T10:29:05.831790+0000","last_peered":"2024-09-10T10:29:05.831790+0000","last_clean":"2024-09-10T10:29:05.831790+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:29:05.831790+0000","last_undegraded":"2024-09-10T10:29:05.831790+0000","last_fullsized":"2024-09-10T10:29:05.831790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:00.173134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832208+0000","last_change":"2024-09-10T10:26:40.786742+0000","last_active":"2024-09-10T10:29:05.832208+0000","last_peered":"2024-09-10T10:29:05.832208+0000","last_clean":"2024-09-10T10:29:05.832208+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:29:05.832208+0000","last_undegraded":"2024-09-10T10:29:05.832208+0000","last_fullsized":"2024-09-10T10:29:05.832208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"78'6","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.212057+0000","last_change":"2024-09-10T10:26:41.795899+0000","last_active":"2024-09-10T10:29:10.212057+0000","last_peered":"2024-09-10T10:29:10.212057+0000","last_clean":"2024-09-10T10:29:10.212057+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:29:10.212057+0000","last_undegraded":"2024-09-10T10:29:10.212057+0000","last_fullsized":"2024-09-10T10:29:10.212057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955552+0000","last_change":"2024-09-10T10:26:45.815168+0000","last_active":"2024-09-10T10:29:05.955552+0000","last_peered":"2024-09-10T10:29:05.955552+0000","last_clean":"2024-09-10T10:29:05.955552+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:29:05.955552+0000","last_undegraded":"2024-09-10T10:29:05.955552+0000","last_fullsized":"2024-09-10T10:29:05.955552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"60'13","reported_seq":159,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521221+0000","last_change":"2024-09-10T10:27:51.060242+0000","last_active":"2024-09-10T10:29:05.521221+0000","last_peered":"2024-09-10T10:29:05.521221+0000","last_clean":"2024-09-10T10:29:05.521221+0000","last_became_active":"2024-09-10T10:27:51.049148+0000","last_became_peered":"2024-09-10T10:27:51.049148+0000","last_unstale":"2024-09-10T10:29:05.521221+0000","last_undegraded":"2024-09-10T10:29:05.521221+0000","last_fullsized":"2024-09-10T10:29:05.521221+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.604144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"78'9","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.217629+0000","last_change":"2024-09-10T10:26:41.785444+0000","last_active":"2024-09-10T10:29:10.217629+0000","last_peered":"2024-09-10T10:29:10.217629+0000","last_clean":"2024-09-10T10:29:10.217629+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:29:10.217629+0000","last_undegraded":"2024-09-10T10:29:10.217629+0000","last_fullsized":"2024-09-10T10:29:10.217629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955770+0000","last_change":"2024-09-10T10:28:24.244697+0000","last_active":"2024-09-10T10:29:05.955770+0000","last_peered":"2024-09-10T10:29:05.955770+0000","last_clean":"2024-09-10T10:29:05.955770+0000","last_became_active":"2024-09-10T10:28:24.244262+0000","last_became_peered":"2024-09-10T10:28:24.244262+0000","last_unstale":"2024-09-10T10:29:05.955770+0000","last_undegraded":"2024-09-10T10:29:05.955770+0000","last_fullsized":"2024-09-10T10:29:05.955770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:50.683186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6,2],"acting":[4,0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"28'5","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520322+0000","last_change":"2024-09-10T10:26:40.793176+0000","last_active":"2024-09-10T10:29:05.520322+0000","last_peered":"2024-09-10T10:29:05.520322+0000","last_clean":"2024-09-10T10:29:05.520322+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:29:05.520322+0000","last_undegraded":"2024-09-10T10:29:05.520322+0000","last_fullsized":"2024-09-10T10:29:05.520322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"78'15","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.197161+0000","last_change":"2024-09-10T10:27:23.954718+0000","last_active":"2024-09-10T10:29:10.197161+0000","last_peered":"2024-09-10T10:29:10.197161+0000","last_clean":"2024-09-10T10:29:10.197161+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:29:10.197161+0000","last_undegraded":"2024-09-10T10:29:10.197161+0000","last_fullsized":"2024-09-10T10:29:10.197161+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955722+0000","last_change":"2024-09-10T10:28:59.443607+0000","last_active":"2024-09-10T10:29:05.955722+0000","last_peered":"2024-09-10T10:29:05.955722+0000","last_clean":"2024-09-10T10:29:05.955722+0000","last_became_active":"2024-09-10T10:28:59.443172+0000","last_became_peered":"2024-09-10T10:28:59.443172+0000","last_unstale":"2024-09-10T10:29:05.955722+0000","last_undegraded":"2024-09-10T10:29:05.955722+0000","last_fullsized":"2024-09-10T10:29:05.955722+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:36.379163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"78'5","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.201235+0000","last_change":"2024-09-10T10:26:41.788902+0000","last_active":"2024-09-10T10:29:10.201235+0000","last_peered":"2024-09-10T10:29:10.201235+0000","last_clean":"2024-09-10T10:29:10.201235+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:29:10.201235+0000","last_undegraded":"2024-09-10T10:29:10.201235+0000","last_fullsized":"2024-09-10T10:29:10.201235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"60'5","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520220+0000","last_change":"2024-09-10T10:26:40.790489+0000","last_active":"2024-09-10T10:29:05.520220+0000","last_peered":"2024-09-10T10:29:05.520220+0000","last_clean":"2024-09-10T10:29:05.520220+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:29:05.520220+0000","last_undegraded":"2024-09-10T10:29:05.520220+0000","last_fullsized":"2024-09-10T10:29:05.520220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:56.339055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956119+0000","last_change":"2024-09-10T10:27:23.955276+0000","last_active":"2024-09-10T10:29:05.956119+0000","last_peered":"2024-09-10T10:29:05.956119+0000","last_clean":"2024-09-10T10:29:05.956119+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:29:05.956119+0000","last_undegraded":"2024-09-10T10:29:05.956119+0000","last_fullsized":"2024-09-10T10:29:05.956119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:22.642266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"78'8","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.206996+0000","last_change":"2024-09-10T10:26:41.800354+0000","last_active":"2024-09-10T10:29:10.206996+0000","last_peered":"2024-09-10T10:29:10.206996+0000","last_clean":"2024-09-10T10:29:10.206996+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:29:10.206996+0000","last_undegraded":"2024-09-10T10:29:10.206996+0000","last_fullsized":"2024-09-10T10:29:10.206996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"78'5","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.955684+0000","last_change":"2024-09-10T10:26:40.789359+0000","last_active":"2024-09-10T10:29:10.955684+0000","last_peered":"2024-09-10T10:29:10.955684+0000","last_clean":"2024-09-10T10:29:10.955684+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:29:10.955684+0000","last_undegraded":"2024-09-10T10:29:10.955684+0000","last_fullsized":"2024-09-10T10:29:10.955684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955843+0000","last_change":"2024-09-10T10:26:45.816874+0000","last_active":"2024-09-10T10:29:05.955843+0000","last_peered":"2024-09-10T10:29:05.955843+0000","last_clean":"2024-09-10T10:29:05.955843+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:29:05.955843+0000","last_undegraded":"2024-09-10T10:29:05.955843+0000","last_fullsized":"2024-09-10T10:29:05.955843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:00.174746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"78'9","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.200530+0000","last_change":"2024-09-10T10:26:41.797926+0000","last_active":"2024-09-10T10:29:10.200530+0000","last_peered":"2024-09-10T10:29:10.200530+0000","last_clean":"2024-09-10T10:29:10.200530+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:29:10.200530+0000","last_undegraded":"2024-09-10T10:29:10.200530+0000","last_fullsized":"2024-09-10T10:29:10.200530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'7","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688280+0000","last_change":"2024-09-10T10:26:40.789141+0000","last_active":"2024-09-10T10:29:05.688280+0000","last_peered":"2024-09-10T10:29:05.688280+0000","last_clean":"2024-09-10T10:29:05.688280+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:29:05.688280+0000","last_undegraded":"2024-09-10T10:29:05.688280+0000","last_fullsized":"2024-09-10T10:29:05.688280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688696+0000","last_change":"2024-09-10T10:26:46.194293+0000","last_active":"2024-09-10T10:29:05.688696+0000","last_peered":"2024-09-10T10:29:05.688696+0000","last_clean":"2024-09-10T10:29:05.688696+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:29:05.688696+0000","last_undegraded":"2024-09-10T10:29:05.688696+0000","last_fullsized":"2024-09-10T10:29:05.688696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"78'2","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.190534+0000","last_change":"2024-09-10T10:26:41.791430+0000","last_active":"2024-09-10T10:29:10.190534+0000","last_peered":"2024-09-10T10:29:10.190534+0000","last_clean":"2024-09-10T10:29:10.190534+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:29:10.190534+0000","last_undegraded":"2024-09-10T10:29:10.190534+0000","last_fullsized":"2024-09-10T10:29:10.190534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:57.344011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'2","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520435+0000","last_change":"2024-09-10T10:26:40.792790+0000","last_active":"2024-09-10T10:29:05.520435+0000","last_peered":"2024-09-10T10:29:05.520435+0000","last_clean":"2024-09-10T10:29:05.520435+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:29:05.520435+0000","last_undegraded":"2024-09-10T10:29:05.520435+0000","last_fullsized":"2024-09-10T10:29:05.520435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956141+0000","last_change":"2024-09-10T10:26:45.815877+0000","last_active":"2024-09-10T10:29:05.956141+0000","last_peered":"2024-09-10T10:29:05.956141+0000","last_clean":"2024-09-10T10:29:05.956141+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:29:05.956141+0000","last_undegraded":"2024-09-10T10:29:05.956141+0000","last_fullsized":"2024-09-10T10:29:05.956141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:50:44.391191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"78'7","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.205770+0000","last_change":"2024-09-10T10:26:41.795895+0000","last_active":"2024-09-10T10:29:10.205770+0000","last_peered":"2024-09-10T10:29:10.205770+0000","last_clean":"2024-09-10T10:29:10.205770+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:29:10.205770+0000","last_undegraded":"2024-09-10T10:29:10.205770+0000","last_fullsized":"2024-09-10T10:29:10.205770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'8","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485743+0000","last_change":"2024-09-10T10:26:40.792863+0000","last_active":"2024-09-10T10:29:05.485743+0000","last_peered":"2024-09-10T10:29:05.485743+0000","last_clean":"2024-09-10T10:29:05.485743+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:29:05.485743+0000","last_undegraded":"2024-09-10T10:29:05.485743+0000","last_fullsized":"2024-09-10T10:29:05.485743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955720+0000","last_change":"2024-09-10T10:26:46.196495+0000","last_active":"2024-09-10T10:29:05.955720+0000","last_peered":"2024-09-10T10:29:05.955720+0000","last_clean":"2024-09-10T10:29:05.955720+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:29:05.955720+0000","last_undegraded":"2024-09-10T10:29:05.955720+0000","last_fullsized":"2024-09-10T10:29:05.955720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"78'5","reported_seq":69,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.217744+0000","last_change":"2024-09-10T10:28:24.374205+0000","last_active":"2024-09-10T10:29:10.217744+0000","last_peered":"2024-09-10T10:29:10.217744+0000","last_clean":"2024-09-10T10:29:10.217744+0000","last_became_active":"2024-09-10T10:28:24.373921+0000","last_became_peered":"2024-09-10T10:28:24.373921+0000","last_unstale":"2024-09-10T10:29:10.217744+0000","last_undegraded":"2024-09-10T10:29:10.217744+0000","last_fullsized":"2024-09-10T10:29:10.217744+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:36.923293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'16","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.849983+0000","last_change":"2024-09-10T10:26:40.796255+0000","last_active":"2024-09-10T10:29:10.849983+0000","last_peered":"2024-09-10T10:29:10.849983+0000","last_clean":"2024-09-10T10:29:10.849983+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:29:10.849983+0000","last_undegraded":"2024-09-10T10:29:10.849983+0000","last_fullsized":"2024-09-10T10:29:10.849983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831546+0000","last_change":"2024-09-10T10:26:46.192439+0000","last_active":"2024-09-10T10:29:05.831546+0000","last_peered":"2024-09-10T10:29:05.831546+0000","last_clean":"2024-09-10T10:29:05.831546+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:29:05.831546+0000","last_undegraded":"2024-09-10T10:29:05.831546+0000","last_fullsized":"2024-09-10T10:29:05.831546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:50.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"78'9","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.195412+0000","last_change":"2024-09-10T10:26:41.788524+0000","last_active":"2024-09-10T10:29:10.195412+0000","last_peered":"2024-09-10T10:29:10.195412+0000","last_clean":"2024-09-10T10:29:10.195412+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:29:10.195412+0000","last_undegraded":"2024-09-10T10:29:10.195412+0000","last_fullsized":"2024-09-10T10:29:10.195412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.521226+0000","last_change":"2024-09-10T10:26:40.792147+0000","last_active":"2024-09-10T10:29:05.521226+0000","last_peered":"2024-09-10T10:29:05.521226+0000","last_clean":"2024-09-10T10:29:05.521226+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:29:05.521226+0000","last_undegraded":"2024-09-10T10:29:05.521226+0000","last_fullsized":"2024-09-10T10:29:05.521226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:33.215230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893131+0000","last_change":"2024-09-10T10:26:46.193336+0000","last_active":"2024-09-10T10:29:05.893131+0000","last_peered":"2024-09-10T10:29:05.893131+0000","last_clean":"2024-09-10T10:29:05.893131+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:29:05.893131+0000","last_undegraded":"2024-09-10T10:29:05.893131+0000","last_fullsized":"2024-09-10T10:29:05.893131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"78'5","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.216267+0000","last_change":"2024-09-10T10:26:41.791108+0000","last_active":"2024-09-10T10:29:10.216267+0000","last_peered":"2024-09-10T10:29:10.216267+0000","last_clean":"2024-09-10T10:29:10.216267+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:29:10.216267+0000","last_undegraded":"2024-09-10T10:29:10.216267+0000","last_fullsized":"2024-09-10T10:29:10.216267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688976+0000","last_change":"2024-09-10T10:26:40.786684+0000","last_active":"2024-09-10T10:29:05.688976+0000","last_peered":"2024-09-10T10:29:05.688976+0000","last_clean":"2024-09-10T10:29:05.688976+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.688976+0000","last_undegraded":"2024-09-10T10:29:05.688976+0000","last_fullsized":"2024-09-10T10:29:05.688976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688499+0000","last_change":"2024-09-10T10:26:45.820966+0000","last_active":"2024-09-10T10:29:05.688499+0000","last_peered":"2024-09-10T10:29:05.688499+0000","last_clean":"2024-09-10T10:29:05.688499+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:29:05.688499+0000","last_undegraded":"2024-09-10T10:29:05.688499+0000","last_fullsized":"2024-09-10T10:29:05.688499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"78'7","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.202409+0000","last_change":"2024-09-10T10:26:41.788329+0000","last_active":"2024-09-10T10:29:10.202409+0000","last_peered":"2024-09-10T10:29:10.202409+0000","last_clean":"2024-09-10T10:29:10.202409+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:29:10.202409+0000","last_undegraded":"2024-09-10T10:29:10.202409+0000","last_fullsized":"2024-09-10T10:29:10.202409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:17.243759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'5","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486359+0000","last_change":"2024-09-10T10:26:40.790693+0000","last_active":"2024-09-10T10:29:05.486359+0000","last_peered":"2024-09-10T10:29:05.486359+0000","last_clean":"2024-09-10T10:29:05.486359+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:29:05.486359+0000","last_undegraded":"2024-09-10T10:29:05.486359+0000","last_fullsized":"2024-09-10T10:29:05.486359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:16.238804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486083+0000","last_change":"2024-09-10T10:28:24.244988+0000","last_active":"2024-09-10T10:29:05.486083+0000","last_peered":"2024-09-10T10:29:05.486083+0000","last_clean":"2024-09-10T10:29:05.486083+0000","last_became_active":"2024-09-10T10:28:24.244556+0000","last_became_peered":"2024-09-10T10:28:24.244556+0000","last_unstale":"2024-09-10T10:29:05.486083+0000","last_undegraded":"2024-09-10T10:29:05.486083+0000","last_fullsized":"2024-09-10T10:29:05.486083+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:08.127492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"78'8","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.195218+0000","last_change":"2024-09-10T10:26:41.792592+0000","last_active":"2024-09-10T10:29:10.195218+0000","last_peered":"2024-09-10T10:29:10.195218+0000","last_clean":"2024-09-10T10:29:10.195218+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:29:10.195218+0000","last_undegraded":"2024-09-10T10:29:10.195218+0000","last_fullsized":"2024-09-10T10:29:10.195218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'9","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482627+0000","last_change":"2024-09-10T10:26:40.786511+0000","last_active":"2024-09-10T10:29:05.482627+0000","last_peered":"2024-09-10T10:29:05.482627+0000","last_clean":"2024-09-10T10:29:05.482627+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:05.482627+0000","last_undegraded":"2024-09-10T10:29:05.482627+0000","last_fullsized":"2024-09-10T10:29:05.482627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":119,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688749+0000","last_change":"2024-09-10T10:26:45.824029+0000","last_active":"2024-09-10T10:29:05.688749+0000","last_peered":"2024-09-10T10:29:05.688749+0000","last_clean":"2024-09-10T10:29:05.688749+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:29:05.688749+0000","last_undegraded":"2024-09-10T10:29:05.688749+0000","last_fullsized":"2024-09-10T10:29:05.688749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"78'5","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.207315+0000","last_change":"2024-09-10T10:28:59.446889+0000","last_active":"2024-09-10T10:29:10.207315+0000","last_peered":"2024-09-10T10:29:10.207315+0000","last_clean":"2024-09-10T10:29:10.207315+0000","last_became_active":"2024-09-10T10:28:59.446544+0000","last_became_peered":"2024-09-10T10:28:59.446544+0000","last_unstale":"2024-09-10T10:29:10.207315+0000","last_undegraded":"2024-09-10T10:29:10.207315+0000","last_fullsized":"2024-09-10T10:29:10.207315+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'4","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520235+0000","last_change":"2024-09-10T10:26:40.791831+0000","last_active":"2024-09-10T10:29:05.520235+0000","last_peered":"2024-09-10T10:29:05.520235+0000","last_clean":"2024-09-10T10:29:05.520235+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:29:05.520235+0000","last_undegraded":"2024-09-10T10:29:05.520235+0000","last_fullsized":"2024-09-10T10:29:05.520235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893322+0000","last_change":"2024-09-10T10:26:46.193973+0000","last_active":"2024-09-10T10:29:05.893322+0000","last_peered":"2024-09-10T10:29:05.893322+0000","last_clean":"2024-09-10T10:29:05.893322+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:29:05.893322+0000","last_undegraded":"2024-09-10T10:29:05.893322+0000","last_fullsized":"2024-09-10T10:29:05.893322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"78'8","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.200376+0000","last_change":"2024-09-10T10:26:41.801830+0000","last_active":"2024-09-10T10:29:10.200376+0000","last_peered":"2024-09-10T10:29:10.200376+0000","last_clean":"2024-09-10T10:29:10.200376+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:29:10.200376+0000","last_undegraded":"2024-09-10T10:29:10.200376+0000","last_fullsized":"2024-09-10T10:29:10.200376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:59.037284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486160+0000","last_change":"2024-09-10T10:26:40.786626+0000","last_active":"2024-09-10T10:29:05.486160+0000","last_peered":"2024-09-10T10:29:05.486160+0000","last_clean":"2024-09-10T10:29:05.486160+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:29:05.486160+0000","last_undegraded":"2024-09-10T10:29:05.486160+0000","last_fullsized":"2024-09-10T10:29:05.486160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":281,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.520647+0000","last_change":"2024-09-10T10:26:32.268385+0000","last_active":"2024-09-10T10:29:05.520647+0000","last_peered":"2024-09-10T10:29:05.520647+0000","last_clean":"2024-09-10T10:29:05.520647+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:29:05.520647+0000","last_undegraded":"2024-09-10T10:29:05.520647+0000","last_fullsized":"2024-09-10T10:29:05.520647+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:31.081647+0000","last_clean_scrub_stamp":"2024-09-10T10:26:31.081647+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485314+0000","last_change":"2024-09-10T10:26:46.192891+0000","last_active":"2024-09-10T10:29:05.485314+0000","last_peered":"2024-09-10T10:29:05.485314+0000","last_clean":"2024-09-10T10:29:05.485314+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:29:05.485314+0000","last_undegraded":"2024-09-10T10:29:05.485314+0000","last_fullsized":"2024-09-10T10:29:05.485314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.990210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"78'7","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.213354+0000","last_change":"2024-09-10T10:26:41.789654+0000","last_active":"2024-09-10T10:29:10.213354+0000","last_peered":"2024-09-10T10:29:10.213354+0000","last_clean":"2024-09-10T10:29:10.213354+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:29:10.213354+0000","last_undegraded":"2024-09-10T10:29:10.213354+0000","last_fullsized":"2024-09-10T10:29:10.213354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.921140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'5","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482874+0000","last_change":"2024-09-10T10:26:40.790543+0000","last_active":"2024-09-10T10:29:05.482874+0000","last_peered":"2024-09-10T10:29:05.482874+0000","last_clean":"2024-09-10T10:29:05.482874+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:29:05.482874+0000","last_undegraded":"2024-09-10T10:29:05.482874+0000","last_fullsized":"2024-09-10T10:29:05.482874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484865+0000","last_change":"2024-09-10T10:26:46.193737+0000","last_active":"2024-09-10T10:29:05.484865+0000","last_peered":"2024-09-10T10:29:05.484865+0000","last_clean":"2024-09-10T10:29:05.484865+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:29:05.484865+0000","last_undegraded":"2024-09-10T10:29:05.484865+0000","last_fullsized":"2024-09-10T10:29:05.484865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"78'6","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.214497+0000","last_change":"2024-09-10T10:26:41.792352+0000","last_active":"2024-09-10T10:29:10.214497+0000","last_peered":"2024-09-10T10:29:10.214497+0000","last_clean":"2024-09-10T10:29:10.214497+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:29:10.214497+0000","last_undegraded":"2024-09-10T10:29:10.214497+0000","last_fullsized":"2024-09-10T10:29:10.214497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"28'3","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.688686+0000","last_change":"2024-09-10T10:26:40.793508+0000","last_active":"2024-09-10T10:29:05.688686+0000","last_peered":"2024-09-10T10:29:05.688686+0000","last_clean":"2024-09-10T10:29:05.688686+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:29:05.688686+0000","last_undegraded":"2024-09-10T10:29:05.688686+0000","last_fullsized":"2024-09-10T10:29:05.688686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893170+0000","last_change":"2024-09-10T10:26:45.820085+0000","last_active":"2024-09-10T10:29:05.893170+0000","last_peered":"2024-09-10T10:29:05.893170+0000","last_clean":"2024-09-10T10:29:05.893170+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:29:05.893170+0000","last_undegraded":"2024-09-10T10:29:05.893170+0000","last_fullsized":"2024-09-10T10:29:05.893170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"78'9","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.217865+0000","last_change":"2024-09-10T10:26:41.802317+0000","last_active":"2024-09-10T10:29:10.217865+0000","last_peered":"2024-09-10T10:29:10.217865+0000","last_clean":"2024-09-10T10:29:10.217865+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:29:10.217865+0000","last_undegraded":"2024-09-10T10:29:10.217865+0000","last_fullsized":"2024-09-10T10:29:10.217865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"28'5","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.483086+0000","last_change":"2024-09-10T10:26:40.784614+0000","last_active":"2024-09-10T10:29:05.483086+0000","last_peered":"2024-09-10T10:29:05.483086+0000","last_clean":"2024-09-10T10:29:05.483086+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:29:05.483086+0000","last_undegraded":"2024-09-10T10:29:05.483086+0000","last_fullsized":"2024-09-10T10:29:05.483086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485530+0000","last_change":"2024-09-10T10:26:46.192961+0000","last_active":"2024-09-10T10:29:05.485530+0000","last_peered":"2024-09-10T10:29:05.485530+0000","last_clean":"2024-09-10T10:29:05.485530+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:29:05.485530+0000","last_undegraded":"2024-09-10T10:29:05.485530+0000","last_fullsized":"2024-09-10T10:29:05.485530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:06.330294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"78'5","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.217756+0000","last_change":"2024-09-10T10:26:41.799045+0000","last_active":"2024-09-10T10:29:10.217756+0000","last_peered":"2024-09-10T10:29:10.217756+0000","last_clean":"2024-09-10T10:29:10.217756+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:29:10.217756+0000","last_undegraded":"2024-09-10T10:29:10.217756+0000","last_fullsized":"2024-09-10T10:29:10.217756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:17.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.956264+0000","last_change":"2024-09-10T10:26:40.791876+0000","last_active":"2024-09-10T10:29:05.956264+0000","last_peered":"2024-09-10T10:29:05.956264+0000","last_clean":"2024-09-10T10:29:05.956264+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:29:05.956264+0000","last_undegraded":"2024-09-10T10:29:05.956264+0000","last_fullsized":"2024-09-10T10:29:05.956264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:26.930799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955704+0000","last_change":"2024-09-10T10:26:45.817654+0000","last_active":"2024-09-10T10:29:05.955704+0000","last_peered":"2024-09-10T10:29:05.955704+0000","last_clean":"2024-09-10T10:29:05.955704+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:29:05.955704+0000","last_undegraded":"2024-09-10T10:29:05.955704+0000","last_fullsized":"2024-09-10T10:29:05.955704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:52.116057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"78'6","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.218024+0000","last_change":"2024-09-10T10:26:41.799690+0000","last_active":"2024-09-10T10:29:10.218024+0000","last_peered":"2024-09-10T10:29:10.218024+0000","last_clean":"2024-09-10T10:29:10.218024+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:29:10.218024+0000","last_undegraded":"2024-09-10T10:29:10.218024+0000","last_fullsized":"2024-09-10T10:29:10.218024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'18","reported_seq":148,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482591+0000","last_change":"2024-09-10T10:26:40.789685+0000","last_active":"2024-09-10T10:29:05.482591+0000","last_peered":"2024-09-10T10:29:05.482591+0000","last_clean":"2024-09-10T10:29:05.482591+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:29:05.482591+0000","last_undegraded":"2024-09-10T10:29:05.482591+0000","last_fullsized":"2024-09-10T10:29:05.482591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":120,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:13.169031+0000","last_change":"2024-09-10T10:26:45.821894+0000","last_active":"2024-09-10T10:29:13.169031+0000","last_peered":"2024-09-10T10:29:13.169031+0000","last_clean":"2024-09-10T10:29:13.169031+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:29:13.169031+0000","last_undegraded":"2024-09-10T10:29:13.169031+0000","last_fullsized":"2024-09-10T10:29:13.169031+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"78'6","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.207781+0000","last_change":"2024-09-10T10:26:41.800681+0000","last_active":"2024-09-10T10:29:10.207781+0000","last_peered":"2024-09-10T10:29:10.207781+0000","last_clean":"2024-09-10T10:29:10.207781+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:29:10.207781+0000","last_undegraded":"2024-09-10T10:29:10.207781+0000","last_fullsized":"2024-09-10T10:29:10.207781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'5","reported_seq":160,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.484912+0000","last_change":"2024-09-10T10:28:59.453146+0000","last_active":"2024-09-10T10:29:05.484912+0000","last_peered":"2024-09-10T10:29:05.484912+0000","last_clean":"2024-09-10T10:29:05.484912+0000","last_became_active":"2024-09-10T10:28:59.446453+0000","last_became_peered":"2024-09-10T10:28:59.446453+0000","last_unstale":"2024-09-10T10:29:05.484912+0000","last_undegraded":"2024-09-10T10:29:05.484912+0000","last_fullsized":"2024-09-10T10:29:05.484912+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:31.347510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1176,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832225+0000","last_change":"2024-09-10T10:26:46.192436+0000","last_active":"2024-09-10T10:29:05.832225+0000","last_peered":"2024-09-10T10:29:05.832225+0000","last_clean":"2024-09-10T10:29:05.832225+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:29:05.832225+0000","last_undegraded":"2024-09-10T10:29:05.832225+0000","last_fullsized":"2024-09-10T10:29:05.832225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:26.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"78'15","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.218573+0000","last_change":"2024-09-10T10:26:41.799647+0000","last_active":"2024-09-10T10:29:10.218573+0000","last_peered":"2024-09-10T10:29:10.218573+0000","last_clean":"2024-09-10T10:29:10.218573+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:29:10.218573+0000","last_undegraded":"2024-09-10T10:29:10.218573+0000","last_fullsized":"2024-09-10T10:29:10.218573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:01.798141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"60'6","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486347+0000","last_change":"2024-09-10T10:26:40.794631+0000","last_active":"2024-09-10T10:29:05.486347+0000","last_peered":"2024-09-10T10:29:05.486347+0000","last_clean":"2024-09-10T10:29:05.486347+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:29:05.486347+0000","last_undegraded":"2024-09-10T10:29:05.486347+0000","last_fullsized":"2024-09-10T10:29:05.486347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832226+0000","last_change":"2024-09-10T10:26:46.192989+0000","last_active":"2024-09-10T10:29:05.832226+0000","last_peered":"2024-09-10T10:29:05.832226+0000","last_clean":"2024-09-10T10:29:05.832226+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:29:05.832226+0000","last_undegraded":"2024-09-10T10:29:05.832226+0000","last_fullsized":"2024-09-10T10:29:05.832226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:46.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"78'9","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.216964+0000","last_change":"2024-09-10T10:26:41.794745+0000","last_active":"2024-09-10T10:29:10.216964+0000","last_peered":"2024-09-10T10:29:10.216964+0000","last_clean":"2024-09-10T10:29:10.216964+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:29:10.216964+0000","last_undegraded":"2024-09-10T10:29:10.216964+0000","last_fullsized":"2024-09-10T10:29:10.216964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'9","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893394+0000","last_change":"2024-09-10T10:26:40.780913+0000","last_active":"2024-09-10T10:29:05.893394+0000","last_peered":"2024-09-10T10:29:05.893394+0000","last_clean":"2024-09-10T10:29:05.893394+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:29:05.893394+0000","last_undegraded":"2024-09-10T10:29:05.893394+0000","last_fullsized":"2024-09-10T10:29:05.893394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486646+0000","last_change":"2024-09-10T10:26:46.192600+0000","last_active":"2024-09-10T10:29:05.486646+0000","last_peered":"2024-09-10T10:29:05.486646+0000","last_clean":"2024-09-10T10:29:05.486646+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:29:05.486646+0000","last_undegraded":"2024-09-10T10:29:05.486646+0000","last_fullsized":"2024-09-10T10:29:05.486646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:54.808588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"78'6","reported_seq":129,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.212309+0000","last_change":"2024-09-10T10:26:41.800474+0000","last_active":"2024-09-10T10:29:10.212309+0000","last_peered":"2024-09-10T10:29:10.212309+0000","last_clean":"2024-09-10T10:29:10.212309+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:29:10.212309+0000","last_undegraded":"2024-09-10T10:29:10.212309+0000","last_fullsized":"2024-09-10T10:29:10.212309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:27.529369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"78'30","reported_seq":159,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.173362+0000","last_change":"2024-09-10T10:26:40.793668+0000","last_active":"2024-09-10T10:29:10.173362+0000","last_peered":"2024-09-10T10:29:10.173362+0000","last_clean":"2024-09-10T10:29:10.173362+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:29:10.173362+0000","last_undegraded":"2024-09-10T10:29:10.173362+0000","last_fullsized":"2024-09-10T10:29:10.173362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:45.918126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37263,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"60'3","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:09.124332+0000","last_change":"2024-09-10T10:26:45.819788+0000","last_active":"2024-09-10T10:29:09.124332+0000","last_peered":"2024-09-10T10:29:09.124332+0000","last_clean":"2024-09-10T10:29:09.124332+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:29:09.124332+0000","last_undegraded":"2024-09-10T10:29:09.124332+0000","last_fullsized":"2024-09-10T10:29:09.124332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"78'11","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.213543+0000","last_change":"2024-09-10T10:26:41.792093+0000","last_active":"2024-09-10T10:29:10.213543+0000","last_peered":"2024-09-10T10:29:10.213543+0000","last_clean":"2024-09-10T10:29:10.213543+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:29:10.213543+0000","last_undegraded":"2024-09-10T10:29:10.213543+0000","last_fullsized":"2024-09-10T10:29:10.213543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:34.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'8","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:06.309194+0000","last_change":"2024-09-10T10:26:40.791933+0000","last_active":"2024-09-10T10:29:06.309194+0000","last_peered":"2024-09-10T10:29:06.309194+0000","last_clean":"2024-09-10T10:29:06.309194+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:29:06.309194+0000","last_undegraded":"2024-09-10T10:29:06.309194+0000","last_fullsized":"2024-09-10T10:29:06.309194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.955996+0000","last_change":"2024-09-10T10:26:45.813673+0000","last_active":"2024-09-10T10:29:05.955996+0000","last_peered":"2024-09-10T10:29:05.955996+0000","last_clean":"2024-09-10T10:29:05.955996+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:29:05.955996+0000","last_undegraded":"2024-09-10T10:29:05.955996+0000","last_fullsized":"2024-09-10T10:29:05.955996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:18.352100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"78'9","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.206514+0000","last_change":"2024-09-10T10:26:41.795607+0000","last_active":"2024-09-10T10:29:10.206514+0000","last_peered":"2024-09-10T10:29:10.206514+0000","last_clean":"2024-09-10T10:29:10.206514+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:29:10.206514+0000","last_undegraded":"2024-09-10T10:29:10.206514+0000","last_fullsized":"2024-09-10T10:29:10.206514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'4","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.486525+0000","last_change":"2024-09-10T10:26:40.792723+0000","last_active":"2024-09-10T10:29:05.486525+0000","last_peered":"2024-09-10T10:29:05.486525+0000","last_clean":"2024-09-10T10:29:05.486525+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:29:05.486525+0000","last_undegraded":"2024-09-10T10:29:05.486525+0000","last_fullsized":"2024-09-10T10:29:05.486525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831619+0000","last_change":"2024-09-10T10:28:24.248827+0000","last_active":"2024-09-10T10:29:05.831619+0000","last_peered":"2024-09-10T10:29:05.831619+0000","last_clean":"2024-09-10T10:29:05.831619+0000","last_became_active":"2024-09-10T10:28:24.248395+0000","last_became_peered":"2024-09-10T10:28:24.248395+0000","last_unstale":"2024-09-10T10:29:05.831619+0000","last_undegraded":"2024-09-10T10:29:05.831619+0000","last_fullsized":"2024-09-10T10:29:05.831619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:05.306669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"78'10","reported_seq":117,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.208775+0000","last_change":"2024-09-10T10:27:24.087925+0000","last_active":"2024-09-10T10:29:10.208775+0000","last_peered":"2024-09-10T10:29:10.208775+0000","last_clean":"2024-09-10T10:29:10.208775+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:29:10.208775+0000","last_undegraded":"2024-09-10T10:29:10.208775+0000","last_fullsized":"2024-09-10T10:29:10.208775+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"78'7","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:09.124426+0000","last_change":"2024-09-10T10:26:40.793104+0000","last_active":"2024-09-10T10:29:09.124426+0000","last_peered":"2024-09-10T10:29:09.124426+0000","last_clean":"2024-09-10T10:29:09.124426+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:29:09.124426+0000","last_undegraded":"2024-09-10T10:29:09.124426+0000","last_fullsized":"2024-09-10T10:29:09.124426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:09.648224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.831826+0000","last_change":"2024-09-10T10:26:46.193602+0000","last_active":"2024-09-10T10:29:05.831826+0000","last_peered":"2024-09-10T10:29:05.831826+0000","last_clean":"2024-09-10T10:29:05.831826+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:29:05.831826+0000","last_undegraded":"2024-09-10T10:29:05.831826+0000","last_fullsized":"2024-09-10T10:29:05.831826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:36.167747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.893217+0000","last_change":"2024-09-10T10:28:24.376473+0000","last_active":"2024-09-10T10:29:05.893217+0000","last_peered":"2024-09-10T10:29:05.893217+0000","last_clean":"2024-09-10T10:29:05.893217+0000","last_became_active":"2024-09-10T10:28:24.376079+0000","last_became_peered":"2024-09-10T10:28:24.376079+0000","last_unstale":"2024-09-10T10:29:05.893217+0000","last_undegraded":"2024-09-10T10:29:05.893217+0000","last_fullsized":"2024-09-10T10:29:05.893217+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,1],"acting":[0,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"78'10","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.198179+0000","last_change":"2024-09-10T10:26:41.796056+0000","last_active":"2024-09-10T10:29:10.198179+0000","last_peered":"2024-09-10T10:29:10.198179+0000","last_clean":"2024-09-10T10:29:10.198179+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:29:10.198179+0000","last_undegraded":"2024-09-10T10:29:10.198179+0000","last_fullsized":"2024-09-10T10:29:10.198179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'3","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485970+0000","last_change":"2024-09-10T10:26:40.783562+0000","last_active":"2024-09-10T10:29:05.485970+0000","last_peered":"2024-09-10T10:29:05.485970+0000","last_clean":"2024-09-10T10:29:05.485970+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:29:05.485970+0000","last_undegraded":"2024-09-10T10:29:05.485970+0000","last_fullsized":"2024-09-10T10:29:05.485970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:56.727263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485377+0000","last_change":"2024-09-10T10:26:45.819028+0000","last_active":"2024-09-10T10:29:05.485377+0000","last_peered":"2024-09-10T10:29:05.485377+0000","last_clean":"2024-09-10T10:29:05.485377+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:29:05.485377+0000","last_undegraded":"2024-09-10T10:29:05.485377+0000","last_fullsized":"2024-09-10T10:29:05.485377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:44.374452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"78'8","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.206897+0000","last_change":"2024-09-10T10:26:41.800438+0000","last_active":"2024-09-10T10:29:10.206897+0000","last_peered":"2024-09-10T10:29:10.206897+0000","last_clean":"2024-09-10T10:29:10.206897+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:29:10.206897+0000","last_undegraded":"2024-09-10T10:29:10.206897+0000","last_fullsized":"2024-09-10T10:29:10.206897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:40.409386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'3","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832298+0000","last_change":"2024-09-10T10:26:40.792793+0000","last_active":"2024-09-10T10:29:05.832298+0000","last_peered":"2024-09-10T10:29:05.832298+0000","last_clean":"2024-09-10T10:29:05.832298+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:29:05.832298+0000","last_undegraded":"2024-09-10T10:29:05.832298+0000","last_fullsized":"2024-09-10T10:29:05.832298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:30.905942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'7","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.485242+0000","last_change":"2024-09-10T10:26:40.791891+0000","last_active":"2024-09-10T10:29:05.485242+0000","last_peered":"2024-09-10T10:29:05.485242+0000","last_clean":"2024-09-10T10:29:05.485242+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:29:05.485242+0000","last_undegraded":"2024-09-10T10:29:05.485242+0000","last_fullsized":"2024-09-10T10:29:05.485242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:55.562853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"78'8","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:13.168887+0000","last_change":"2024-09-10T10:26:41.798645+0000","last_active":"2024-09-10T10:29:13.168887+0000","last_peered":"2024-09-10T10:29:13.168887+0000","last_clean":"2024-09-10T10:29:13.168887+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:29:13.168887+0000","last_undegraded":"2024-09-10T10:29:13.168887+0000","last_fullsized":"2024-09-10T10:29:13.168887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.482692+0000","last_change":"2024-09-10T10:26:45.810200+0000","last_active":"2024-09-10T10:29:05.482692+0000","last_peered":"2024-09-10T10:29:05.482692+0000","last_clean":"2024-09-10T10:29:05.482692+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:29:05.482692+0000","last_undegraded":"2024-09-10T10:29:05.482692+0000","last_fullsized":"2024-09-10T10:29:05.482692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'7","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.169354+0000","last_change":"2024-09-10T10:26:40.788665+0000","last_active":"2024-09-10T10:29:10.169354+0000","last_peered":"2024-09-10T10:29:10.169354+0000","last_clean":"2024-09-10T10:29:10.169354+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:29:10.169354+0000","last_undegraded":"2024-09-10T10:29:10.169354+0000","last_fullsized":"2024-09-10T10:29:10.169354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:39.762559+0000","last_clean_scrub_stamp":"2024-09-10T10:26:39.762559+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:32.037713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"78'7","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.196643+0000","last_change":"2024-09-10T10:26:41.795619+0000","last_active":"2024-09-10T10:29:10.196643+0000","last_peered":"2024-09-10T10:29:10.196643+0000","last_clean":"2024-09-10T10:29:10.196643+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:29:10.196643+0000","last_undegraded":"2024-09-10T10:29:10.196643+0000","last_fullsized":"2024-09-10T10:29:10.196643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:57.732219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:05.832152+0000","last_change":"2024-09-10T10:26:45.820487+0000","last_active":"2024-09-10T10:29:05.832152+0000","last_peered":"2024-09-10T10:29:05.832152+0000","last_clean":"2024-09-10T10:29:05.832152+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:29:05.832152+0000","last_undegraded":"2024-09-10T10:29:05.832152+0000","last_fullsized":"2024-09-10T10:29:05.832152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:44.794211+0000","last_clean_scrub_stamp":"2024-09-10T10:26:44.794211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:43.134060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"78'9","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.209267+0000","last_change":"2024-09-10T10:26:41.801079+0000","last_active":"2024-09-10T10:29:10.209267+0000","last_peered":"2024-09-10T10:29:10.209267+0000","last_clean":"2024-09-10T10:29:10.209267+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:29:10.209267+0000","last_undegraded":"2024-09-10T10:29:10.209267+0000","last_fullsized":"2024-09-10T10:29:10.209267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:27:31.040346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"78'12","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.216778+0000","last_change":"2024-09-10T10:26:41.793861+0000","last_active":"2024-09-10T10:29:10.216778+0000","last_peered":"2024-09-10T10:29:10.216778+0000","last_clean":"2024-09-10T10:29:10.216778+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:29:10.216778+0000","last_undegraded":"2024-09-10T10:29:10.216778+0000","last_fullsized":"2024-09-10T10:29:10.216778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"78'8","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.212626+0000","last_change":"2024-09-10T10:26:41.794762+0000","last_active":"2024-09-10T10:29:10.212626+0000","last_peered":"2024-09-10T10:29:10.212626+0000","last_clean":"2024-09-10T10:29:10.212626+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:29:10.212626+0000","last_undegraded":"2024-09-10T10:29:10.212626+0000","last_fullsized":"2024-09-10T10:29:10.212626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:48.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"78'7","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.208801+0000","last_change":"2024-09-10T10:26:41.802015+0000","last_active":"2024-09-10T10:29:10.208801+0000","last_peered":"2024-09-10T10:29:10.208801+0000","last_clean":"2024-09-10T10:29:10.208801+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:29:10.208801+0000","last_undegraded":"2024-09-10T10:29:10.208801+0000","last_fullsized":"2024-09-10T10:29:10.208801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"78'4","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.209949+0000","last_change":"2024-09-10T10:26:41.802897+0000","last_active":"2024-09-10T10:29:10.209949+0000","last_peered":"2024-09-10T10:29:10.209949+0000","last_clean":"2024-09-10T10:29:10.209949+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:29:10.209949+0000","last_undegraded":"2024-09-10T10:29:10.209949+0000","last_fullsized":"2024-09-10T10:29:10.209949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"78'6","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:13.641318+0000","last_change":"2024-09-10T10:29:13.641363+0000","last_active":"2024-09-10T10:29:13.641318+0000","last_peered":"2024-09-10T10:29:13.641318+0000","last_clean":"2024-09-10T10:29:13.641318+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:29:13.641318+0000","last_undegraded":"2024-09-10T10:29:13.641318+0000","last_fullsized":"2024-09-10T10:29:13.641318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:13.641240+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:13.641240+0000","last_clean_scrub_stamp":"2024-09-10T10:29:13.641240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:32.473985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"78'15","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.217361+0000","last_change":"2024-09-10T10:26:41.795874+0000","last_active":"2024-09-10T10:29:10.217361+0000","last_peered":"2024-09-10T10:29:10.217361+0000","last_clean":"2024-09-10T10:29:10.217361+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:29:10.217361+0000","last_undegraded":"2024-09-10T10:29:10.217361+0000","last_fullsized":"2024-09-10T10:29:10.217361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"78'2","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.201741+0000","last_change":"2024-09-10T10:28:59.446917+0000","last_active":"2024-09-10T10:29:10.201741+0000","last_peered":"2024-09-10T10:29:10.201741+0000","last_clean":"2024-09-10T10:29:10.201741+0000","last_became_active":"2024-09-10T10:28:59.446546+0000","last_became_peered":"2024-09-10T10:28:59.446546+0000","last_unstale":"2024-09-10T10:29:10.201741+0000","last_undegraded":"2024-09-10T10:29:10.201741+0000","last_fullsized":"2024-09-10T10:29:10.201741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:30.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"78'6","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.215226+0000","last_change":"2024-09-10T10:27:23.958304+0000","last_active":"2024-09-10T10:29:10.215226+0000","last_peered":"2024-09-10T10:29:10.215226+0000","last_clean":"2024-09-10T10:29:10.215226+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:29:10.215226+0000","last_undegraded":"2024-09-10T10:29:10.215226+0000","last_fullsized":"2024-09-10T10:29:10.215226+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:45.176181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"78'8","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.209871+0000","last_change":"2024-09-10T10:26:41.802977+0000","last_active":"2024-09-10T10:29:10.209871+0000","last_peered":"2024-09-10T10:29:10.209871+0000","last_clean":"2024-09-10T10:29:10.209871+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:29:10.209871+0000","last_undegraded":"2024-09-10T10:29:10.209871+0000","last_fullsized":"2024-09-10T10:29:10.209871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"78'10","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.217594+0000","last_change":"2024-09-10T10:26:41.790139+0000","last_active":"2024-09-10T10:29:10.217594+0000","last_peered":"2024-09-10T10:29:10.217594+0000","last_clean":"2024-09-10T10:29:10.217594+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:29:10.217594+0000","last_undegraded":"2024-09-10T10:29:10.217594+0000","last_fullsized":"2024-09-10T10:29:10.217594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"78'11","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.216957+0000","last_change":"2024-09-10T10:26:41.794915+0000","last_active":"2024-09-10T10:29:10.216957+0000","last_peered":"2024-09-10T10:29:10.216957+0000","last_clean":"2024-09-10T10:29:10.216957+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:29:10.216957+0000","last_undegraded":"2024-09-10T10:29:10.216957+0000","last_fullsized":"2024-09-10T10:29:10.216957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"78'8","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.208520+0000","last_change":"2024-09-10T10:28:23.249392+0000","last_active":"2024-09-10T10:29:10.208520+0000","last_peered":"2024-09-10T10:29:10.208520+0000","last_clean":"2024-09-10T10:29:10.208520+0000","last_became_active":"2024-09-10T10:28:23.249039+0000","last_became_peered":"2024-09-10T10:28:23.249039+0000","last_unstale":"2024-09-10T10:29:10.208520+0000","last_undegraded":"2024-09-10T10:29:10.208520+0000","last_fullsized":"2024-09-10T10:29:10.208520+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"78'7","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.199601+0000","last_change":"2024-09-10T10:28:24.244885+0000","last_active":"2024-09-10T10:29:10.199601+0000","last_peered":"2024-09-10T10:29:10.199601+0000","last_clean":"2024-09-10T10:29:10.199601+0000","last_became_active":"2024-09-10T10:28:24.244560+0000","last_became_peered":"2024-09-10T10:28:24.244560+0000","last_unstale":"2024-09-10T10:29:10.199601+0000","last_undegraded":"2024-09-10T10:29:10.199601+0000","last_fullsized":"2024-09-10T10:29:10.199601+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:36.642112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2e","version":"78'10","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.200534+0000","last_change":"2024-09-10T10:26:41.790240+0000","last_active":"2024-09-10T10:29:10.200534+0000","last_peered":"2024-09-10T10:29:10.200534+0000","last_clean":"2024-09-10T10:29:10.200534+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:29:10.200534+0000","last_undegraded":"2024-09-10T10:29:10.200534+0000","last_fullsized":"2024-09-10T10:29:10.200534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"78'8","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.213960+0000","last_change":"2024-09-10T10:26:41.793501+0000","last_active":"2024-09-10T10:29:10.213960+0000","last_peered":"2024-09-10T10:29:10.213960+0000","last_clean":"2024-09-10T10:29:10.213960+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:29:10.213960+0000","last_undegraded":"2024-09-10T10:29:10.213960+0000","last_fullsized":"2024-09-10T10:29:10.213960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:07.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"78'9","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.214882+0000","last_change":"2024-09-10T10:26:41.794759+0000","last_active":"2024-09-10T10:29:10.214882+0000","last_peered":"2024-09-10T10:29:10.214882+0000","last_clean":"2024-09-10T10:29:10.214882+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:29:10.214882+0000","last_undegraded":"2024-09-10T10:29:10.214882+0000","last_fullsized":"2024-09-10T10:29:10.214882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"78'7","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.200868+0000","last_change":"2024-09-10T10:26:41.782139+0000","last_active":"2024-09-10T10:29:10.200868+0000","last_peered":"2024-09-10T10:29:10.200868+0000","last_clean":"2024-09-10T10:29:10.200868+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:29:10.200868+0000","last_undegraded":"2024-09-10T10:29:10.200868+0000","last_fullsized":"2024-09-10T10:29:10.200868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"78'8","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:13.950124+0000","last_change":"2024-09-10T10:29:13.950147+0000","last_active":"2024-09-10T10:29:13.950124+0000","last_peered":"2024-09-10T10:29:13.950124+0000","last_clean":"2024-09-10T10:29:13.950124+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:29:13.950124+0000","last_undegraded":"2024-09-10T10:29:13.950124+0000","last_fullsized":"2024-09-10T10:29:13.950124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:13.950052+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:13.950052+0000","last_clean_scrub_stamp":"2024-09-10T10:29:13.950052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:50.172624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"78'10","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.197774+0000","last_change":"2024-09-10T10:28:24.243479+0000","last_active":"2024-09-10T10:29:10.197774+0000","last_peered":"2024-09-10T10:29:10.197774+0000","last_clean":"2024-09-10T10:29:10.197774+0000","last_became_active":"2024-09-10T10:28:24.243088+0000","last_became_peered":"2024-09-10T10:28:24.243088+0000","last_unstale":"2024-09-10T10:29:10.197774+0000","last_undegraded":"2024-09-10T10:29:10.197774+0000","last_fullsized":"2024-09-10T10:29:10.197774+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"78'7","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.208145+0000","last_change":"2024-09-10T10:27:58.327792+0000","last_active":"2024-09-10T10:29:10.208145+0000","last_peered":"2024-09-10T10:29:10.208145+0000","last_clean":"2024-09-10T10:29:10.208145+0000","last_became_active":"2024-09-10T10:27:58.327502+0000","last_became_peered":"2024-09-10T10:27:58.327502+0000","last_unstale":"2024-09-10T10:29:10.208145+0000","last_undegraded":"2024-09-10T10:29:10.208145+0000","last_fullsized":"2024-09-10T10:29:10.208145+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.35","version":"78'6","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.202072+0000","last_change":"2024-09-10T10:26:41.794699+0000","last_active":"2024-09-10T10:29:10.202072+0000","last_peered":"2024-09-10T10:29:10.202072+0000","last_clean":"2024-09-10T10:29:10.202072+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:29:10.202072+0000","last_undegraded":"2024-09-10T10:29:10.202072+0000","last_fullsized":"2024-09-10T10:29:10.202072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:16.875672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"78'16","reported_seq":153,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.212337+0000","last_change":"2024-09-10T10:28:40.345087+0000","last_active":"2024-09-10T10:29:10.212337+0000","last_peered":"2024-09-10T10:29:10.212337+0000","last_clean":"2024-09-10T10:29:10.212337+0000","last_became_active":"2024-09-10T10:28:40.344766+0000","last_became_peered":"2024-09-10T10:28:40.344766+0000","last_unstale":"2024-09-10T10:29:10.212337+0000","last_undegraded":"2024-09-10T10:29:10.212337+0000","last_fullsized":"2024-09-10T10:29:10.212337+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:44.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"78'11","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.214575+0000","last_change":"2024-09-10T10:26:41.797333+0000","last_active":"2024-09-10T10:29:10.214575+0000","last_peered":"2024-09-10T10:29:10.214575+0000","last_clean":"2024-09-10T10:29:10.214575+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:29:10.214575+0000","last_undegraded":"2024-09-10T10:29:10.214575+0000","last_fullsized":"2024-09-10T10:29:10.214575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:49.204371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"78'9","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.212577+0000","last_change":"2024-09-10T10:26:41.801910+0000","last_active":"2024-09-10T10:29:10.212577+0000","last_peered":"2024-09-10T10:29:10.212577+0000","last_clean":"2024-09-10T10:29:10.212577+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:29:10.212577+0000","last_undegraded":"2024-09-10T10:29:10.212577+0000","last_fullsized":"2024-09-10T10:29:10.212577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:27.128488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"78'5","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.216975+0000","last_change":"2024-09-10T10:26:41.790547+0000","last_active":"2024-09-10T10:29:10.216975+0000","last_peered":"2024-09-10T10:29:10.216975+0000","last_clean":"2024-09-10T10:29:10.216975+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:29:10.216975+0000","last_undegraded":"2024-09-10T10:29:10.216975+0000","last_fullsized":"2024-09-10T10:29:10.216975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:33.534632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"78'6","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.215318+0000","last_change":"2024-09-10T10:26:41.787928+0000","last_active":"2024-09-10T10:29:10.215318+0000","last_peered":"2024-09-10T10:29:10.215318+0000","last_clean":"2024-09-10T10:29:10.215318+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:29:10.215318+0000","last_undegraded":"2024-09-10T10:29:10.215318+0000","last_fullsized":"2024-09-10T10:29:10.215318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:39.200452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"78'4","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.216986+0000","last_change":"2024-09-10T10:26:41.793291+0000","last_active":"2024-09-10T10:29:10.216986+0000","last_peered":"2024-09-10T10:29:10.216986+0000","last_clean":"2024-09-10T10:29:10.216986+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:29:10.216986+0000","last_undegraded":"2024-09-10T10:29:10.216986+0000","last_fullsized":"2024-09-10T10:29:10.216986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:22.392825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"78'8","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.216304+0000","last_change":"2024-09-10T10:26:41.793084+0000","last_active":"2024-09-10T10:29:10.216304+0000","last_peered":"2024-09-10T10:29:10.216304+0000","last_clean":"2024-09-10T10:29:10.216304+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:29:10.216304+0000","last_undegraded":"2024-09-10T10:29:10.216304+0000","last_fullsized":"2024-09-10T10:29:10.216304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:03.529815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"78'7","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.207936+0000","last_change":"2024-09-10T10:26:41.792658+0000","last_active":"2024-09-10T10:29:10.207936+0000","last_peered":"2024-09-10T10:29:10.207936+0000","last_clean":"2024-09-10T10:29:10.207936+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:29:10.207936+0000","last_undegraded":"2024-09-10T10:29:10.207936+0000","last_fullsized":"2024-09-10T10:29:10.207936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"78'4","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.200280+0000","last_change":"2024-09-10T10:26:41.789735+0000","last_active":"2024-09-10T10:29:10.200280+0000","last_peered":"2024-09-10T10:29:10.200280+0000","last_clean":"2024-09-10T10:29:10.200280+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:29:10.200280+0000","last_undegraded":"2024-09-10T10:29:10.200280+0000","last_fullsized":"2024-09-10T10:29:10.200280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"78'8","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:10.210357+0000","last_change":"2024-09-10T10:26:41.798291+0000","last_active":"2024-09-10T10:29:10.210357+0000","last_peered":"2024-09-10T10:29:10.210357+0000","last_clean":"2024-09-10T10:29:10.210357+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:29:10.210357+0000","last_undegraded":"2024-09-10T10:29:10.210357+0000","last_fullsized":"2024-09-10T10:29:10.210357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:26:40.767514+0000","last_clean_scrub_stamp":"2024-09-10T10:26:40.767514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:58.927522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":501,"ondisk_log_size":501,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44494,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":256,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":88988,"data_compressed":3324,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":1074512,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":974848,"data_stored":519700,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":485579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":482977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":520543,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":954368,"data_stored":483521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":483179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":1072889,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":38896,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":38053,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:29:16.467 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.468 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.468 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.468 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.468 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.468 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.468 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.468 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.469 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.469 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.469 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.469 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.469 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.469 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.469 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.470 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.470 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.470 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.470 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.470 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.470 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.470 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.470 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.470 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.471 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.471 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.471 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.471 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.471 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.471 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.471 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.471 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.471 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.472 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.472 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.472 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.472 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.472 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.472 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.472 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.473 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.473 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.473 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.473 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.473 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.473 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.473 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.473 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T10:26:31.081647+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=31, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.473 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.473 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.474 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.474 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.474 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.474 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.474 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.474 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.474 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.474 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.474 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.475 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.475 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.475 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.475 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.475 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.475 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.475 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.475 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.476 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.476 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.476 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.476 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.477 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.477 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.477 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.477 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.477 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.477 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.477 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.477 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.478 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.478 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.478 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.478 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.478 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.478 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.478 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.478 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.478 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.479 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.479 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.479 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.479 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.479 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.479 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.479 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-10T10:26:39.762559+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=39, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.479 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.479 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-10T10:26:44.794211+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.480 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.480 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.480 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.480 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.480 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.480 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.480 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.480 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.481 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.481 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.481 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.481 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.481 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.481 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.481 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.481 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.482 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.482 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.482 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.482 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.482 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.482 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.482 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.482 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.482 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.483 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.483 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.483 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.483 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.483 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-10T10:26:40.767514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=26, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=29, tm_sec=11, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:29:16.483 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T10:29:36.485 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:29:36.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.594+0000 7f5cf77b0640 1 Processor -- start 2024-09-10T10:29:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.594+0000 7f5cf77b0640 1 -- start start 2024-09-10T10:29:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.595+0000 7f5cf77b0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5cf0154820 0x7f5cf0072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.595+0000 7f5cf77b0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 0x7f5cf00c8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.595+0000 7f5cf77b0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cf00730f0 con 0x7f5cf015b470 2024-09-10T10:29:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.595+0000 7f5cf77b0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5cf006bc10 con 0x7f5cf0154820 2024-09-10T10:29:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.595+0000 7f5cf77b0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf00c7300 con 0x7f5cf0154c00 2024-09-10T10:29:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf5d4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 0x7f5cf00c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf554c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5cf0154820 0x7f5cf0072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf4d4b640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f5cf015b470 0x7f5cf00c7d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.120:55040/0 (socket says 172.21.15.120:55040) 2024-09-10T10:29:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf554c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5cf0154820 0x7f5cf0072ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.120:47006/0 (socket says 172.21.15.120:47006) 2024-09-10T10:29:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf5d4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 0x7f5cf00c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.120:39860/0 (socket says 172.21.15.120:39860) 2024-09-10T10:29:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf4d4b640 1 -- 172.21.15.120:0/4199498390 learned_addr learned my addr 172.21.15.120:0/4199498390 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf5d4d640 1 -- 172.21.15.120:0/4199498390 >> v2:172.21.15.134:3300/0 conn(0x7f5cf0154820 msgr2=0x7f5cf0072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf5d4d640 1 --2- 172.21.15.120:0/4199498390 >> v2:172.21.15.134:3300/0 conn(0x7f5cf0154820 0x7f5cf0072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf5d4d640 1 -- 172.21.15.120:0/4199498390 >> v1:172.21.15.134:6789/0 conn(0x7f5cf015b470 legacy=0x7f5cf00c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf5d4d640 1 -- 172.21.15.120:0/4199498390 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf0073390 con 0x7f5cf0154c00 2024-09-10T10:29:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5ce7fff640 1 -- 172.21.15.120:0/4199498390 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1032512223 0 0) 0x7f5cf00730f0 con 0x7f5cf015b470 2024-09-10T10:29:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.596+0000 7f5cf554c640 1 --2- 172.21.15.120:0/4199498390 >> v2:172.21.15.134:3300/0 conn(0x7f5cf0154820 0x7f5cf0072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:29:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.597+0000 7f5cf5d4d640 1 --2- 172.21.15.120:0/4199498390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 0x7f5cf00c8790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009d90 tx=0x7f5ce802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3523cc36b06d77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.597+0000 7f5ce7fff640 1 -- 172.21.15.120:0/4199498390 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8007740 con 0x7f5cf0154c00 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.597+0000 7f5ce7fff640 1 -- 172.21.15.120:0/4199498390 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce80078e0 con 0x7f5cf0154c00 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.597+0000 7f5ce7fff640 1 -- 172.21.15.120:0/4199498390 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8007cb0 con 0x7f5cf0154c00 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.598+0000 7f5cf77b0640 1 -- 172.21.15.120:0/4199498390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 msgr2=0x7f5cf00c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.598+0000 7f5cf77b0640 1 --2- 172.21.15.120:0/4199498390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 0x7f5cf00c8790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009d90 tx=0x7f5ce802a6f0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.598+0000 7f5cf77b0640 1 -- 172.21.15.120:0/4199498390 shutdown_connections 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.598+0000 7f5cf77b0640 1 --2- 172.21.15.120:0/4199498390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 0x7f5cf00c8790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.598+0000 7f5cf77b0640 1 --2- 172.21.15.120:0/4199498390 >> v2:172.21.15.134:3300/0 conn(0x7f5cf0154820 0x7f5cf0072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.598+0000 7f5cf77b0640 1 -- 172.21.15.120:0/4199498390 >> 172.21.15.120:0/4199498390 conn(0x7f5cf0070290 msgr2=0x7f5cf00c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.598+0000 7f5cf77b0640 1 -- 172.21.15.120:0/4199498390 shutdown_connections 2024-09-10T10:29:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.598+0000 7f5cf77b0640 1 -- 172.21.15.120:0/4199498390 wait complete. 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.599+0000 7f5cf77b0640 1 Processor -- start 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.599+0000 7f5cf77b0640 1 -- start start 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf77b0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cf0154820 0x7f5cf00cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf77b0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 0x7f5cf00c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf77b0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf015b470 0x7f5cf01f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf77b0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf00c7550 con 0x7f5cf015b470 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf77b0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf00c78c0 con 0x7f5cf0154820 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf77b0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf00c44f0 con 0x7f5cf0154c00 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf5d4d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf015b470 0x7f5cf01f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf5d4d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf015b470 0x7f5cf01f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39112/0 (socket says 172.21.15.120:39112) 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf5d4d640 1 -- 172.21.15.120:0/2026769622 learned_addr learned my addr 172.21.15.120:0/2026769622 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf554c640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cf0154820 0x7f5cf00cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf4d4b640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 0x7f5cf00c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf5d4d640 1 -- 172.21.15.120:0/2026769622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 msgr2=0x7f5cf00c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf5d4d640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 0x7f5cf00c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf5d4d640 1 -- 172.21.15.120:0/2026769622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cf0154820 msgr2=0x7f5cf00cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf5d4d640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cf0154820 0x7f5cf00cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf5d4d640 1 -- 172.21.15.120:0/2026769622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf00cb230 con 0x7f5cf015b470 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5cf5d4d640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf015b470 0x7f5cf01f0660 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009d90 tx=0x7f5ce8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5ce5ffb640 1 -- 172.21.15.120:0/2026769622 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8040070 con 0x7f5cf015b470 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.600+0000 7f5ce5ffb640 1 -- 172.21.15.120:0/2026769622 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce8031c40 con 0x7f5cf015b470 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.601+0000 7f5ce5ffb640 1 -- 172.21.15.120:0/2026769622 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8009450 con 0x7f5cf015b470 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.601+0000 7f5cf77b0640 1 -- 172.21.15.120:0/2026769622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf01f3b70 con 0x7f5cf015b470 2024-09-10T10:29:36.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.601+0000 7f5cf77b0640 1 -- 172.21.15.120:0/2026769622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf01f4130 con 0x7f5cf015b470 2024-09-10T10:29:36.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.602+0000 7f5cf77b0640 1 -- 172.21.15.120:0/2026769622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf015b850 con 0x7f5cf015b470 2024-09-10T10:29:36.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.602+0000 7f5ce5ffb640 1 -- 172.21.15.120:0/2026769622 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ce8039040 con 0x7f5cf015b470 2024-09-10T10:29:36.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.603+0000 7f5ce5ffb640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5cd406be30 0x7f5cd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.603+0000 7f5ce5ffb640 1 -- 172.21.15.120:0/2026769622 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7f5ce80ac150 con 0x7f5cf015b470 2024-09-10T10:29:36.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.603+0000 7f5cf554c640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5cd406be30 0x7f5cd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.603+0000 7f5cf554c640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5cd406be30 0x7f5cd406e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009560 tx=0x7f5cec009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:36.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.607+0000 7f5ce5ffb640 1 -- 172.21.15.120:0/2026769622 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ce8079fb0 con 0x7f5cf015b470 2024-09-10T10:29:36.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.771+0000 7f5cf77b0640 1 -- 172.21.15.120:0/2026769622 --> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cf0159b10 con 0x7f5cd406be30 2024-09-10T10:29:36.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.779+0000 7f5ce5ffb640 1 -- 172.21.15.120:0/2026769622 <== mgr.4110 v2:172.21.15.120:6832/3379691116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333349 (secure 0 0 0) 0x7f5cf0159b10 con 0x7f5cd406be30 2024-09-10T10:29:36.785 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:36.787 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T10:29:36.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 -- 172.21.15.120:0/2026769622 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5cd406be30 msgr2=0x7f5cd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5cd406be30 0x7f5cd406e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009560 tx=0x7f5cec009190 comp rx=0 tx=0).stop 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 -- 172.21.15.120:0/2026769622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf015b470 msgr2=0x7f5cf01f0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf015b470 0x7f5cf01f0660 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009d90 tx=0x7f5ce8004a40 comp rx=0 tx=0).stop 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 -- 172.21.15.120:0/2026769622 shutdown_connections 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f5cd406be30 0x7f5cd406e2f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf015b470 0x7f5cf01f0660 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cf0154c00 0x7f5cf00c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 --2- 172.21.15.120:0/2026769622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cf0154820 0x7f5cf00cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 -- 172.21.15.120:0/2026769622 >> 172.21.15.120:0/2026769622 conn(0x7f5cf0070290 msgr2=0x7f5cf0159400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 -- 172.21.15.120:0/2026769622 shutdown_connections 2024-09-10T10:29:36.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.783+0000 7f5cf77b0640 1 -- 172.21.15.120:0/2026769622 wait complete. 2024-09-10T10:29:36.797 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-10T10:29:36.127685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634862,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":119,"num_read_kb":103,"num_write":417,"num_write_kb":2544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":941,"ondisk_log_size":941,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232128,"kb_used_data":7616,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754742592,"statfs":{"total":773094113280,"available":772856414208,"internally_reserved":0,"allocated":7798784,"data_stored":5122900,"data_compressed":68176,"data_compressed_allocated":98304,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003308"},"pg_stats":[{"pgid":"2.1f","version":"78'19","reported_seq":154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:16.014249+0000","last_change":"2024-09-10T10:29:16.014296+0000","last_active":"2024-09-10T10:29:16.014249+0000","last_peered":"2024-09-10T10:29:16.014249+0000","last_clean":"2024-09-10T10:29:16.014249+0000","last_became_active":"2024-09-10T10:26:40.784666+0000","last_became_peered":"2024-09-10T10:26:40.784666+0000","last_unstale":"2024-09-10T10:29:16.014249+0000","last_undegraded":"2024-09-10T10:29:16.014249+0000","last_fullsized":"2024-09-10T10:29:16.014249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'19","last_scrub_stamp":"2024-09-10T10:29:16.014158+0000","last_deep_scrub":"78'19","last_deep_scrub_stamp":"2024-09-10T10:29:16.014158+0000","last_clean_scrub_stamp":"2024-09-10T10:29:16.014158+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:22.961081+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"78'5","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:17.891491+0000","last_change":"2024-09-10T10:29:17.891519+0000","last_active":"2024-09-10T10:29:17.891491+0000","last_peered":"2024-09-10T10:29:17.891491+0000","last_clean":"2024-09-10T10:29:17.891491+0000","last_became_active":"2024-09-10T10:26:41.802053+0000","last_became_peered":"2024-09-10T10:26:41.802053+0000","last_unstale":"2024-09-10T10:29:17.891491+0000","last_undegraded":"2024-09-10T10:29:17.891491+0000","last_fullsized":"2024-09-10T10:29:17.891491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'5","last_scrub_stamp":"2024-09-10T10:29:17.891408+0000","last_deep_scrub":"78'5","last_deep_scrub_stamp":"2024-09-10T10:29:17.891408+0000","last_clean_scrub_stamp":"2024-09-10T10:29:17.891408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:56.691101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:26.990412+0000","last_change":"2024-09-10T10:29:26.990438+0000","last_active":"2024-09-10T10:29:26.990412+0000","last_peered":"2024-09-10T10:29:26.990412+0000","last_clean":"2024-09-10T10:29:26.990412+0000","last_became_active":"2024-09-10T10:26:46.192045+0000","last_became_peered":"2024-09-10T10:26:46.192045+0000","last_unstale":"2024-09-10T10:29:26.990412+0000","last_undegraded":"2024-09-10T10:29:26.990412+0000","last_fullsized":"2024-09-10T10:29:26.990412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:26.990322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:26.990322+0000","last_clean_scrub_stamp":"2024-09-10T10:29:26.990322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.988551+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'2","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:18.898582+0000","last_change":"2024-09-10T10:29:18.898613+0000","last_active":"2024-09-10T10:29:18.898582+0000","last_peered":"2024-09-10T10:29:18.898582+0000","last_clean":"2024-09-10T10:29:18.898582+0000","last_became_active":"2024-09-10T10:26:40.786468+0000","last_became_peered":"2024-09-10T10:26:40.786468+0000","last_unstale":"2024-09-10T10:29:18.898582+0000","last_undegraded":"2024-09-10T10:29:18.898582+0000","last_fullsized":"2024-09-10T10:29:18.898582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-09-10T10:29:18.898486+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-09-10T10:29:18.898486+0000","last_clean_scrub_stamp":"2024-09-10T10:29:18.898486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:30.611038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"78'6","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:22.109832+0000","last_change":"2024-09-10T10:29:22.109887+0000","last_active":"2024-09-10T10:29:22.109832+0000","last_peered":"2024-09-10T10:29:22.109832+0000","last_clean":"2024-09-10T10:29:22.109832+0000","last_became_active":"2024-09-10T10:26:41.795339+0000","last_became_peered":"2024-09-10T10:26:41.795339+0000","last_unstale":"2024-09-10T10:29:22.109832+0000","last_undegraded":"2024-09-10T10:29:22.109832+0000","last_fullsized":"2024-09-10T10:29:22.109832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:22.109708+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:22.109708+0000","last_clean_scrub_stamp":"2024-09-10T10:29:22.109708+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:57.837520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:21.807508+0000","last_change":"2024-09-10T10:29:21.807536+0000","last_active":"2024-09-10T10:29:21.807508+0000","last_peered":"2024-09-10T10:29:21.807508+0000","last_clean":"2024-09-10T10:29:21.807508+0000","last_became_active":"2024-09-10T10:26:45.814748+0000","last_became_peered":"2024-09-10T10:26:45.814748+0000","last_unstale":"2024-09-10T10:29:21.807508+0000","last_undegraded":"2024-09-10T10:29:21.807508+0000","last_fullsized":"2024-09-10T10:29:21.807508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:21.807402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:21.807402+0000","last_clean_scrub_stamp":"2024-09-10T10:29:21.807402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:22.903108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'20","reported_seq":175,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:32.580843+0000","last_change":"2024-09-10T10:29:32.580883+0000","last_active":"2024-09-10T10:29:32.580843+0000","last_peered":"2024-09-10T10:29:32.580843+0000","last_clean":"2024-09-10T10:29:32.580843+0000","last_became_active":"2024-09-10T10:27:51.049148+0000","last_became_peered":"2024-09-10T10:27:51.049148+0000","last_unstale":"2024-09-10T10:29:32.580843+0000","last_undegraded":"2024-09-10T10:29:32.580843+0000","last_fullsized":"2024-09-10T10:29:32.580843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'20","last_scrub_stamp":"2024-09-10T10:29:32.580736+0000","last_deep_scrub":"78'20","last_deep_scrub_stamp":"2024-09-10T10:29:32.580736+0000","last_clean_scrub_stamp":"2024-09-10T10:29:32.580736+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:17.022519+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"78'9","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:23.338111+0000","last_change":"2024-09-10T10:29:23.338139+0000","last_active":"2024-09-10T10:29:23.338111+0000","last_peered":"2024-09-10T10:29:23.338111+0000","last_clean":"2024-09-10T10:29:23.338111+0000","last_became_active":"2024-09-10T10:26:41.785109+0000","last_became_peered":"2024-09-10T10:26:41.785109+0000","last_unstale":"2024-09-10T10:29:23.338111+0000","last_undegraded":"2024-09-10T10:29:23.338111+0000","last_fullsized":"2024-09-10T10:29:23.338111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:23.338024+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:23.338024+0000","last_clean_scrub_stamp":"2024-09-10T10:29:23.338024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:42:17.808839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:27.801114+0000","last_change":"2024-09-10T10:29:27.801143+0000","last_active":"2024-09-10T10:29:27.801114+0000","last_peered":"2024-09-10T10:29:27.801114+0000","last_clean":"2024-09-10T10:29:27.801114+0000","last_became_active":"2024-09-10T10:28:24.244262+0000","last_became_peered":"2024-09-10T10:28:24.244262+0000","last_unstale":"2024-09-10T10:29:27.801114+0000","last_undegraded":"2024-09-10T10:29:27.801114+0000","last_fullsized":"2024-09-10T10:29:27.801114+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:27.801029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:27.801029+0000","last_clean_scrub_stamp":"2024-09-10T10:29:27.801029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:45.945886+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6,2],"acting":[4,0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'8","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:22.457131+0000","last_change":"2024-09-10T10:29:22.457168+0000","last_active":"2024-09-10T10:29:22.457131+0000","last_peered":"2024-09-10T10:29:22.457131+0000","last_clean":"2024-09-10T10:29:22.457131+0000","last_became_active":"2024-09-10T10:26:40.791023+0000","last_became_peered":"2024-09-10T10:26:40.791023+0000","last_unstale":"2024-09-10T10:29:22.457131+0000","last_undegraded":"2024-09-10T10:29:22.457131+0000","last_fullsized":"2024-09-10T10:29:22.457131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:22.457034+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:22.457034+0000","last_clean_scrub_stamp":"2024-09-10T10:29:22.457034+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:25.559755+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"78'15","reported_seq":134,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:21.815405+0000","last_change":"2024-09-10T10:29:21.815433+0000","last_active":"2024-09-10T10:29:21.815405+0000","last_peered":"2024-09-10T10:29:21.815405+0000","last_clean":"2024-09-10T10:29:21.815405+0000","last_became_active":"2024-09-10T10:27:23.954363+0000","last_became_peered":"2024-09-10T10:27:23.954363+0000","last_unstale":"2024-09-10T10:29:21.815405+0000","last_undegraded":"2024-09-10T10:29:21.815405+0000","last_fullsized":"2024-09-10T10:29:21.815405+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'15","last_scrub_stamp":"2024-09-10T10:29:21.815330+0000","last_deep_scrub":"78'15","last_deep_scrub_stamp":"2024-09-10T10:29:21.815330+0000","last_clean_scrub_stamp":"2024-09-10T10:29:21.815330+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:25.224293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:28.848411+0000","last_change":"2024-09-10T10:29:28.848441+0000","last_active":"2024-09-10T10:29:28.848411+0000","last_peered":"2024-09-10T10:29:28.848411+0000","last_clean":"2024-09-10T10:29:28.848411+0000","last_became_active":"2024-09-10T10:28:59.443172+0000","last_became_peered":"2024-09-10T10:28:59.443172+0000","last_unstale":"2024-09-10T10:29:28.848411+0000","last_undegraded":"2024-09-10T10:29:28.848411+0000","last_fullsized":"2024-09-10T10:29:28.848411+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:28.848323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:28.848323+0000","last_clean_scrub_stamp":"2024-09-10T10:29:28.848323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:13.290106+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"78'5","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:19.941888+0000","last_change":"2024-09-10T10:29:19.941936+0000","last_active":"2024-09-10T10:29:19.941888+0000","last_peered":"2024-09-10T10:29:19.941888+0000","last_clean":"2024-09-10T10:29:19.941888+0000","last_became_active":"2024-09-10T10:26:41.788630+0000","last_became_peered":"2024-09-10T10:26:41.788630+0000","last_unstale":"2024-09-10T10:29:19.941888+0000","last_undegraded":"2024-09-10T10:29:19.941888+0000","last_fullsized":"2024-09-10T10:29:19.941888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'5","last_scrub_stamp":"2024-09-10T10:29:19.941783+0000","last_deep_scrub":"78'5","last_deep_scrub_stamp":"2024-09-10T10:29:19.941783+0000","last_clean_scrub_stamp":"2024-09-10T10:29:19.941783+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:37.240840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"78'7","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:18.383630+0000","last_change":"2024-09-10T10:29:18.383664+0000","last_active":"2024-09-10T10:29:18.383630+0000","last_peered":"2024-09-10T10:29:18.383630+0000","last_clean":"2024-09-10T10:29:18.383630+0000","last_became_active":"2024-09-10T10:26:40.790193+0000","last_became_peered":"2024-09-10T10:26:40.790193+0000","last_unstale":"2024-09-10T10:29:18.383630+0000","last_undegraded":"2024-09-10T10:29:18.383630+0000","last_fullsized":"2024-09-10T10:29:18.383630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:18.383550+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:18.383550+0000","last_clean_scrub_stamp":"2024-09-10T10:29:18.383550+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:37.216295+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:26.809619+0000","last_change":"2024-09-10T10:29:26.809650+0000","last_active":"2024-09-10T10:29:26.809619+0000","last_peered":"2024-09-10T10:29:26.809619+0000","last_clean":"2024-09-10T10:29:26.809619+0000","last_became_active":"2024-09-10T10:27:23.954704+0000","last_became_peered":"2024-09-10T10:27:23.954704+0000","last_unstale":"2024-09-10T10:29:26.809619+0000","last_undegraded":"2024-09-10T10:29:26.809619+0000","last_fullsized":"2024-09-10T10:29:26.809619+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:26.809536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:26.809536+0000","last_clean_scrub_stamp":"2024-09-10T10:29:26.809536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:26.121667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"78'8","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:22.930868+0000","last_change":"2024-09-10T10:29:22.930897+0000","last_active":"2024-09-10T10:29:22.930868+0000","last_peered":"2024-09-10T10:29:22.930868+0000","last_clean":"2024-09-10T10:29:22.930868+0000","last_became_active":"2024-09-10T10:26:41.800056+0000","last_became_peered":"2024-09-10T10:26:41.800056+0000","last_unstale":"2024-09-10T10:29:22.930868+0000","last_undegraded":"2024-09-10T10:29:22.930868+0000","last_fullsized":"2024-09-10T10:29:22.930868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:22.930766+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:22.930766+0000","last_clean_scrub_stamp":"2024-09-10T10:29:22.930766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:32.371577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"78'6","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:17.717571+0000","last_change":"2024-09-10T10:29:17.717627+0000","last_active":"2024-09-10T10:29:17.717571+0000","last_peered":"2024-09-10T10:29:17.717571+0000","last_clean":"2024-09-10T10:29:17.717571+0000","last_became_active":"2024-09-10T10:26:40.788518+0000","last_became_peered":"2024-09-10T10:26:40.788518+0000","last_unstale":"2024-09-10T10:29:17.717571+0000","last_undegraded":"2024-09-10T10:29:17.717571+0000","last_fullsized":"2024-09-10T10:29:17.717571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:17.717471+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:17.717471+0000","last_clean_scrub_stamp":"2024-09-10T10:29:17.717471+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:20.820192+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:23.787595+0000","last_change":"2024-09-10T10:29:23.787651+0000","last_active":"2024-09-10T10:29:23.787595+0000","last_peered":"2024-09-10T10:29:23.787595+0000","last_clean":"2024-09-10T10:29:23.787595+0000","last_became_active":"2024-09-10T10:26:45.816469+0000","last_became_peered":"2024-09-10T10:26:45.816469+0000","last_unstale":"2024-09-10T10:29:23.787595+0000","last_undegraded":"2024-09-10T10:29:23.787595+0000","last_fullsized":"2024-09-10T10:29:23.787595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:23.787488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:23.787488+0000","last_clean_scrub_stamp":"2024-09-10T10:29:23.787488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:59.515300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"78'9","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:21.916956+0000","last_change":"2024-09-10T10:29:21.916986+0000","last_active":"2024-09-10T10:29:21.916956+0000","last_peered":"2024-09-10T10:29:21.916956+0000","last_clean":"2024-09-10T10:29:21.916956+0000","last_became_active":"2024-09-10T10:26:41.797593+0000","last_became_peered":"2024-09-10T10:26:41.797593+0000","last_unstale":"2024-09-10T10:29:21.916956+0000","last_undegraded":"2024-09-10T10:29:21.916956+0000","last_fullsized":"2024-09-10T10:29:21.916956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:21.916875+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:21.916875+0000","last_clean_scrub_stamp":"2024-09-10T10:29:21.916875+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:06.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"78'11","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:16.696600+0000","last_change":"2024-09-10T10:29:16.696631+0000","last_active":"2024-09-10T10:29:16.696600+0000","last_peered":"2024-09-10T10:29:16.696600+0000","last_clean":"2024-09-10T10:29:16.696600+0000","last_became_active":"2024-09-10T10:26:40.788544+0000","last_became_peered":"2024-09-10T10:26:40.788544+0000","last_unstale":"2024-09-10T10:29:16.696600+0000","last_undegraded":"2024-09-10T10:29:16.696600+0000","last_fullsized":"2024-09-10T10:29:16.696600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'11","last_scrub_stamp":"2024-09-10T10:29:16.696531+0000","last_deep_scrub":"78'11","last_deep_scrub_stamp":"2024-09-10T10:29:16.696531+0000","last_clean_scrub_stamp":"2024-09-10T10:29:16.696531+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:23.621814+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:28.864727+0000","last_change":"2024-09-10T10:29:28.864767+0000","last_active":"2024-09-10T10:29:28.864727+0000","last_peered":"2024-09-10T10:29:28.864727+0000","last_clean":"2024-09-10T10:29:28.864727+0000","last_became_active":"2024-09-10T10:26:46.193861+0000","last_became_peered":"2024-09-10T10:26:46.193861+0000","last_unstale":"2024-09-10T10:29:28.864727+0000","last_undegraded":"2024-09-10T10:29:28.864727+0000","last_fullsized":"2024-09-10T10:29:28.864727+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:28.864597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:28.864597+0000","last_clean_scrub_stamp":"2024-09-10T10:29:28.864597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:38.305408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"78'2","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:17.729095+0000","last_change":"2024-09-10T10:29:17.729122+0000","last_active":"2024-09-10T10:29:17.729095+0000","last_peered":"2024-09-10T10:29:17.729095+0000","last_clean":"2024-09-10T10:29:17.729095+0000","last_became_active":"2024-09-10T10:26:41.791148+0000","last_became_peered":"2024-09-10T10:26:41.791148+0000","last_unstale":"2024-09-10T10:29:17.729095+0000","last_undegraded":"2024-09-10T10:29:17.729095+0000","last_fullsized":"2024-09-10T10:29:17.729095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T10:29:17.728999+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T10:29:17.728999+0000","last_clean_scrub_stamp":"2024-09-10T10:29:17.728999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:24.675922+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"78'3","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:21.408996+0000","last_change":"2024-09-10T10:29:21.409024+0000","last_active":"2024-09-10T10:29:21.408996+0000","last_peered":"2024-09-10T10:29:21.408996+0000","last_clean":"2024-09-10T10:29:21.408996+0000","last_became_active":"2024-09-10T10:26:40.792527+0000","last_became_peered":"2024-09-10T10:26:40.792527+0000","last_unstale":"2024-09-10T10:29:21.408996+0000","last_undegraded":"2024-09-10T10:29:21.408996+0000","last_fullsized":"2024-09-10T10:29:21.408996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'3","last_scrub_stamp":"2024-09-10T10:29:21.408897+0000","last_deep_scrub":"78'3","last_deep_scrub_stamp":"2024-09-10T10:29:21.408897+0000","last_clean_scrub_stamp":"2024-09-10T10:29:21.408897+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:28.334180+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:22.770104+0000","last_change":"2024-09-10T10:29:22.770129+0000","last_active":"2024-09-10T10:29:22.770104+0000","last_peered":"2024-09-10T10:29:22.770104+0000","last_clean":"2024-09-10T10:29:22.770104+0000","last_became_active":"2024-09-10T10:26:45.815135+0000","last_became_peered":"2024-09-10T10:26:45.815135+0000","last_unstale":"2024-09-10T10:29:22.770104+0000","last_undegraded":"2024-09-10T10:29:22.770104+0000","last_fullsized":"2024-09-10T10:29:22.770104+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:22.770011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:22.770011+0000","last_clean_scrub_stamp":"2024-09-10T10:29:22.770011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:42:17.240826+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"78'7","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:21.140176+0000","last_change":"2024-09-10T10:29:21.140204+0000","last_active":"2024-09-10T10:29:21.140176+0000","last_peered":"2024-09-10T10:29:21.140176+0000","last_clean":"2024-09-10T10:29:21.140176+0000","last_became_active":"2024-09-10T10:26:41.795595+0000","last_became_peered":"2024-09-10T10:26:41.795595+0000","last_unstale":"2024-09-10T10:29:21.140176+0000","last_undegraded":"2024-09-10T10:29:21.140176+0000","last_fullsized":"2024-09-10T10:29:21.140176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:21.140093+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:21.140093+0000","last_clean_scrub_stamp":"2024-09-10T10:29:21.140093+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:42:15.610907+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"78'12","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:16.998979+0000","last_change":"2024-09-10T10:29:16.999020+0000","last_active":"2024-09-10T10:29:16.998979+0000","last_peered":"2024-09-10T10:29:16.998979+0000","last_clean":"2024-09-10T10:29:16.998979+0000","last_became_active":"2024-09-10T10:26:40.792293+0000","last_became_peered":"2024-09-10T10:26:40.792293+0000","last_unstale":"2024-09-10T10:29:16.998979+0000","last_undegraded":"2024-09-10T10:29:16.998979+0000","last_fullsized":"2024-09-10T10:29:16.998979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'12","last_scrub_stamp":"2024-09-10T10:29:16.998895+0000","last_deep_scrub":"78'12","last_deep_scrub_stamp":"2024-09-10T10:29:16.998895+0000","last_clean_scrub_stamp":"2024-09-10T10:29:16.998895+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:36.654355+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:25.856803+0000","last_change":"2024-09-10T10:29:25.856835+0000","last_active":"2024-09-10T10:29:25.856803+0000","last_peered":"2024-09-10T10:29:25.856803+0000","last_clean":"2024-09-10T10:29:25.856803+0000","last_became_active":"2024-09-10T10:26:46.196094+0000","last_became_peered":"2024-09-10T10:26:46.196094+0000","last_unstale":"2024-09-10T10:29:25.856803+0000","last_undegraded":"2024-09-10T10:29:25.856803+0000","last_fullsized":"2024-09-10T10:29:25.856803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:25.856707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:25.856707+0000","last_clean_scrub_stamp":"2024-09-10T10:29:25.856707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:21.821952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"78'5","reported_seq":78,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:28.910809+0000","last_change":"2024-09-10T10:29:28.910846+0000","last_active":"2024-09-10T10:29:28.910809+0000","last_peered":"2024-09-10T10:29:28.910809+0000","last_clean":"2024-09-10T10:29:28.910809+0000","last_became_active":"2024-09-10T10:28:24.373921+0000","last_became_peered":"2024-09-10T10:28:24.373921+0000","last_unstale":"2024-09-10T10:29:28.910809+0000","last_undegraded":"2024-09-10T10:29:28.910809+0000","last_fullsized":"2024-09-10T10:29:28.910809+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'5","last_scrub_stamp":"2024-09-10T10:29:28.910726+0000","last_deep_scrub":"78'5","last_deep_scrub_stamp":"2024-09-10T10:29:28.910726+0000","last_clean_scrub_stamp":"2024-09-10T10:29:28.910726+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:13.352509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'19","reported_seq":153,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:20.753990+0000","last_change":"2024-09-10T10:29:20.754025+0000","last_active":"2024-09-10T10:29:20.753990+0000","last_peered":"2024-09-10T10:29:20.753990+0000","last_clean":"2024-09-10T10:29:20.753990+0000","last_became_active":"2024-09-10T10:26:40.796016+0000","last_became_peered":"2024-09-10T10:26:40.796016+0000","last_unstale":"2024-09-10T10:29:20.753990+0000","last_undegraded":"2024-09-10T10:29:20.753990+0000","last_fullsized":"2024-09-10T10:29:20.753990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'19","last_scrub_stamp":"2024-09-10T10:29:20.753913+0000","last_deep_scrub":"78'19","last_deep_scrub_stamp":"2024-09-10T10:29:20.753913+0000","last_clean_scrub_stamp":"2024-09-10T10:29:20.753913+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:21.849619+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:28.026657+0000","last_change":"2024-09-10T10:29:28.026698+0000","last_active":"2024-09-10T10:29:28.026657+0000","last_peered":"2024-09-10T10:29:28.026657+0000","last_clean":"2024-09-10T10:29:28.026657+0000","last_became_active":"2024-09-10T10:26:46.192080+0000","last_became_peered":"2024-09-10T10:26:46.192080+0000","last_unstale":"2024-09-10T10:29:28.026657+0000","last_undegraded":"2024-09-10T10:29:28.026657+0000","last_fullsized":"2024-09-10T10:29:28.026657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:28.026531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:28.026531+0000","last_clean_scrub_stamp":"2024-09-10T10:29:28.026531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:56.629044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"78'9","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:22.758380+0000","last_change":"2024-09-10T10:29:22.758426+0000","last_active":"2024-09-10T10:29:22.758380+0000","last_peered":"2024-09-10T10:29:22.758380+0000","last_clean":"2024-09-10T10:29:22.758380+0000","last_became_active":"2024-09-10T10:26:41.788217+0000","last_became_peered":"2024-09-10T10:26:41.788217+0000","last_unstale":"2024-09-10T10:29:22.758380+0000","last_undegraded":"2024-09-10T10:29:22.758380+0000","last_fullsized":"2024-09-10T10:29:22.758380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:22.758259+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:22.758259+0000","last_clean_scrub_stamp":"2024-09-10T10:29:22.758259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:21.205775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"78'7","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:20.392563+0000","last_change":"2024-09-10T10:29:20.392595+0000","last_active":"2024-09-10T10:29:20.392563+0000","last_peered":"2024-09-10T10:29:20.392563+0000","last_clean":"2024-09-10T10:29:20.392563+0000","last_became_active":"2024-09-10T10:26:40.791829+0000","last_became_peered":"2024-09-10T10:26:40.791829+0000","last_unstale":"2024-09-10T10:29:20.392563+0000","last_undegraded":"2024-09-10T10:29:20.392563+0000","last_fullsized":"2024-09-10T10:29:20.392563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:20.392476+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:20.392476+0000","last_clean_scrub_stamp":"2024-09-10T10:29:20.392476+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:35.927961+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:27.065925+0000","last_change":"2024-09-10T10:29:27.065954+0000","last_active":"2024-09-10T10:29:27.065925+0000","last_peered":"2024-09-10T10:29:27.065925+0000","last_clean":"2024-09-10T10:29:27.065925+0000","last_became_active":"2024-09-10T10:26:46.192967+0000","last_became_peered":"2024-09-10T10:26:46.192967+0000","last_unstale":"2024-09-10T10:29:27.065925+0000","last_undegraded":"2024-09-10T10:29:27.065925+0000","last_fullsized":"2024-09-10T10:29:27.065925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:27.065840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:27.065840+0000","last_clean_scrub_stamp":"2024-09-10T10:29:27.065840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:11.507623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"78'5","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:17.203737+0000","last_change":"2024-09-10T10:29:17.203780+0000","last_active":"2024-09-10T10:29:17.203737+0000","last_peered":"2024-09-10T10:29:17.203737+0000","last_clean":"2024-09-10T10:29:17.203737+0000","last_became_active":"2024-09-10T10:26:41.790855+0000","last_became_peered":"2024-09-10T10:26:41.790855+0000","last_unstale":"2024-09-10T10:29:17.203737+0000","last_undegraded":"2024-09-10T10:29:17.203737+0000","last_fullsized":"2024-09-10T10:29:17.203737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'5","last_scrub_stamp":"2024-09-10T10:29:17.203632+0000","last_deep_scrub":"78'5","last_deep_scrub_stamp":"2024-09-10T10:29:17.203632+0000","last_clean_scrub_stamp":"2024-09-10T10:29:17.203632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:36.859092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'1","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:15.658959+0000","last_change":"2024-09-10T10:29:15.659000+0000","last_active":"2024-09-10T10:29:15.658959+0000","last_peered":"2024-09-10T10:29:15.658959+0000","last_clean":"2024-09-10T10:29:15.658959+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:15.658959+0000","last_undegraded":"2024-09-10T10:29:15.658959+0000","last_fullsized":"2024-09-10T10:29:15.658959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-10T10:29:15.658870+0000","last_deep_scrub":"26'1","last_deep_scrub_stamp":"2024-09-10T10:29:15.658870+0000","last_clean_scrub_stamp":"2024-09-10T10:29:15.658870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:35.314330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:26.807766+0000","last_change":"2024-09-10T10:29:26.807790+0000","last_active":"2024-09-10T10:29:26.807766+0000","last_peered":"2024-09-10T10:29:26.807766+0000","last_clean":"2024-09-10T10:29:26.807766+0000","last_became_active":"2024-09-10T10:26:45.820375+0000","last_became_peered":"2024-09-10T10:26:45.820375+0000","last_unstale":"2024-09-10T10:29:26.807766+0000","last_undegraded":"2024-09-10T10:29:26.807766+0000","last_fullsized":"2024-09-10T10:29:26.807766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:26.807673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:26.807673+0000","last_clean_scrub_stamp":"2024-09-10T10:29:26.807673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:25.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"78'7","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:16.776355+0000","last_change":"2024-09-10T10:29:16.776398+0000","last_active":"2024-09-10T10:29:16.776355+0000","last_peered":"2024-09-10T10:29:16.776355+0000","last_clean":"2024-09-10T10:29:16.776355+0000","last_became_active":"2024-09-10T10:26:41.788071+0000","last_became_peered":"2024-09-10T10:26:41.788071+0000","last_unstale":"2024-09-10T10:29:16.776355+0000","last_undegraded":"2024-09-10T10:29:16.776355+0000","last_fullsized":"2024-09-10T10:29:16.776355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:16.776227+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:16.776227+0000","last_clean_scrub_stamp":"2024-09-10T10:29:16.776227+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:36.431687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"78'7","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:20.282468+0000","last_change":"2024-09-10T10:29:20.282521+0000","last_active":"2024-09-10T10:29:20.282468+0000","last_peered":"2024-09-10T10:29:20.282468+0000","last_clean":"2024-09-10T10:29:20.282468+0000","last_became_active":"2024-09-10T10:26:40.790199+0000","last_became_peered":"2024-09-10T10:26:40.790199+0000","last_unstale":"2024-09-10T10:29:20.282468+0000","last_undegraded":"2024-09-10T10:29:20.282468+0000","last_fullsized":"2024-09-10T10:29:20.282468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:20.282360+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:20.282360+0000","last_clean_scrub_stamp":"2024-09-10T10:29:20.282360+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:09.675186+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:33.383526+0000","last_change":"2024-09-10T10:29:33.383555+0000","last_active":"2024-09-10T10:29:33.383526+0000","last_peered":"2024-09-10T10:29:33.383526+0000","last_clean":"2024-09-10T10:29:33.383526+0000","last_became_active":"2024-09-10T10:28:24.244556+0000","last_became_peered":"2024-09-10T10:28:24.244556+0000","last_unstale":"2024-09-10T10:29:33.383526+0000","last_undegraded":"2024-09-10T10:29:33.383526+0000","last_fullsized":"2024-09-10T10:29:33.383526+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:33.383437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:33.383437+0000","last_clean_scrub_stamp":"2024-09-10T10:29:33.383437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:35.319774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"78'8","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:23.716373+0000","last_change":"2024-09-10T10:29:23.716400+0000","last_active":"2024-09-10T10:29:23.716373+0000","last_peered":"2024-09-10T10:29:23.716373+0000","last_clean":"2024-09-10T10:29:23.716373+0000","last_became_active":"2024-09-10T10:26:41.792312+0000","last_became_peered":"2024-09-10T10:26:41.792312+0000","last_unstale":"2024-09-10T10:29:23.716373+0000","last_undegraded":"2024-09-10T10:29:23.716373+0000","last_fullsized":"2024-09-10T10:29:23.716373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:23.716265+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:23.716265+0000","last_clean_scrub_stamp":"2024-09-10T10:29:23.716265+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:11.316226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"78'12","reported_seq":153,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:15.909037+0000","last_change":"2024-09-10T10:29:15.909085+0000","last_active":"2024-09-10T10:29:15.909037+0000","last_peered":"2024-09-10T10:29:15.909037+0000","last_clean":"2024-09-10T10:29:15.909037+0000","last_became_active":"2024-09-10T10:26:40.786225+0000","last_became_peered":"2024-09-10T10:26:40.786225+0000","last_unstale":"2024-09-10T10:29:15.909037+0000","last_undegraded":"2024-09-10T10:29:15.909037+0000","last_fullsized":"2024-09-10T10:29:15.909037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'12","last_scrub_stamp":"2024-09-10T10:29:15.908969+0000","last_deep_scrub":"78'12","last_deep_scrub_stamp":"2024-09-10T10:29:15.908969+0000","last_clean_scrub_stamp":"2024-09-10T10:29:15.908969+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:55.221882+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":127,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:27.836492+0000","last_change":"2024-09-10T10:29:27.836520+0000","last_active":"2024-09-10T10:29:27.836492+0000","last_peered":"2024-09-10T10:29:27.836492+0000","last_clean":"2024-09-10T10:29:27.836492+0000","last_became_active":"2024-09-10T10:26:45.823640+0000","last_became_peered":"2024-09-10T10:26:45.823640+0000","last_unstale":"2024-09-10T10:29:27.836492+0000","last_undegraded":"2024-09-10T10:29:27.836492+0000","last_fullsized":"2024-09-10T10:29:27.836492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:27.836403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:27.836403+0000","last_clean_scrub_stamp":"2024-09-10T10:29:27.836403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.680402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"78'5","reported_seq":148,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:30.017804+0000","last_change":"2024-09-10T10:29:30.017856+0000","last_active":"2024-09-10T10:29:30.017804+0000","last_peered":"2024-09-10T10:29:30.017804+0000","last_clean":"2024-09-10T10:29:30.017804+0000","last_became_active":"2024-09-10T10:28:59.446544+0000","last_became_peered":"2024-09-10T10:28:59.446544+0000","last_unstale":"2024-09-10T10:29:30.017804+0000","last_undegraded":"2024-09-10T10:29:30.017804+0000","last_fullsized":"2024-09-10T10:29:30.017804+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'5","last_scrub_stamp":"2024-09-10T10:29:30.017676+0000","last_deep_scrub":"78'5","last_deep_scrub_stamp":"2024-09-10T10:29:30.017676+0000","last_clean_scrub_stamp":"2024-09-10T10:29:30.017676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:58.704085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"78'5","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:19.352653+0000","last_change":"2024-09-10T10:29:19.352707+0000","last_active":"2024-09-10T10:29:19.352653+0000","last_peered":"2024-09-10T10:29:19.352653+0000","last_clean":"2024-09-10T10:29:19.352653+0000","last_became_active":"2024-09-10T10:26:40.791455+0000","last_became_peered":"2024-09-10T10:26:40.791455+0000","last_unstale":"2024-09-10T10:29:19.352653+0000","last_undegraded":"2024-09-10T10:29:19.352653+0000","last_fullsized":"2024-09-10T10:29:19.352653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'5","last_scrub_stamp":"2024-09-10T10:29:19.352561+0000","last_deep_scrub":"78'5","last_deep_scrub_stamp":"2024-09-10T10:29:19.352561+0000","last_clean_scrub_stamp":"2024-09-10T10:29:19.352561+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:55.575134+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:28.050295+0000","last_change":"2024-09-10T10:29:28.050326+0000","last_active":"2024-09-10T10:29:28.050295+0000","last_peered":"2024-09-10T10:29:28.050295+0000","last_clean":"2024-09-10T10:29:28.050295+0000","last_became_active":"2024-09-10T10:26:46.192995+0000","last_became_peered":"2024-09-10T10:26:46.192995+0000","last_unstale":"2024-09-10T10:29:28.050295+0000","last_undegraded":"2024-09-10T10:29:28.050295+0000","last_fullsized":"2024-09-10T10:29:28.050295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:28.050206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:28.050206+0000","last_clean_scrub_stamp":"2024-09-10T10:29:28.050206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:37.491017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"78'8","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:24.925583+0000","last_change":"2024-09-10T10:29:24.925611+0000","last_active":"2024-09-10T10:29:24.925583+0000","last_peered":"2024-09-10T10:29:24.925583+0000","last_clean":"2024-09-10T10:29:24.925583+0000","last_became_active":"2024-09-10T10:26:41.801532+0000","last_became_peered":"2024-09-10T10:26:41.801532+0000","last_unstale":"2024-09-10T10:29:24.925583+0000","last_undegraded":"2024-09-10T10:29:24.925583+0000","last_fullsized":"2024-09-10T10:29:24.925583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:24.925503+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:24.925503+0000","last_clean_scrub_stamp":"2024-09-10T10:29:24.925503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:53.611912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'11","reported_seq":149,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:19.282972+0000","last_change":"2024-09-10T10:29:19.283003+0000","last_active":"2024-09-10T10:29:19.282972+0000","last_peered":"2024-09-10T10:29:19.282972+0000","last_clean":"2024-09-10T10:29:19.282972+0000","last_became_active":"2024-09-10T10:26:40.785766+0000","last_became_peered":"2024-09-10T10:26:40.785766+0000","last_unstale":"2024-09-10T10:29:19.282972+0000","last_undegraded":"2024-09-10T10:29:19.282972+0000","last_fullsized":"2024-09-10T10:29:19.282972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'11","last_scrub_stamp":"2024-09-10T10:29:19.282888+0000","last_deep_scrub":"78'11","last_deep_scrub_stamp":"2024-09-10T10:29:19.282888+0000","last_clean_scrub_stamp":"2024-09-10T10:29:19.282888+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:30.692971+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"78'119","reported_seq":292,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:17.431586+0000","last_change":"2024-09-10T10:29:17.431621+0000","last_active":"2024-09-10T10:29:17.431586+0000","last_peered":"2024-09-10T10:29:17.431586+0000","last_clean":"2024-09-10T10:29:17.431586+0000","last_became_active":"2024-09-10T10:26:32.267216+0000","last_became_peered":"2024-09-10T10:26:32.267216+0000","last_unstale":"2024-09-10T10:29:17.431586+0000","last_undegraded":"2024-09-10T10:29:17.431586+0000","last_fullsized":"2024-09-10T10:29:17.431586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'119","last_scrub_stamp":"2024-09-10T10:29:17.431501+0000","last_deep_scrub":"78'119","last_deep_scrub_stamp":"2024-09-10T10:29:17.431501+0000","last_clean_scrub_stamp":"2024-09-10T10:29:17.431501+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:32.888644+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:25.806839+0000","last_change":"2024-09-10T10:29:25.806869+0000","last_active":"2024-09-10T10:29:25.806839+0000","last_peered":"2024-09-10T10:29:25.806839+0000","last_clean":"2024-09-10T10:29:25.806839+0000","last_became_active":"2024-09-10T10:26:46.192495+0000","last_became_peered":"2024-09-10T10:26:46.192495+0000","last_unstale":"2024-09-10T10:29:25.806839+0000","last_undegraded":"2024-09-10T10:29:25.806839+0000","last_fullsized":"2024-09-10T10:29:25.806839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:25.806736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:25.806736+0000","last_clean_scrub_stamp":"2024-09-10T10:29:25.806736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:21.771981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"78'7","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:23.468131+0000","last_change":"2024-09-10T10:29:23.468175+0000","last_active":"2024-09-10T10:29:23.468131+0000","last_peered":"2024-09-10T10:29:23.468131+0000","last_clean":"2024-09-10T10:29:23.468131+0000","last_became_active":"2024-09-10T10:26:41.789365+0000","last_became_peered":"2024-09-10T10:26:41.789365+0000","last_unstale":"2024-09-10T10:29:23.468131+0000","last_undegraded":"2024-09-10T10:29:23.468131+0000","last_fullsized":"2024-09-10T10:29:23.468131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:23.468039+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:23.468039+0000","last_clean_scrub_stamp":"2024-09-10T10:29:23.468039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:40.767097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'8","reported_seq":146,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:17.872354+0000","last_change":"2024-09-10T10:29:17.872385+0000","last_active":"2024-09-10T10:29:17.872354+0000","last_peered":"2024-09-10T10:29:17.872354+0000","last_clean":"2024-09-10T10:29:17.872354+0000","last_became_active":"2024-09-10T10:26:40.790265+0000","last_became_peered":"2024-09-10T10:26:40.790265+0000","last_unstale":"2024-09-10T10:29:17.872354+0000","last_undegraded":"2024-09-10T10:29:17.872354+0000","last_fullsized":"2024-09-10T10:29:17.872354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:17.872274+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:17.872274+0000","last_clean_scrub_stamp":"2024-09-10T10:29:17.872274+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:14.028053+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:26.848554+0000","last_change":"2024-09-10T10:29:26.848600+0000","last_active":"2024-09-10T10:29:26.848554+0000","last_peered":"2024-09-10T10:29:26.848554+0000","last_clean":"2024-09-10T10:29:26.848554+0000","last_became_active":"2024-09-10T10:26:46.192135+0000","last_became_peered":"2024-09-10T10:26:46.192135+0000","last_unstale":"2024-09-10T10:29:26.848554+0000","last_undegraded":"2024-09-10T10:29:26.848554+0000","last_fullsized":"2024-09-10T10:29:26.848554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:26.848457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:26.848457+0000","last_clean_scrub_stamp":"2024-09-10T10:29:26.848457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:38.270825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"78'6","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:16.159228+0000","last_change":"2024-09-10T10:29:16.159273+0000","last_active":"2024-09-10T10:29:16.159228+0000","last_peered":"2024-09-10T10:29:16.159228+0000","last_clean":"2024-09-10T10:29:16.159228+0000","last_became_active":"2024-09-10T10:26:41.792072+0000","last_became_peered":"2024-09-10T10:26:41.792072+0000","last_unstale":"2024-09-10T10:29:16.159228+0000","last_undegraded":"2024-09-10T10:29:16.159228+0000","last_fullsized":"2024-09-10T10:29:16.159228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:16.159131+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:16.159131+0000","last_clean_scrub_stamp":"2024-09-10T10:29:16.159131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:52.381703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"78'4","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:19.716192+0000","last_change":"2024-09-10T10:29:19.716222+0000","last_active":"2024-09-10T10:29:19.716192+0000","last_peered":"2024-09-10T10:29:19.716192+0000","last_clean":"2024-09-10T10:29:19.716192+0000","last_became_active":"2024-09-10T10:26:40.793100+0000","last_became_peered":"2024-09-10T10:26:40.793100+0000","last_unstale":"2024-09-10T10:29:19.716192+0000","last_undegraded":"2024-09-10T10:29:19.716192+0000","last_fullsized":"2024-09-10T10:29:19.716192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'4","last_scrub_stamp":"2024-09-10T10:29:19.716110+0000","last_deep_scrub":"78'4","last_deep_scrub_stamp":"2024-09-10T10:29:19.716110+0000","last_clean_scrub_stamp":"2024-09-10T10:29:19.716110+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:48.330005+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:25.072931+0000","last_change":"2024-09-10T10:29:25.072957+0000","last_active":"2024-09-10T10:29:25.072931+0000","last_peered":"2024-09-10T10:29:25.072931+0000","last_clean":"2024-09-10T10:29:25.072931+0000","last_became_active":"2024-09-10T10:26:45.819674+0000","last_became_peered":"2024-09-10T10:26:45.819674+0000","last_unstale":"2024-09-10T10:29:25.072931+0000","last_undegraded":"2024-09-10T10:29:25.072931+0000","last_fullsized":"2024-09-10T10:29:25.072931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:25.072836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:25.072836+0000","last_clean_scrub_stamp":"2024-09-10T10:29:25.072836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:24.384967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"78'9","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:31.552600+0000","last_change":"2024-09-10T10:29:31.552634+0000","last_active":"2024-09-10T10:29:31.552600+0000","last_peered":"2024-09-10T10:29:31.552600+0000","last_clean":"2024-09-10T10:29:31.552600+0000","last_became_active":"2024-09-10T10:26:41.802015+0000","last_became_peered":"2024-09-10T10:26:41.802015+0000","last_unstale":"2024-09-10T10:29:31.552600+0000","last_undegraded":"2024-09-10T10:29:31.552600+0000","last_fullsized":"2024-09-10T10:29:31.552600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:31.552496+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:31.552496+0000","last_clean_scrub_stamp":"2024-09-10T10:29:31.552496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:49.697353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'7","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:14.946681+0000","last_change":"2024-09-10T10:29:14.946711+0000","last_active":"2024-09-10T10:29:14.946681+0000","last_peered":"2024-09-10T10:29:14.946681+0000","last_clean":"2024-09-10T10:29:14.946681+0000","last_became_active":"2024-09-10T10:26:40.784366+0000","last_became_peered":"2024-09-10T10:26:40.784366+0000","last_unstale":"2024-09-10T10:29:14.946681+0000","last_undegraded":"2024-09-10T10:29:14.946681+0000","last_fullsized":"2024-09-10T10:29:14.946681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:14.946617+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:14.946617+0000","last_clean_scrub_stamp":"2024-09-10T10:29:14.946617+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:54.662550+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:31.358474+0000","last_change":"2024-09-10T10:29:31.358525+0000","last_active":"2024-09-10T10:29:31.358474+0000","last_peered":"2024-09-10T10:29:31.358474+0000","last_clean":"2024-09-10T10:29:31.358474+0000","last_became_active":"2024-09-10T10:26:46.192542+0000","last_became_peered":"2024-09-10T10:26:46.192542+0000","last_unstale":"2024-09-10T10:29:31.358474+0000","last_undegraded":"2024-09-10T10:29:31.358474+0000","last_fullsized":"2024-09-10T10:29:31.358474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:31.358367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:31.358367+0000","last_clean_scrub_stamp":"2024-09-10T10:29:31.358367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:57.127630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"78'5","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:27.417204+0000","last_change":"2024-09-10T10:29:27.417230+0000","last_active":"2024-09-10T10:29:27.417204+0000","last_peered":"2024-09-10T10:29:27.417204+0000","last_clean":"2024-09-10T10:29:27.417204+0000","last_became_active":"2024-09-10T10:26:41.798767+0000","last_became_peered":"2024-09-10T10:26:41.798767+0000","last_unstale":"2024-09-10T10:29:27.417204+0000","last_undegraded":"2024-09-10T10:29:27.417204+0000","last_fullsized":"2024-09-10T10:29:27.417204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'5","last_scrub_stamp":"2024-09-10T10:29:27.417127+0000","last_deep_scrub":"78'5","last_deep_scrub_stamp":"2024-09-10T10:29:27.417127+0000","last_clean_scrub_stamp":"2024-09-10T10:29:27.417127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:03.144938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"78'4","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:15.747899+0000","last_change":"2024-09-10T10:29:15.747934+0000","last_active":"2024-09-10T10:29:15.747899+0000","last_peered":"2024-09-10T10:29:15.747899+0000","last_clean":"2024-09-10T10:29:15.747899+0000","last_became_active":"2024-09-10T10:26:40.791589+0000","last_became_peered":"2024-09-10T10:26:40.791589+0000","last_unstale":"2024-09-10T10:29:15.747899+0000","last_undegraded":"2024-09-10T10:29:15.747899+0000","last_fullsized":"2024-09-10T10:29:15.747899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'4","last_scrub_stamp":"2024-09-10T10:29:15.747785+0000","last_deep_scrub":"78'4","last_deep_scrub_stamp":"2024-09-10T10:29:15.747785+0000","last_clean_scrub_stamp":"2024-09-10T10:29:15.747785+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:51.970357+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:24.826735+0000","last_change":"2024-09-10T10:29:24.826762+0000","last_active":"2024-09-10T10:29:24.826735+0000","last_peered":"2024-09-10T10:29:24.826735+0000","last_clean":"2024-09-10T10:29:24.826735+0000","last_became_active":"2024-09-10T10:26:45.815944+0000","last_became_peered":"2024-09-10T10:26:45.815944+0000","last_unstale":"2024-09-10T10:29:24.826735+0000","last_undegraded":"2024-09-10T10:29:24.826735+0000","last_fullsized":"2024-09-10T10:29:24.826735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:24.826645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:24.826645+0000","last_clean_scrub_stamp":"2024-09-10T10:29:24.826645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.426606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"78'6","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:29.462724+0000","last_change":"2024-09-10T10:29:29.462763+0000","last_active":"2024-09-10T10:29:29.462724+0000","last_peered":"2024-09-10T10:29:29.462724+0000","last_clean":"2024-09-10T10:29:29.462724+0000","last_became_active":"2024-09-10T10:26:41.799306+0000","last_became_peered":"2024-09-10T10:26:41.799306+0000","last_unstale":"2024-09-10T10:29:29.462724+0000","last_undegraded":"2024-09-10T10:29:29.462724+0000","last_fullsized":"2024-09-10T10:29:29.462724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:29.462600+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:29.462600+0000","last_clean_scrub_stamp":"2024-09-10T10:29:29.462600+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:25.427845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'26","reported_seq":164,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:16.870018+0000","last_change":"2024-09-10T10:29:16.870051+0000","last_active":"2024-09-10T10:29:16.870018+0000","last_peered":"2024-09-10T10:29:16.870018+0000","last_clean":"2024-09-10T10:29:16.870018+0000","last_became_active":"2024-09-10T10:26:40.789393+0000","last_became_peered":"2024-09-10T10:26:40.789393+0000","last_unstale":"2024-09-10T10:29:16.870018+0000","last_undegraded":"2024-09-10T10:29:16.870018+0000","last_fullsized":"2024-09-10T10:29:16.870018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'26","last_scrub_stamp":"2024-09-10T10:29:16.869954+0000","last_deep_scrub":"78'26","last_deep_scrub_stamp":"2024-09-10T10:29:16.869954+0000","last_clean_scrub_stamp":"2024-09-10T10:29:16.869954+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:04.139757+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":128,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:26.082834+0000","last_change":"2024-09-10T10:29:26.082886+0000","last_active":"2024-09-10T10:29:26.082834+0000","last_peered":"2024-09-10T10:29:26.082834+0000","last_clean":"2024-09-10T10:29:26.082834+0000","last_became_active":"2024-09-10T10:26:45.821525+0000","last_became_peered":"2024-09-10T10:26:45.821525+0000","last_unstale":"2024-09-10T10:29:26.082834+0000","last_undegraded":"2024-09-10T10:29:26.082834+0000","last_fullsized":"2024-09-10T10:29:26.082834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:26.082717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:26.082717+0000","last_clean_scrub_stamp":"2024-09-10T10:29:26.082717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:34.301928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"78'6","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:22.909829+0000","last_change":"2024-09-10T10:29:22.909857+0000","last_active":"2024-09-10T10:29:22.909829+0000","last_peered":"2024-09-10T10:29:22.909829+0000","last_clean":"2024-09-10T10:29:22.909829+0000","last_became_active":"2024-09-10T10:26:41.800175+0000","last_became_peered":"2024-09-10T10:26:41.800175+0000","last_unstale":"2024-09-10T10:29:22.909829+0000","last_undegraded":"2024-09-10T10:29:22.909829+0000","last_fullsized":"2024-09-10T10:29:22.909829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:22.909733+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:22.909733+0000","last_clean_scrub_stamp":"2024-09-10T10:29:22.909733+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:44.376545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"78'8","reported_seq":172,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:30.886145+0000","last_change":"2024-09-10T10:29:30.886178+0000","last_active":"2024-09-10T10:29:30.886145+0000","last_peered":"2024-09-10T10:29:30.886145+0000","last_clean":"2024-09-10T10:29:30.886145+0000","last_became_active":"2024-09-10T10:28:59.446453+0000","last_became_peered":"2024-09-10T10:28:59.446453+0000","last_unstale":"2024-09-10T10:29:30.886145+0000","last_undegraded":"2024-09-10T10:29:30.886145+0000","last_fullsized":"2024-09-10T10:29:30.886145+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:30.886059+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:30.886059+0000","last_clean_scrub_stamp":"2024-09-10T10:29:30.886059+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:56.655322+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1176,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:30.011533+0000","last_change":"2024-09-10T10:29:30.011561+0000","last_active":"2024-09-10T10:29:30.011533+0000","last_peered":"2024-09-10T10:29:30.011533+0000","last_clean":"2024-09-10T10:29:30.011533+0000","last_became_active":"2024-09-10T10:26:46.191787+0000","last_became_peered":"2024-09-10T10:26:46.191787+0000","last_unstale":"2024-09-10T10:29:30.011533+0000","last_undegraded":"2024-09-10T10:29:30.011533+0000","last_fullsized":"2024-09-10T10:29:30.011533+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:30.011446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:30.011446+0000","last_clean_scrub_stamp":"2024-09-10T10:29:30.011446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:11.586322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"78'15","reported_seq":147,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:28.446705+0000","last_change":"2024-09-10T10:29:28.446730+0000","last_active":"2024-09-10T10:29:28.446705+0000","last_peered":"2024-09-10T10:29:28.446705+0000","last_clean":"2024-09-10T10:29:28.446705+0000","last_became_active":"2024-09-10T10:26:41.799354+0000","last_became_peered":"2024-09-10T10:26:41.799354+0000","last_unstale":"2024-09-10T10:29:28.446705+0000","last_undegraded":"2024-09-10T10:29:28.446705+0000","last_fullsized":"2024-09-10T10:29:28.446705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'15","last_scrub_stamp":"2024-09-10T10:29:28.446587+0000","last_deep_scrub":"78'15","last_deep_scrub_stamp":"2024-09-10T10:29:28.446587+0000","last_clean_scrub_stamp":"2024-09-10T10:29:28.446587+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:32.826900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"78'9","reported_seq":147,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:22.315117+0000","last_change":"2024-09-10T10:29:22.315149+0000","last_active":"2024-09-10T10:29:22.315117+0000","last_peered":"2024-09-10T10:29:22.315117+0000","last_clean":"2024-09-10T10:29:22.315117+0000","last_became_active":"2024-09-10T10:26:40.794169+0000","last_became_peered":"2024-09-10T10:26:40.794169+0000","last_unstale":"2024-09-10T10:29:22.315117+0000","last_undegraded":"2024-09-10T10:29:22.315117+0000","last_fullsized":"2024-09-10T10:29:22.315117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:22.315038+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:22.315038+0000","last_clean_scrub_stamp":"2024-09-10T10:29:22.315038+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:23.410745+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:29.007099+0000","last_change":"2024-09-10T10:29:29.007122+0000","last_active":"2024-09-10T10:29:29.007099+0000","last_peered":"2024-09-10T10:29:29.007099+0000","last_clean":"2024-09-10T10:29:29.007099+0000","last_became_active":"2024-09-10T10:26:46.192093+0000","last_became_peered":"2024-09-10T10:26:46.192093+0000","last_unstale":"2024-09-10T10:29:29.007099+0000","last_undegraded":"2024-09-10T10:29:29.007099+0000","last_fullsized":"2024-09-10T10:29:29.007099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:29.007018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:29.007018+0000","last_clean_scrub_stamp":"2024-09-10T10:29:29.007018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:13.246478+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"78'9","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:18.061983+0000","last_change":"2024-09-10T10:29:18.062032+0000","last_active":"2024-09-10T10:29:18.061983+0000","last_peered":"2024-09-10T10:29:18.061983+0000","last_clean":"2024-09-10T10:29:18.061983+0000","last_became_active":"2024-09-10T10:26:41.794489+0000","last_became_peered":"2024-09-10T10:26:41.794489+0000","last_unstale":"2024-09-10T10:29:18.061983+0000","last_undegraded":"2024-09-10T10:29:18.061983+0000","last_fullsized":"2024-09-10T10:29:18.061983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:18.061872+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:18.061872+0000","last_clean_scrub_stamp":"2024-09-10T10:29:18.061872+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:07.454697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"78'15","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:14.985118+0000","last_change":"2024-09-10T10:29:14.985179+0000","last_active":"2024-09-10T10:29:14.985118+0000","last_peered":"2024-09-10T10:29:14.985118+0000","last_clean":"2024-09-10T10:29:14.985118+0000","last_became_active":"2024-09-10T10:26:40.778988+0000","last_became_peered":"2024-09-10T10:26:40.778988+0000","last_unstale":"2024-09-10T10:29:14.985118+0000","last_undegraded":"2024-09-10T10:29:14.985118+0000","last_fullsized":"2024-09-10T10:29:14.985118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'15","last_scrub_stamp":"2024-09-10T10:29:14.985012+0000","last_deep_scrub":"78'15","last_deep_scrub_stamp":"2024-09-10T10:29:14.985012+0000","last_clean_scrub_stamp":"2024-09-10T10:29:14.985012+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:34.640472+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:30.332316+0000","last_change":"2024-09-10T10:29:30.332356+0000","last_active":"2024-09-10T10:29:30.332316+0000","last_peered":"2024-09-10T10:29:30.332316+0000","last_clean":"2024-09-10T10:29:30.332316+0000","last_became_active":"2024-09-10T10:26:46.192070+0000","last_became_peered":"2024-09-10T10:26:46.192070+0000","last_unstale":"2024-09-10T10:29:30.332316+0000","last_undegraded":"2024-09-10T10:29:30.332316+0000","last_fullsized":"2024-09-10T10:29:30.332316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:30.332206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:30.332206+0000","last_clean_scrub_stamp":"2024-09-10T10:29:30.332206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:49.176206+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"78'6","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:30.512067+0000","last_change":"2024-09-10T10:29:30.512092+0000","last_active":"2024-09-10T10:29:30.512067+0000","last_peered":"2024-09-10T10:29:30.512067+0000","last_clean":"2024-09-10T10:29:30.512067+0000","last_became_active":"2024-09-10T10:26:41.800151+0000","last_became_peered":"2024-09-10T10:26:41.800151+0000","last_unstale":"2024-09-10T10:29:30.512067+0000","last_undegraded":"2024-09-10T10:29:30.512067+0000","last_fullsized":"2024-09-10T10:29:30.512067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:30.511980+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:30.511980+0000","last_clean_scrub_stamp":"2024-09-10T10:29:30.511980+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:29.824111+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"78'34","reported_seq":172,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:18.956002+0000","last_change":"2024-09-10T10:29:18.956043+0000","last_active":"2024-09-10T10:29:18.956002+0000","last_peered":"2024-09-10T10:29:18.956002+0000","last_clean":"2024-09-10T10:29:18.956002+0000","last_became_active":"2024-09-10T10:26:40.793379+0000","last_became_peered":"2024-09-10T10:26:40.793379+0000","last_unstale":"2024-09-10T10:29:18.956002+0000","last_undegraded":"2024-09-10T10:29:18.956002+0000","last_fullsized":"2024-09-10T10:29:18.956002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'34","last_scrub_stamp":"2024-09-10T10:29:18.955916+0000","last_deep_scrub":"78'34","last_deep_scrub_stamp":"2024-09-10T10:29:18.955916+0000","last_clean_scrub_stamp":"2024-09-10T10:29:18.955916+0000","objects_scrubbed":6,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:30.366000+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37263,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"60'3","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:25.769790+0000","last_change":"2024-09-10T10:29:25.769839+0000","last_active":"2024-09-10T10:29:25.769790+0000","last_peered":"2024-09-10T10:29:25.769790+0000","last_clean":"2024-09-10T10:29:25.769790+0000","last_became_active":"2024-09-10T10:26:45.819416+0000","last_became_peered":"2024-09-10T10:26:45.819416+0000","last_unstale":"2024-09-10T10:29:25.769790+0000","last_undegraded":"2024-09-10T10:29:25.769790+0000","last_fullsized":"2024-09-10T10:29:25.769790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'3","last_scrub_stamp":"2024-09-10T10:29:25.769664+0000","last_deep_scrub":"60'3","last_deep_scrub_stamp":"2024-09-10T10:29:25.769664+0000","last_clean_scrub_stamp":"2024-09-10T10:29:25.769664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:37.192032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"78'11","reported_seq":146,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:18.758076+0000","last_change":"2024-09-10T10:29:18.758103+0000","last_active":"2024-09-10T10:29:18.758076+0000","last_peered":"2024-09-10T10:29:18.758076+0000","last_clean":"2024-09-10T10:29:18.758076+0000","last_became_active":"2024-09-10T10:26:41.791837+0000","last_became_peered":"2024-09-10T10:26:41.791837+0000","last_unstale":"2024-09-10T10:29:18.758076+0000","last_undegraded":"2024-09-10T10:29:18.758076+0000","last_fullsized":"2024-09-10T10:29:18.758076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'11","last_scrub_stamp":"2024-09-10T10:29:18.757992+0000","last_deep_scrub":"78'11","last_deep_scrub_stamp":"2024-09-10T10:29:18.757992+0000","last_clean_scrub_stamp":"2024-09-10T10:29:18.757992+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:30.168076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'11","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:18.751917+0000","last_change":"2024-09-10T10:29:18.751951+0000","last_active":"2024-09-10T10:29:18.751917+0000","last_peered":"2024-09-10T10:29:18.751917+0000","last_clean":"2024-09-10T10:29:18.751917+0000","last_became_active":"2024-09-10T10:26:40.791660+0000","last_became_peered":"2024-09-10T10:26:40.791660+0000","last_unstale":"2024-09-10T10:29:18.751917+0000","last_undegraded":"2024-09-10T10:29:18.751917+0000","last_fullsized":"2024-09-10T10:29:18.751917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'11","last_scrub_stamp":"2024-09-10T10:29:18.751834+0000","last_deep_scrub":"78'11","last_deep_scrub_stamp":"2024-09-10T10:29:18.751834+0000","last_clean_scrub_stamp":"2024-09-10T10:29:18.751834+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:08.144660+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":130,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:20.815847+0000","last_change":"2024-09-10T10:29:20.815878+0000","last_active":"2024-09-10T10:29:20.815847+0000","last_peered":"2024-09-10T10:29:20.815847+0000","last_clean":"2024-09-10T10:29:20.815847+0000","last_became_active":"2024-09-10T10:26:45.813301+0000","last_became_peered":"2024-09-10T10:26:45.813301+0000","last_unstale":"2024-09-10T10:29:20.815847+0000","last_undegraded":"2024-09-10T10:29:20.815847+0000","last_fullsized":"2024-09-10T10:29:20.815847+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:20.815746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:20.815746+0000","last_clean_scrub_stamp":"2024-09-10T10:29:20.815746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:49.429642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"78'9","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:20.091274+0000","last_change":"2024-09-10T10:29:20.091301+0000","last_active":"2024-09-10T10:29:20.091274+0000","last_peered":"2024-09-10T10:29:20.091274+0000","last_clean":"2024-09-10T10:29:20.091274+0000","last_became_active":"2024-09-10T10:26:41.794942+0000","last_became_peered":"2024-09-10T10:26:41.794942+0000","last_unstale":"2024-09-10T10:29:20.091274+0000","last_undegraded":"2024-09-10T10:29:20.091274+0000","last_fullsized":"2024-09-10T10:29:20.091274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:20.091164+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:20.091164+0000","last_clean_scrub_stamp":"2024-09-10T10:29:20.091164+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:21.186870+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"78'6","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:21.277690+0000","last_change":"2024-09-10T10:29:21.277723+0000","last_active":"2024-09-10T10:29:21.277690+0000","last_peered":"2024-09-10T10:29:21.277690+0000","last_clean":"2024-09-10T10:29:21.277690+0000","last_became_active":"2024-09-10T10:26:40.792413+0000","last_became_peered":"2024-09-10T10:26:40.792413+0000","last_unstale":"2024-09-10T10:29:21.277690+0000","last_undegraded":"2024-09-10T10:29:21.277690+0000","last_fullsized":"2024-09-10T10:29:21.277690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:21.277589+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:21.277589+0000","last_clean_scrub_stamp":"2024-09-10T10:29:21.277589+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:49.891484+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":152,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:33.088210+0000","last_change":"2024-09-10T10:29:33.088237+0000","last_active":"2024-09-10T10:29:33.088210+0000","last_peered":"2024-09-10T10:29:33.088210+0000","last_clean":"2024-09-10T10:29:33.088210+0000","last_became_active":"2024-09-10T10:28:24.248395+0000","last_became_peered":"2024-09-10T10:28:24.248395+0000","last_unstale":"2024-09-10T10:29:33.088210+0000","last_undegraded":"2024-09-10T10:29:33.088210+0000","last_fullsized":"2024-09-10T10:29:33.088210+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:33.088070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:33.088070+0000","last_clean_scrub_stamp":"2024-09-10T10:29:33.088070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:29.095437+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"78'10","reported_seq":125,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:29.909816+0000","last_change":"2024-09-10T10:29:29.909877+0000","last_active":"2024-09-10T10:29:29.909816+0000","last_peered":"2024-09-10T10:29:29.909816+0000","last_clean":"2024-09-10T10:29:29.909816+0000","last_became_active":"2024-09-10T10:27:24.087615+0000","last_became_peered":"2024-09-10T10:27:24.087615+0000","last_unstale":"2024-09-10T10:29:29.909816+0000","last_undegraded":"2024-09-10T10:29:29.909816+0000","last_fullsized":"2024-09-10T10:29:29.909816+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'10","last_scrub_stamp":"2024-09-10T10:29:29.909678+0000","last_deep_scrub":"78'10","last_deep_scrub_stamp":"2024-09-10T10:29:29.909678+0000","last_clean_scrub_stamp":"2024-09-10T10:29:29.909678+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:41.297030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"78'9","reported_seq":147,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:17.959349+0000","last_change":"2024-09-10T10:29:17.959405+0000","last_active":"2024-09-10T10:29:17.959349+0000","last_peered":"2024-09-10T10:29:17.959349+0000","last_clean":"2024-09-10T10:29:17.959349+0000","last_became_active":"2024-09-10T10:26:40.792576+0000","last_became_peered":"2024-09-10T10:26:40.792576+0000","last_unstale":"2024-09-10T10:29:17.959349+0000","last_undegraded":"2024-09-10T10:29:17.959349+0000","last_fullsized":"2024-09-10T10:29:17.959349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:17.959247+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:17.959247+0000","last_clean_scrub_stamp":"2024-09-10T10:29:17.959247+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:24.906169+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:31.059203+0000","last_change":"2024-09-10T10:29:31.059233+0000","last_active":"2024-09-10T10:29:31.059203+0000","last_peered":"2024-09-10T10:29:31.059203+0000","last_clean":"2024-09-10T10:29:31.059203+0000","last_became_active":"2024-09-10T10:26:46.192877+0000","last_became_peered":"2024-09-10T10:26:46.192877+0000","last_unstale":"2024-09-10T10:29:31.059203+0000","last_undegraded":"2024-09-10T10:29:31.059203+0000","last_fullsized":"2024-09-10T10:29:31.059203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:31.059114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:31.059114+0000","last_clean_scrub_stamp":"2024-09-10T10:29:31.059114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:41.855911+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:29.054535+0000","last_change":"2024-09-10T10:29:29.054588+0000","last_active":"2024-09-10T10:29:29.054535+0000","last_peered":"2024-09-10T10:29:29.054535+0000","last_clean":"2024-09-10T10:29:29.054535+0000","last_became_active":"2024-09-10T10:28:24.376079+0000","last_became_peered":"2024-09-10T10:28:24.376079+0000","last_unstale":"2024-09-10T10:29:29.054535+0000","last_undegraded":"2024-09-10T10:29:29.054535+0000","last_fullsized":"2024-09-10T10:29:29.054535+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:29.054442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:29.054442+0000","last_clean_scrub_stamp":"2024-09-10T10:29:29.054442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:41.597955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,1],"acting":[0,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"78'10","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:24.758767+0000","last_change":"2024-09-10T10:29:24.758791+0000","last_active":"2024-09-10T10:29:24.758767+0000","last_peered":"2024-09-10T10:29:24.758767+0000","last_clean":"2024-09-10T10:29:24.758767+0000","last_became_active":"2024-09-10T10:26:41.795773+0000","last_became_peered":"2024-09-10T10:26:41.795773+0000","last_unstale":"2024-09-10T10:29:24.758767+0000","last_undegraded":"2024-09-10T10:29:24.758767+0000","last_fullsized":"2024-09-10T10:29:24.758767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'10","last_scrub_stamp":"2024-09-10T10:29:24.758686+0000","last_deep_scrub":"78'10","last_deep_scrub_stamp":"2024-09-10T10:29:24.758686+0000","last_clean_scrub_stamp":"2024-09-10T10:29:24.758686+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:36.471238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"78'4","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:18.250728+0000","last_change":"2024-09-10T10:29:18.250783+0000","last_active":"2024-09-10T10:29:18.250728+0000","last_peered":"2024-09-10T10:29:18.250728+0000","last_clean":"2024-09-10T10:29:18.250728+0000","last_became_active":"2024-09-10T10:26:40.783197+0000","last_became_peered":"2024-09-10T10:26:40.783197+0000","last_unstale":"2024-09-10T10:29:18.250728+0000","last_undegraded":"2024-09-10T10:29:18.250728+0000","last_fullsized":"2024-09-10T10:29:18.250728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'4","last_scrub_stamp":"2024-09-10T10:29:18.250624+0000","last_deep_scrub":"78'4","last_deep_scrub_stamp":"2024-09-10T10:29:18.250624+0000","last_clean_scrub_stamp":"2024-09-10T10:29:18.250624+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:25.197547+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:24.847884+0000","last_change":"2024-09-10T10:29:24.847920+0000","last_active":"2024-09-10T10:29:24.847884+0000","last_peered":"2024-09-10T10:29:24.847884+0000","last_clean":"2024-09-10T10:29:24.847884+0000","last_became_active":"2024-09-10T10:26:45.818528+0000","last_became_peered":"2024-09-10T10:26:45.818528+0000","last_unstale":"2024-09-10T10:29:24.847884+0000","last_undegraded":"2024-09-10T10:29:24.847884+0000","last_fullsized":"2024-09-10T10:29:24.847884+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:24.847791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:24.847791+0000","last_clean_scrub_stamp":"2024-09-10T10:29:24.847791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.447752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"78'8","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:23.888920+0000","last_change":"2024-09-10T10:29:23.888956+0000","last_active":"2024-09-10T10:29:23.888920+0000","last_peered":"2024-09-10T10:29:23.888920+0000","last_clean":"2024-09-10T10:29:23.888920+0000","last_became_active":"2024-09-10T10:26:41.800125+0000","last_became_peered":"2024-09-10T10:26:41.800125+0000","last_unstale":"2024-09-10T10:29:23.888920+0000","last_undegraded":"2024-09-10T10:29:23.888920+0000","last_fullsized":"2024-09-10T10:29:23.888920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:23.888826+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:23.888826+0000","last_clean_scrub_stamp":"2024-09-10T10:29:23.888826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:36.432339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"78'5","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:20.944451+0000","last_change":"2024-09-10T10:29:20.944481+0000","last_active":"2024-09-10T10:29:20.944451+0000","last_peered":"2024-09-10T10:29:20.944451+0000","last_clean":"2024-09-10T10:29:20.944451+0000","last_became_active":"2024-09-10T10:26:40.792485+0000","last_became_peered":"2024-09-10T10:26:40.792485+0000","last_unstale":"2024-09-10T10:29:20.944451+0000","last_undegraded":"2024-09-10T10:29:20.944451+0000","last_fullsized":"2024-09-10T10:29:20.944451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'5","last_scrub_stamp":"2024-09-10T10:29:20.944340+0000","last_deep_scrub":"78'5","last_deep_scrub_stamp":"2024-09-10T10:29:20.944340+0000","last_clean_scrub_stamp":"2024-09-10T10:29:20.944340+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:39.089196+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"78'10","reported_seq":148,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:15.959761+0000","last_change":"2024-09-10T10:29:15.959830+0000","last_active":"2024-09-10T10:29:15.959761+0000","last_peered":"2024-09-10T10:29:15.959761+0000","last_clean":"2024-09-10T10:29:15.959761+0000","last_became_active":"2024-09-10T10:26:40.791586+0000","last_became_peered":"2024-09-10T10:26:40.791586+0000","last_unstale":"2024-09-10T10:29:15.959761+0000","last_undegraded":"2024-09-10T10:29:15.959761+0000","last_fullsized":"2024-09-10T10:29:15.959761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'10","last_scrub_stamp":"2024-09-10T10:29:15.959652+0000","last_deep_scrub":"78'10","last_deep_scrub_stamp":"2024-09-10T10:29:15.959652+0000","last_clean_scrub_stamp":"2024-09-10T10:29:15.959652+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:52.182225+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"78'8","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:23.114747+0000","last_change":"2024-09-10T10:29:23.114777+0000","last_active":"2024-09-10T10:29:23.114747+0000","last_peered":"2024-09-10T10:29:23.114747+0000","last_clean":"2024-09-10T10:29:23.114747+0000","last_became_active":"2024-09-10T10:26:41.798385+0000","last_became_peered":"2024-09-10T10:26:41.798385+0000","last_unstale":"2024-09-10T10:29:23.114747+0000","last_undegraded":"2024-09-10T10:29:23.114747+0000","last_fullsized":"2024-09-10T10:29:23.114747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:23.114642+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:23.114642+0000","last_clean_scrub_stamp":"2024-09-10T10:29:23.114642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:10.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:20.843458+0000","last_change":"2024-09-10T10:29:20.843481+0000","last_active":"2024-09-10T10:29:20.843458+0000","last_peered":"2024-09-10T10:29:20.843458+0000","last_clean":"2024-09-10T10:29:20.843458+0000","last_became_active":"2024-09-10T10:26:45.809782+0000","last_became_peered":"2024-09-10T10:26:45.809782+0000","last_unstale":"2024-09-10T10:29:20.843458+0000","last_undegraded":"2024-09-10T10:29:20.843458+0000","last_fullsized":"2024-09-10T10:29:20.843458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:20.843385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:20.843385+0000","last_clean_scrub_stamp":"2024-09-10T10:29:20.843385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:22.512452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'8","reported_seq":146,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:19.923132+0000","last_change":"2024-09-10T10:29:19.923167+0000","last_active":"2024-09-10T10:29:19.923132+0000","last_peered":"2024-09-10T10:29:19.923132+0000","last_clean":"2024-09-10T10:29:19.923132+0000","last_became_active":"2024-09-10T10:26:40.788349+0000","last_became_peered":"2024-09-10T10:26:40.788349+0000","last_unstale":"2024-09-10T10:29:19.923132+0000","last_undegraded":"2024-09-10T10:29:19.923132+0000","last_fullsized":"2024-09-10T10:29:19.923132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:19.923031+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:19.923031+0000","last_clean_scrub_stamp":"2024-09-10T10:29:19.923031+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:31.345399+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"78'7","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:19.788319+0000","last_change":"2024-09-10T10:29:19.788359+0000","last_active":"2024-09-10T10:29:19.788319+0000","last_peered":"2024-09-10T10:29:19.788319+0000","last_clean":"2024-09-10T10:29:19.788319+0000","last_became_active":"2024-09-10T10:26:41.795333+0000","last_became_peered":"2024-09-10T10:26:41.795333+0000","last_unstale":"2024-09-10T10:29:19.788319+0000","last_undegraded":"2024-09-10T10:29:19.788319+0000","last_fullsized":"2024-09-10T10:29:19.788319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:19.788212+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:19.788212+0000","last_clean_scrub_stamp":"2024-09-10T10:29:19.788212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:09.181038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:16.939235+0000","last_change":"2024-09-10T10:29:16.939235+0000","last_active":"2024-09-10T10:29:16.939235+0000","last_peered":"2024-09-10T10:29:16.939235+0000","last_clean":"2024-09-10T10:29:16.939235+0000","last_became_active":"2024-09-10T10:26:45.820113+0000","last_became_peered":"2024-09-10T10:26:45.820113+0000","last_unstale":"2024-09-10T10:29:16.939235+0000","last_undegraded":"2024-09-10T10:29:16.939235+0000","last_fullsized":"2024-09-10T10:29:16.939235+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:29:16.939067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:29:16.939067+0000","last_clean_scrub_stamp":"2024-09-10T10:29:16.939067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:15.386583+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"78'9","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:23.886559+0000","last_change":"2024-09-10T10:29:23.886604+0000","last_active":"2024-09-10T10:29:23.886559+0000","last_peered":"2024-09-10T10:29:23.886559+0000","last_clean":"2024-09-10T10:29:23.886559+0000","last_became_active":"2024-09-10T10:26:41.800770+0000","last_became_peered":"2024-09-10T10:26:41.800770+0000","last_unstale":"2024-09-10T10:29:23.886559+0000","last_undegraded":"2024-09-10T10:29:23.886559+0000","last_fullsized":"2024-09-10T10:29:23.886559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:23.886458+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:23.886458+0000","last_clean_scrub_stamp":"2024-09-10T10:29:23.886458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:22.333974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"78'12","reported_seq":148,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:19.840072+0000","last_change":"2024-09-10T10:29:19.840105+0000","last_active":"2024-09-10T10:29:19.840072+0000","last_peered":"2024-09-10T10:29:19.840072+0000","last_clean":"2024-09-10T10:29:19.840072+0000","last_became_active":"2024-09-10T10:26:41.793576+0000","last_became_peered":"2024-09-10T10:26:41.793576+0000","last_unstale":"2024-09-10T10:29:19.840072+0000","last_undegraded":"2024-09-10T10:29:19.840072+0000","last_fullsized":"2024-09-10T10:29:19.840072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'12","last_scrub_stamp":"2024-09-10T10:29:19.839979+0000","last_deep_scrub":"78'12","last_deep_scrub_stamp":"2024-09-10T10:29:19.839979+0000","last_clean_scrub_stamp":"2024-09-10T10:29:19.839979+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:43.173259+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"78'8","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:29.347343+0000","last_change":"2024-09-10T10:29:29.347391+0000","last_active":"2024-09-10T10:29:29.347343+0000","last_peered":"2024-09-10T10:29:29.347343+0000","last_clean":"2024-09-10T10:29:29.347343+0000","last_became_active":"2024-09-10T10:26:41.794496+0000","last_became_peered":"2024-09-10T10:26:41.794496+0000","last_unstale":"2024-09-10T10:29:29.347343+0000","last_undegraded":"2024-09-10T10:29:29.347343+0000","last_fullsized":"2024-09-10T10:29:29.347343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:29.347248+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:29.347248+0000","last_clean_scrub_stamp":"2024-09-10T10:29:29.347248+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:13.789031+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"78'7","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:24.098233+0000","last_change":"2024-09-10T10:29:24.098277+0000","last_active":"2024-09-10T10:29:24.098233+0000","last_peered":"2024-09-10T10:29:24.098233+0000","last_clean":"2024-09-10T10:29:24.098233+0000","last_became_active":"2024-09-10T10:26:41.801701+0000","last_became_peered":"2024-09-10T10:26:41.801701+0000","last_unstale":"2024-09-10T10:29:24.098233+0000","last_undegraded":"2024-09-10T10:29:24.098233+0000","last_fullsized":"2024-09-10T10:29:24.098233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:24.098133+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:24.098133+0000","last_clean_scrub_stamp":"2024-09-10T10:29:24.098133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:20.063379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"78'4","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:15.937543+0000","last_change":"2024-09-10T10:29:15.937565+0000","last_active":"2024-09-10T10:29:15.937543+0000","last_peered":"2024-09-10T10:29:15.937543+0000","last_clean":"2024-09-10T10:29:15.937543+0000","last_became_active":"2024-09-10T10:26:41.802578+0000","last_became_peered":"2024-09-10T10:26:41.802578+0000","last_unstale":"2024-09-10T10:29:15.937543+0000","last_undegraded":"2024-09-10T10:29:15.937543+0000","last_fullsized":"2024-09-10T10:29:15.937543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'4","last_scrub_stamp":"2024-09-10T10:29:15.937476+0000","last_deep_scrub":"78'4","last_deep_scrub_stamp":"2024-09-10T10:29:15.937476+0000","last_clean_scrub_stamp":"2024-09-10T10:29:15.937476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:31.574423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"78'6","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:13.641318+0000","last_change":"2024-09-10T10:29:13.641363+0000","last_active":"2024-09-10T10:29:13.641318+0000","last_peered":"2024-09-10T10:29:13.641318+0000","last_clean":"2024-09-10T10:29:13.641318+0000","last_became_active":"2024-09-10T10:26:41.792221+0000","last_became_peered":"2024-09-10T10:26:41.792221+0000","last_unstale":"2024-09-10T10:29:13.641318+0000","last_undegraded":"2024-09-10T10:29:13.641318+0000","last_fullsized":"2024-09-10T10:29:13.641318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:13.641240+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:13.641240+0000","last_clean_scrub_stamp":"2024-09-10T10:29:13.641240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:32.473985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"78'15","reported_seq":151,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:21.935424+0000","last_change":"2024-09-10T10:29:21.935465+0000","last_active":"2024-09-10T10:29:21.935424+0000","last_peered":"2024-09-10T10:29:21.935424+0000","last_clean":"2024-09-10T10:29:21.935424+0000","last_became_active":"2024-09-10T10:26:41.795600+0000","last_became_peered":"2024-09-10T10:26:41.795600+0000","last_unstale":"2024-09-10T10:29:21.935424+0000","last_undegraded":"2024-09-10T10:29:21.935424+0000","last_fullsized":"2024-09-10T10:29:21.935424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'15","last_scrub_stamp":"2024-09-10T10:29:21.935320+0000","last_deep_scrub":"78'15","last_deep_scrub_stamp":"2024-09-10T10:29:21.935320+0000","last_clean_scrub_stamp":"2024-09-10T10:29:21.935320+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:23.031026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"78'2","reported_seq":148,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:34.079919+0000","last_change":"2024-09-10T10:29:34.079956+0000","last_active":"2024-09-10T10:29:34.079919+0000","last_peered":"2024-09-10T10:29:34.079919+0000","last_clean":"2024-09-10T10:29:34.079919+0000","last_became_active":"2024-09-10T10:28:59.446546+0000","last_became_peered":"2024-09-10T10:28:59.446546+0000","last_unstale":"2024-09-10T10:29:34.079919+0000","last_undegraded":"2024-09-10T10:29:34.079919+0000","last_fullsized":"2024-09-10T10:29:34.079919+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T10:29:34.079818+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T10:29:34.079818+0000","last_clean_scrub_stamp":"2024-09-10T10:29:34.079818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:29.275745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"78'6","reported_seq":147,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:32.348672+0000","last_change":"2024-09-10T10:29:32.348700+0000","last_active":"2024-09-10T10:29:32.348672+0000","last_peered":"2024-09-10T10:29:32.348672+0000","last_clean":"2024-09-10T10:29:32.348672+0000","last_became_active":"2024-09-10T10:27:23.957958+0000","last_became_peered":"2024-09-10T10:27:23.957958+0000","last_unstale":"2024-09-10T10:29:32.348672+0000","last_undegraded":"2024-09-10T10:29:32.348672+0000","last_fullsized":"2024-09-10T10:29:32.348672+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:32.348568+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:32.348568+0000","last_clean_scrub_stamp":"2024-09-10T10:29:32.348568+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:51:13.848999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"78'8","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:14.960212+0000","last_change":"2024-09-10T10:29:14.960250+0000","last_active":"2024-09-10T10:29:14.960212+0000","last_peered":"2024-09-10T10:29:14.960212+0000","last_clean":"2024-09-10T10:29:14.960212+0000","last_became_active":"2024-09-10T10:26:41.802676+0000","last_became_peered":"2024-09-10T10:26:41.802676+0000","last_unstale":"2024-09-10T10:29:14.960212+0000","last_undegraded":"2024-09-10T10:29:14.960212+0000","last_fullsized":"2024-09-10T10:29:14.960212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:14.960085+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:14.960085+0000","last_clean_scrub_stamp":"2024-09-10T10:29:14.960085+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:42:09.430900+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"78'10","reported_seq":146,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:25.360354+0000","last_change":"2024-09-10T10:29:25.360381+0000","last_active":"2024-09-10T10:29:25.360354+0000","last_peered":"2024-09-10T10:29:25.360354+0000","last_clean":"2024-09-10T10:29:25.360354+0000","last_became_active":"2024-09-10T10:26:41.789887+0000","last_became_peered":"2024-09-10T10:26:41.789887+0000","last_unstale":"2024-09-10T10:29:25.360354+0000","last_undegraded":"2024-09-10T10:29:25.360354+0000","last_fullsized":"2024-09-10T10:29:25.360354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'10","last_scrub_stamp":"2024-09-10T10:29:25.360258+0000","last_deep_scrub":"78'10","last_deep_scrub_stamp":"2024-09-10T10:29:25.360258+0000","last_clean_scrub_stamp":"2024-09-10T10:29:25.360258+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.960219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"78'11","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:19.093272+0000","last_change":"2024-09-10T10:29:19.093324+0000","last_active":"2024-09-10T10:29:19.093272+0000","last_peered":"2024-09-10T10:29:19.093272+0000","last_clean":"2024-09-10T10:29:19.093272+0000","last_became_active":"2024-09-10T10:26:41.794269+0000","last_became_peered":"2024-09-10T10:26:41.794269+0000","last_unstale":"2024-09-10T10:29:19.093272+0000","last_undegraded":"2024-09-10T10:29:19.093272+0000","last_fullsized":"2024-09-10T10:29:19.093272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'11","last_scrub_stamp":"2024-09-10T10:29:19.093168+0000","last_deep_scrub":"78'11","last_deep_scrub_stamp":"2024-09-10T10:29:19.093168+0000","last_clean_scrub_stamp":"2024-09-10T10:29:19.093168+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:47.707064+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"78'8","reported_seq":80,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:30.916613+0000","last_change":"2024-09-10T10:29:30.916640+0000","last_active":"2024-09-10T10:29:30.916613+0000","last_peered":"2024-09-10T10:29:30.916613+0000","last_clean":"2024-09-10T10:29:30.916613+0000","last_became_active":"2024-09-10T10:28:23.249039+0000","last_became_peered":"2024-09-10T10:28:23.249039+0000","last_unstale":"2024-09-10T10:29:30.916613+0000","last_undegraded":"2024-09-10T10:29:30.916613+0000","last_fullsized":"2024-09-10T10:29:30.916613+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:30.916525+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:30.916525+0000","last_clean_scrub_stamp":"2024-09-10T10:29:30.916525+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:51:12.416957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"78'7","reported_seq":80,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:27.865044+0000","last_change":"2024-09-10T10:29:27.865086+0000","last_active":"2024-09-10T10:29:27.865044+0000","last_peered":"2024-09-10T10:29:27.865044+0000","last_clean":"2024-09-10T10:29:27.865044+0000","last_became_active":"2024-09-10T10:28:24.244560+0000","last_became_peered":"2024-09-10T10:28:24.244560+0000","last_unstale":"2024-09-10T10:29:27.865044+0000","last_undegraded":"2024-09-10T10:29:27.865044+0000","last_fullsized":"2024-09-10T10:29:27.865044+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:27.864962+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:27.864962+0000","last_clean_scrub_stamp":"2024-09-10T10:29:27.864962+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:46.009818+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2e","version":"78'10","reported_seq":146,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:18.829981+0000","last_change":"2024-09-10T10:29:18.830007+0000","last_active":"2024-09-10T10:29:18.829981+0000","last_peered":"2024-09-10T10:29:18.829981+0000","last_clean":"2024-09-10T10:29:18.829981+0000","last_became_active":"2024-09-10T10:26:41.789909+0000","last_became_peered":"2024-09-10T10:26:41.789909+0000","last_unstale":"2024-09-10T10:29:18.829981+0000","last_undegraded":"2024-09-10T10:29:18.829981+0000","last_fullsized":"2024-09-10T10:29:18.829981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'10","last_scrub_stamp":"2024-09-10T10:29:18.829901+0000","last_deep_scrub":"78'10","last_deep_scrub_stamp":"2024-09-10T10:29:18.829901+0000","last_clean_scrub_stamp":"2024-09-10T10:29:18.829901+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:24.718875+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"78'8","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:27.359655+0000","last_change":"2024-09-10T10:29:27.359682+0000","last_active":"2024-09-10T10:29:27.359655+0000","last_peered":"2024-09-10T10:29:27.359655+0000","last_clean":"2024-09-10T10:29:27.359655+0000","last_became_active":"2024-09-10T10:26:41.793203+0000","last_became_peered":"2024-09-10T10:26:41.793203+0000","last_unstale":"2024-09-10T10:29:27.359655+0000","last_undegraded":"2024-09-10T10:29:27.359655+0000","last_fullsized":"2024-09-10T10:29:27.359655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:27.359553+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:27.359553+0000","last_clean_scrub_stamp":"2024-09-10T10:29:27.359553+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:26.671684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"78'9","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:28.351908+0000","last_change":"2024-09-10T10:29:28.351937+0000","last_active":"2024-09-10T10:29:28.351908+0000","last_peered":"2024-09-10T10:29:28.351908+0000","last_clean":"2024-09-10T10:29:28.351908+0000","last_became_active":"2024-09-10T10:26:41.794303+0000","last_became_peered":"2024-09-10T10:26:41.794303+0000","last_unstale":"2024-09-10T10:29:28.351908+0000","last_undegraded":"2024-09-10T10:29:28.351908+0000","last_fullsized":"2024-09-10T10:29:28.351908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:28.351810+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:28.351810+0000","last_clean_scrub_stamp":"2024-09-10T10:29:28.351810+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:46.496667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"78'7","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:21.749020+0000","last_change":"2024-09-10T10:29:21.749053+0000","last_active":"2024-09-10T10:29:21.749020+0000","last_peered":"2024-09-10T10:29:21.749020+0000","last_clean":"2024-09-10T10:29:21.749020+0000","last_became_active":"2024-09-10T10:26:41.781838+0000","last_became_peered":"2024-09-10T10:26:41.781838+0000","last_unstale":"2024-09-10T10:29:21.749020+0000","last_undegraded":"2024-09-10T10:29:21.749020+0000","last_fullsized":"2024-09-10T10:29:21.749020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:21.748920+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:21.748920+0000","last_clean_scrub_stamp":"2024-09-10T10:29:21.748920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:43.215732+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"78'8","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:13.950124+0000","last_change":"2024-09-10T10:29:13.950147+0000","last_active":"2024-09-10T10:29:13.950124+0000","last_peered":"2024-09-10T10:29:13.950124+0000","last_clean":"2024-09-10T10:29:13.950124+0000","last_became_active":"2024-09-10T10:26:41.797151+0000","last_became_peered":"2024-09-10T10:26:41.797151+0000","last_unstale":"2024-09-10T10:29:13.950124+0000","last_undegraded":"2024-09-10T10:29:13.950124+0000","last_fullsized":"2024-09-10T10:29:13.950124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:13.950052+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:13.950052+0000","last_clean_scrub_stamp":"2024-09-10T10:29:13.950052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:50.172624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"78'10","reported_seq":83,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:22.769298+0000","last_change":"2024-09-10T10:29:22.769340+0000","last_active":"2024-09-10T10:29:22.769298+0000","last_peered":"2024-09-10T10:29:22.769298+0000","last_clean":"2024-09-10T10:29:22.769298+0000","last_became_active":"2024-09-10T10:28:24.243088+0000","last_became_peered":"2024-09-10T10:28:24.243088+0000","last_unstale":"2024-09-10T10:29:22.769298+0000","last_undegraded":"2024-09-10T10:29:22.769298+0000","last_fullsized":"2024-09-10T10:29:22.769298+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'10","last_scrub_stamp":"2024-09-10T10:29:22.769211+0000","last_deep_scrub":"78'10","last_deep_scrub_stamp":"2024-09-10T10:29:22.769211+0000","last_clean_scrub_stamp":"2024-09-10T10:29:22.769211+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:59.749915+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"78'7","reported_seq":102,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:32.047388+0000","last_change":"2024-09-10T10:29:32.047437+0000","last_active":"2024-09-10T10:29:32.047388+0000","last_peered":"2024-09-10T10:29:32.047388+0000","last_clean":"2024-09-10T10:29:32.047388+0000","last_became_active":"2024-09-10T10:27:58.327502+0000","last_became_peered":"2024-09-10T10:27:58.327502+0000","last_unstale":"2024-09-10T10:29:32.047388+0000","last_undegraded":"2024-09-10T10:29:32.047388+0000","last_fullsized":"2024-09-10T10:29:32.047388+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:32.047288+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:32.047288+0000","last_clean_scrub_stamp":"2024-09-10T10:29:32.047288+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:16.632308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.35","version":"78'6","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:20.916831+0000","last_change":"2024-09-10T10:29:20.916861+0000","last_active":"2024-09-10T10:29:20.916831+0000","last_peered":"2024-09-10T10:29:20.916831+0000","last_clean":"2024-09-10T10:29:20.916831+0000","last_became_active":"2024-09-10T10:26:41.794266+0000","last_became_peered":"2024-09-10T10:26:41.794266+0000","last_unstale":"2024-09-10T10:29:20.916831+0000","last_undegraded":"2024-09-10T10:29:20.916831+0000","last_fullsized":"2024-09-10T10:29:20.916831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:20.916738+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:20.916738+0000","last_clean_scrub_stamp":"2024-09-10T10:29:20.916738+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:49.530634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"78'16","reported_seq":162,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:29.869527+0000","last_change":"2024-09-10T10:29:29.869585+0000","last_active":"2024-09-10T10:29:29.869527+0000","last_peered":"2024-09-10T10:29:29.869527+0000","last_clean":"2024-09-10T10:29:29.869527+0000","last_became_active":"2024-09-10T10:28:40.344766+0000","last_became_peered":"2024-09-10T10:28:40.344766+0000","last_unstale":"2024-09-10T10:29:29.869527+0000","last_undegraded":"2024-09-10T10:29:29.869527+0000","last_fullsized":"2024-09-10T10:29:29.869527+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'16","last_scrub_stamp":"2024-09-10T10:29:29.869427+0000","last_deep_scrub":"78'16","last_deep_scrub_stamp":"2024-09-10T10:29:29.869427+0000","last_clean_scrub_stamp":"2024-09-10T10:29:29.869427+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:48.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"78'11","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:25.446061+0000","last_change":"2024-09-10T10:29:25.446100+0000","last_active":"2024-09-10T10:29:25.446061+0000","last_peered":"2024-09-10T10:29:25.446061+0000","last_clean":"2024-09-10T10:29:25.446061+0000","last_became_active":"2024-09-10T10:26:41.797054+0000","last_became_peered":"2024-09-10T10:26:41.797054+0000","last_unstale":"2024-09-10T10:29:25.446061+0000","last_undegraded":"2024-09-10T10:29:25.446061+0000","last_fullsized":"2024-09-10T10:29:25.446061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'11","last_scrub_stamp":"2024-09-10T10:29:25.445962+0000","last_deep_scrub":"78'11","last_deep_scrub_stamp":"2024-09-10T10:29:25.445962+0000","last_clean_scrub_stamp":"2024-09-10T10:29:25.445962+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:56.748191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"78'9","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:25.952600+0000","last_change":"2024-09-10T10:29:25.952643+0000","last_active":"2024-09-10T10:29:25.952600+0000","last_peered":"2024-09-10T10:29:25.952600+0000","last_clean":"2024-09-10T10:29:25.952600+0000","last_became_active":"2024-09-10T10:26:41.801603+0000","last_became_peered":"2024-09-10T10:26:41.801603+0000","last_unstale":"2024-09-10T10:29:25.952600+0000","last_undegraded":"2024-09-10T10:29:25.952600+0000","last_fullsized":"2024-09-10T10:29:25.952600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'9","last_scrub_stamp":"2024-09-10T10:29:25.952508+0000","last_deep_scrub":"78'9","last_deep_scrub_stamp":"2024-09-10T10:29:25.952508+0000","last_clean_scrub_stamp":"2024-09-10T10:29:25.952508+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:46.517389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"78'5","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:24.485630+0000","last_change":"2024-09-10T10:29:24.485674+0000","last_active":"2024-09-10T10:29:24.485630+0000","last_peered":"2024-09-10T10:29:24.485630+0000","last_clean":"2024-09-10T10:29:24.485630+0000","last_became_active":"2024-09-10T10:26:41.790238+0000","last_became_peered":"2024-09-10T10:26:41.790238+0000","last_unstale":"2024-09-10T10:29:24.485630+0000","last_undegraded":"2024-09-10T10:29:24.485630+0000","last_fullsized":"2024-09-10T10:29:24.485630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'5","last_scrub_stamp":"2024-09-10T10:29:24.485541+0000","last_deep_scrub":"78'5","last_deep_scrub_stamp":"2024-09-10T10:29:24.485541+0000","last_clean_scrub_stamp":"2024-09-10T10:29:24.485541+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:03.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"78'6","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:24.384512+0000","last_change":"2024-09-10T10:29:24.384543+0000","last_active":"2024-09-10T10:29:24.384512+0000","last_peered":"2024-09-10T10:29:24.384512+0000","last_clean":"2024-09-10T10:29:24.384512+0000","last_became_active":"2024-09-10T10:26:41.787655+0000","last_became_peered":"2024-09-10T10:26:41.787655+0000","last_unstale":"2024-09-10T10:29:24.384512+0000","last_undegraded":"2024-09-10T10:29:24.384512+0000","last_fullsized":"2024-09-10T10:29:24.384512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'6","last_scrub_stamp":"2024-09-10T10:29:24.384422+0000","last_deep_scrub":"78'6","last_deep_scrub_stamp":"2024-09-10T10:29:24.384422+0000","last_clean_scrub_stamp":"2024-09-10T10:29:24.384422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:00.112234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"78'4","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:16.426202+0000","last_change":"2024-09-10T10:29:16.426230+0000","last_active":"2024-09-10T10:29:16.426202+0000","last_peered":"2024-09-10T10:29:16.426202+0000","last_clean":"2024-09-10T10:29:16.426202+0000","last_became_active":"2024-09-10T10:26:41.792993+0000","last_became_peered":"2024-09-10T10:26:41.792993+0000","last_unstale":"2024-09-10T10:29:16.426202+0000","last_undegraded":"2024-09-10T10:29:16.426202+0000","last_fullsized":"2024-09-10T10:29:16.426202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'4","last_scrub_stamp":"2024-09-10T10:29:16.426112+0000","last_deep_scrub":"78'4","last_deep_scrub_stamp":"2024-09-10T10:29:16.426112+0000","last_clean_scrub_stamp":"2024-09-10T10:29:16.426112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:40.538723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"78'8","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:26.348445+0000","last_change":"2024-09-10T10:29:26.348470+0000","last_active":"2024-09-10T10:29:26.348445+0000","last_peered":"2024-09-10T10:29:26.348445+0000","last_clean":"2024-09-10T10:29:26.348445+0000","last_became_active":"2024-09-10T10:26:41.792782+0000","last_became_peered":"2024-09-10T10:26:41.792782+0000","last_unstale":"2024-09-10T10:29:26.348445+0000","last_undegraded":"2024-09-10T10:29:26.348445+0000","last_fullsized":"2024-09-10T10:29:26.348445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:26.348361+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:26.348361+0000","last_clean_scrub_stamp":"2024-09-10T10:29:26.348361+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:22.313606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"78'7","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:17.013198+0000","last_change":"2024-09-10T10:29:17.013223+0000","last_active":"2024-09-10T10:29:17.013198+0000","last_peered":"2024-09-10T10:29:17.013198+0000","last_clean":"2024-09-10T10:29:17.013198+0000","last_became_active":"2024-09-10T10:26:41.792362+0000","last_became_peered":"2024-09-10T10:26:41.792362+0000","last_unstale":"2024-09-10T10:29:17.013198+0000","last_undegraded":"2024-09-10T10:29:17.013198+0000","last_fullsized":"2024-09-10T10:29:17.013198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'7","last_scrub_stamp":"2024-09-10T10:29:17.013126+0000","last_deep_scrub":"78'7","last_deep_scrub_stamp":"2024-09-10T10:29:17.013126+0000","last_clean_scrub_stamp":"2024-09-10T10:29:17.013126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:28.423210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"78'4","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:14.662528+0000","last_change":"2024-09-10T10:29:14.662566+0000","last_active":"2024-09-10T10:29:14.662528+0000","last_peered":"2024-09-10T10:29:14.662528+0000","last_clean":"2024-09-10T10:29:14.662528+0000","last_became_active":"2024-09-10T10:26:41.789457+0000","last_became_peered":"2024-09-10T10:26:41.789457+0000","last_unstale":"2024-09-10T10:29:14.662528+0000","last_undegraded":"2024-09-10T10:29:14.662528+0000","last_fullsized":"2024-09-10T10:29:14.662528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'4","last_scrub_stamp":"2024-09-10T10:29:14.662449+0000","last_deep_scrub":"78'4","last_deep_scrub_stamp":"2024-09-10T10:29:14.662449+0000","last_clean_scrub_stamp":"2024-09-10T10:29:14.662449+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:50.885021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"78'8","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T10:29:26.460716+0000","last_change":"2024-09-10T10:29:26.460745+0000","last_active":"2024-09-10T10:29:26.460716+0000","last_peered":"2024-09-10T10:29:26.460716+0000","last_clean":"2024-09-10T10:29:26.460716+0000","last_became_active":"2024-09-10T10:26:41.798045+0000","last_became_peered":"2024-09-10T10:26:41.798045+0000","last_unstale":"2024-09-10T10:29:26.460716+0000","last_undegraded":"2024-09-10T10:29:26.460716+0000","last_fullsized":"2024-09-10T10:29:26.460716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'8","last_scrub_stamp":"2024-09-10T10:29:26.460607+0000","last_deep_scrub":"78'8","last_deep_scrub_stamp":"2024-09-10T10:29:26.460607+0000","last_clean_scrub_stamp":"2024-09-10T10:29:26.460607+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:47.927420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":501,"ondisk_log_size":501,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44494,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":256,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1176,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":88988,"data_compressed":3324,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64852,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":1074512,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":974848,"data_stored":519700,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":485579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":482977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":520543,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":954368,"data_stored":483521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":483179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":1072889,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":38896,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":38053,"data_compressed":1662,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:29:36.805 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T10:29:36.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.938+0000 7fb6cd49b640 1 Processor -- start 2024-09-10T10:29:36.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.938+0000 7fb6cd49b640 1 -- start start 2024-09-10T10:29:36.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6cd49b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb6c806fdb0 0x7fb6c80c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6cd49b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb6c80c7320 0x7fb6c80ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6cd49b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6c80c99b0 con 0x7fb6c8154be0 2024-09-10T10:29:36.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6cd49b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c80c92b0 con 0x7fb6c80c7320 2024-09-10T10:29:36.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6cd49b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c815aec0 con 0x7fb6c806fdb0 2024-09-10T10:29:36.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6c77fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb6c80c7320 0x7fb6c80ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6c6ffd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fb6c8154be0 0x7fb6c815b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:55256/0 (socket says 172.21.15.120:55256) 2024-09-10T10:29:36.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6c6ffd640 1 -- 172.21.15.120:0/3120347183 learned_addr learned my addr 172.21.15.120:0/3120347183 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:36.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6c67fc640 1 --2- 172.21.15.120:0/3120347183 >> v2:172.21.15.134:3300/0 conn(0x7fb6c806fdb0 0x7fb6c80c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6c77fe640 1 -- 172.21.15.120:0/3120347183 >> v2:172.21.15.134:3300/0 conn(0x7fb6c806fdb0 msgr2=0x7fb6c80c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:36.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6c77fe640 1 --2- 172.21.15.120:0/3120347183 >> v2:172.21.15.134:3300/0 conn(0x7fb6c806fdb0 0x7fb6c80c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6c77fe640 1 -- 172.21.15.120:0/3120347183 >> v1:172.21.15.120:6789/0 conn(0x7fb6c8154be0 legacy=0x7fb6c815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:36.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6c5ffb640 1 -- 172.21.15.120:0/3120347183 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2324770894 0 0) 0x7fb6c80c99b0 con 0x7fb6c8154be0 2024-09-10T10:29:36.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6c77fe640 1 -- 172.21.15.120:0/3120347183 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c80c8cb0 con 0x7fb6c80c7320 2024-09-10T10:29:36.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.939+0000 7fb6c67fc640 1 --2- 172.21.15.120:0/3120347183 >> v2:172.21.15.134:3300/0 conn(0x7fb6c806fdb0 0x7fb6c80c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:36.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.940+0000 7fb6c77fe640 1 --2- 172.21.15.120:0/3120347183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 0x7fb6c80ca170 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb6c815b040 tx=0x7fb6c002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6968823b72dbcc11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:36.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.940+0000 7fb6c5ffb640 1 -- 172.21.15.120:0/3120347183 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0007740 con 0x7fb6c80c7320 2024-09-10T10:29:36.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.940+0000 7fb6c5ffb640 1 -- 172.21.15.120:0/3120347183 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c00078e0 con 0x7fb6c80c7320 2024-09-10T10:29:36.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.940+0000 7fb6c5ffb640 1 -- 172.21.15.120:0/3120347183 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0007cb0 con 0x7fb6c80c7320 2024-09-10T10:29:36.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.941+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3120347183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 msgr2=0x7fb6c80ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:36.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.941+0000 7fb6cd49b640 1 --2- 172.21.15.120:0/3120347183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 0x7fb6c80ca170 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb6c815b040 tx=0x7fb6c002a6f0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.941+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3120347183 shutdown_connections 2024-09-10T10:29:36.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.941+0000 7fb6cd49b640 1 --2- 172.21.15.120:0/3120347183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 0x7fb6c80ca170 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.941+0000 7fb6cd49b640 1 --2- 172.21.15.120:0/3120347183 >> v2:172.21.15.134:3300/0 conn(0x7fb6c806fdb0 0x7fb6c80c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.941+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3120347183 >> 172.21.15.120:0/3120347183 conn(0x7fb6c80680e0 msgr2=0x7fb6c8070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:36.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.941+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3120347183 shutdown_connections 2024-09-10T10:29:36.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.941+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3120347183 wait complete. 2024-09-10T10:29:36.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.942+0000 7fb6cd49b640 1 Processor -- start 2024-09-10T10:29:36.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.942+0000 7fb6cd49b640 1 -- start start 2024-09-10T10:29:36.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6cd49b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c806fdb0 0x7fb6c8159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6cd49b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 0x7fb6c8147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6cd49b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c8154be0 0x7fb6c81f06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6cd49b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8154fc0 con 0x7fb6c80c7320 2024-09-10T10:29:36.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6cd49b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c815ab50 con 0x7fb6c8154be0 2024-09-10T10:29:36.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6cd49b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c8155140 con 0x7fb6c806fdb0 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 0x7fb6c8147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 0x7fb6c8147ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39132/0 (socket says 172.21.15.120:39132) 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c67fc640 1 -- 172.21.15.120:0/3984554400 learned_addr learned my addr 172.21.15.120:0/3984554400 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c77fe640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c8154be0 0x7fb6c81f06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c6ffd640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c806fdb0 0x7fb6c8159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c67fc640 1 -- 172.21.15.120:0/3984554400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c806fdb0 msgr2=0x7fb6c8159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c67fc640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c806fdb0 0x7fb6c8159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c67fc640 1 -- 172.21.15.120:0/3984554400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c8154be0 msgr2=0x7fb6c81f06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c67fc640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c8154be0 0x7fb6c81f06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c67fc640 1 -- 172.21.15.120:0/3984554400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c81f4780 con 0x7fb6c80c7320 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c77fe640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c8154be0 0x7fb6c81f06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c6ffd640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c806fdb0 0x7fb6c8159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6c67fc640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 0x7fb6c8147ff0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00c7f0 tx=0x7fb6bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:36.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6af7fe640 1 -- 172.21.15.120:0/3984554400 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc015070 con 0x7fb6c80c7320 2024-09-10T10:29:36.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6af7fe640 1 -- 172.21.15.120:0/3984554400 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6bc007a30 con 0x7fb6c80c7320 2024-09-10T10:29:36.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3984554400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c81f3920 con 0x7fb6c80c7320 2024-09-10T10:29:36.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.943+0000 7fb6af7fe640 1 -- 172.21.15.120:0/3984554400 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc007e00 con 0x7fb6c80c7320 2024-09-10T10:29:36.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.944+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3984554400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c81f3ee0 con 0x7fb6c80c7320 2024-09-10T10:29:36.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.945+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3984554400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c806c5b0 con 0x7fb6c80c7320 2024-09-10T10:29:36.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.945+0000 7fb6af7fe640 1 -- 172.21.15.120:0/3984554400 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6bc018070 con 0x7fb6c80c7320 2024-09-10T10:29:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.946+0000 7fb6af7fe640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb6b406be30 0x7fb6b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.946+0000 7fb6af7fe640 1 -- 172.21.15.120:0/3984554400 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fb6bc08b370 con 0x7fb6c80c7320 2024-09-10T10:29:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.946+0000 7fb6c6ffd640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb6b406be30 0x7fb6b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.946+0000 7fb6c6ffd640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb6b406be30 0x7fb6b406e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8000f60 tx=0x7fb6b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:36.950+0000 7fb6af7fe640 1 -- 172.21.15.120:0/3984554400 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6bc059790 con 0x7fb6c80c7320 2024-09-10T10:29:37.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.122+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3984554400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fb6c80c5770 con 0x7fb6c80c7320 2024-09-10T10:29:37.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.126+0000 7fb6af7fe640 1 -- 172.21.15.120:0/3984554400 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fb6bc059090 con 0x7fb6c80c7320 2024-09-10T10:29:37.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.128+0000 7fb6af7fe640 1 -- 172.21.15.120:0/3984554400 <== mon.0 v2:172.21.15.120:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb6bc00ce40 con 0x7fb6c80c7320 2024-09-10T10:29:37.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3984554400 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb6b406be30 msgr2=0x7fb6b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb6b406be30 0x7fb6b406e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8000f60 tx=0x7fb6b8009040 comp rx=0 tx=0).stop 2024-09-10T10:29:37.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3984554400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 msgr2=0x7fb6c8147ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 0x7fb6c8147ff0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00c7f0 tx=0x7fb6bc00cbf0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3984554400 shutdown_connections 2024-09-10T10:29:37.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb6b406be30 0x7fb6b406e2f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6c8154be0 0x7fb6c81f06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6c80c7320 0x7fb6c8147ff0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 --2- 172.21.15.120:0/3984554400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6c806fdb0 0x7fb6c8159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3984554400 >> 172.21.15.120:0/3984554400 conn(0x7fb6c80680e0 msgr2=0x7fb6c80c68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:37.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3984554400 shutdown_connections 2024-09-10T10:29:37.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.129+0000 7fb6cd49b640 1 -- 172.21.15.120:0/3984554400 wait complete. 2024-09-10T10:29:37.142 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:29:37.143 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:29:37.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.243+0000 7ff488265640 1 Processor -- start 2024-09-10T10:29:37.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.244+0000 7ff488265640 1 -- start start 2024-09-10T10:29:37.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.244+0000 7ff488265640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff48015d9a0 0x7ff4800ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff488265640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff48015ed70 0x7ff4800ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff488265640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff48006d2f0 con 0x7ff4800c9b30 2024-09-10T10:29:37.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff488265640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff48006de10 con 0x7ff48015ed70 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff488265640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff48006d570 con 0x7ff48015d9a0 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff485800640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff4800c9b30 0x7ff48015e660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:55262/0 (socket says 172.21.15.120:55262) 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff485800640 1 -- 172.21.15.120:0/3291322587 learned_addr learned my addr 172.21.15.120:0/3291322587 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff486802640 1 --2- 172.21.15.120:0/3291322587 >> v2:172.21.15.120:3300/0 conn(0x7ff48015ed70 0x7ff4800ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff486001640 1 --2- 172.21.15.120:0/3291322587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff48015d9a0 0x7ff4800ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff484fff640 1 -- 172.21.15.120:0/3291322587 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3464936767 0 0) 0x7ff474001110 con 0x7ff4800c9b30 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff484fff640 1 -- 172.21.15.120:0/3291322587 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff46c003520 con 0x7ff4800c9b30 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff486802640 1 -- 172.21.15.120:0/3291322587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff48015d9a0 msgr2=0x7ff4800ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff486802640 1 --2- 172.21.15.120:0/3291322587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff48015d9a0 0x7ff4800ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff486802640 1 -- 172.21.15.120:0/3291322587 >> v1:172.21.15.120:6789/0 conn(0x7ff4800c9b30 legacy=0x7ff48015e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.245+0000 7ff486802640 1 -- 172.21.15.120:0/3291322587 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48006d850 con 0x7ff48015ed70 2024-09-10T10:29:37.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.246+0000 7ff486802640 1 --2- 172.21.15.120:0/3291322587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015ed70 0x7ff4800ce640 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff478009d90 tx=0x7ff47802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b920f47760a29c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.246+0000 7ff484fff640 1 -- 172.21.15.120:0/3291322587 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478007740 con 0x7ff48015ed70 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.246+0000 7ff484fff640 1 -- 172.21.15.120:0/3291322587 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff4780078e0 con 0x7ff48015ed70 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.246+0000 7ff484fff640 1 -- 172.21.15.120:0/3291322587 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478007cd0 con 0x7ff48015ed70 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.247+0000 7ff488265640 1 -- 172.21.15.120:0/3291322587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015ed70 msgr2=0x7ff4800ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.247+0000 7ff488265640 1 --2- 172.21.15.120:0/3291322587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015ed70 0x7ff4800ce640 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff478009d90 tx=0x7ff47802a660 comp rx=0 tx=0).stop 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.247+0000 7ff488265640 1 -- 172.21.15.120:0/3291322587 shutdown_connections 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.247+0000 7ff488265640 1 --2- 172.21.15.120:0/3291322587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015ed70 0x7ff4800ce640 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.247+0000 7ff488265640 1 --2- 172.21.15.120:0/3291322587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff48015d9a0 0x7ff4800ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.247+0000 7ff488265640 1 -- 172.21.15.120:0/3291322587 >> 172.21.15.120:0/3291322587 conn(0x7ff4800715e0 msgr2=0x7ff4800719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.247+0000 7ff488265640 1 -- 172.21.15.120:0/3291322587 shutdown_connections 2024-09-10T10:29:37.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.247+0000 7ff488265640 1 -- 172.21.15.120:0/3291322587 wait complete. 2024-09-10T10:29:37.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.248+0000 7ff488265640 1 Processor -- start 2024-09-10T10:29:37.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.248+0000 7ff488265640 1 -- start start 2024-09-10T10:29:37.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff488265640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4800c9b30 0x7ff4801545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff488265640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015d9a0 0x7ff480155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff488265640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff48015ed70 0x7ff480156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff488265640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4800ca9e0 con 0x7ff48015d9a0 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff488265640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff48015e140 con 0x7ff4800c9b30 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff488265640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff48015dd80 con 0x7ff48015ed70 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff485800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015d9a0 0x7ff480155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff485800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015d9a0 0x7ff480155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39140/0 (socket says 172.21.15.120:39140) 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff485800640 1 -- 172.21.15.120:0/3191733522 learned_addr learned my addr 172.21.15.120:0/3191733522 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff486001640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4800c9b30 0x7ff4801545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.249+0000 7ff486802640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff48015ed70 0x7ff480156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff485800640 1 -- 172.21.15.120:0/3191733522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff48015ed70 msgr2=0x7ff480156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff485800640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff48015ed70 0x7ff480156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff485800640 1 -- 172.21.15.120:0/3191733522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4800c9b30 msgr2=0x7ff4801545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff485800640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4800c9b30 0x7ff4801545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff485800640 1 -- 172.21.15.120:0/3191733522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4801f1040 con 0x7ff48015d9a0 2024-09-10T10:29:37.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff485800640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015d9a0 0x7ff480155490 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff47400c3d0 tx=0x7ff47400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:37.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff4727fc640 1 -- 172.21.15.120:0/3191733522 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474017070 con 0x7ff48015d9a0 2024-09-10T10:29:37.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff4727fc640 1 -- 172.21.15.120:0/3191733522 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff474005ce0 con 0x7ff48015d9a0 2024-09-10T10:29:37.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff488265640 1 -- 172.21.15.120:0/3191733522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4801e6550 con 0x7ff48015d9a0 2024-09-10T10:29:37.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff4727fc640 1 -- 172.21.15.120:0/3191733522 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474007330 con 0x7ff48015d9a0 2024-09-10T10:29:37.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.250+0000 7ff488265640 1 -- 172.21.15.120:0/3191733522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4801f12b0 con 0x7ff48015d9a0 2024-09-10T10:29:37.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.251+0000 7ff488265640 1 -- 172.21.15.120:0/3191733522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff480070b90 con 0x7ff48015d9a0 2024-09-10T10:29:37.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.252+0000 7ff4727fc640 1 -- 172.21.15.120:0/3191733522 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff47401a070 con 0x7ff48015d9a0 2024-09-10T10:29:37.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.252+0000 7ff4727fc640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff46406be60 0x7ff46406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.252+0000 7ff4727fc640 1 -- 172.21.15.120:0/3191733522 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7ff47408a760 con 0x7ff48015d9a0 2024-09-10T10:29:37.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.252+0000 7ff486001640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff46406be60 0x7ff46406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.252+0000 7ff486001640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff46406be60 0x7ff46406e320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009aa0 tx=0x7ff47c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:37.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.256+0000 7ff4727fc640 1 -- 172.21.15.120:0/3191733522 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff48006d2f0 con 0x7ff48015d9a0 2024-09-10T10:29:37.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.463+0000 7ff488265640 1 -- 172.21.15.120:0/3191733522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4800c7770 con 0x7ff48015d9a0 2024-09-10T10:29:37.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.464+0000 7ff4727fc640 1 -- 172.21.15.120:0/3191733522 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6962 (secure 0 0 0) 0x7ff474058530 con 0x7ff48015d9a0 2024-09-10T10:29:37.465 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:37.465 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":18,"btime":"2024-09-10T10:27:26:951215+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:26:41.113510+0000","modified":"2024-09-10T10:27:26.951208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4303,"mds_2":4288,"mds_3":4284,"mds_4":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":4,"incarnation":17,"state":"up:active","state_seq":14,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":13,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4281,4284,4288,4303,4306]},"id":1}]} 2024-09-10T10:29:37.466 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 18 2024-09-10T10:29:37.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 -- 172.21.15.120:0/3191733522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff46406be60 msgr2=0x7ff46406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff46406be60 0x7ff46406e320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009aa0 tx=0x7ff47c009190 comp rx=0 tx=0).stop 2024-09-10T10:29:37.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 -- 172.21.15.120:0/3191733522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015d9a0 msgr2=0x7ff480155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015d9a0 0x7ff480155490 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff47400c3d0 tx=0x7ff47400c890 comp rx=0 tx=0).stop 2024-09-10T10:29:37.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 -- 172.21.15.120:0/3191733522 shutdown_connections 2024-09-10T10:29:37.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7ff46406be60 0x7ff46406e320 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff48015ed70 0x7ff480156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff48015d9a0 0x7ff480155490 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 --2- 172.21.15.120:0/3191733522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4800c9b30 0x7ff4801545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 -- 172.21.15.120:0/3191733522 >> 172.21.15.120:0/3191733522 conn(0x7ff4800715e0 msgr2=0x7ff4800ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:37.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 -- 172.21.15.120:0/3191733522 shutdown_connections 2024-09-10T10:29:37.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.466+0000 7ff488265640 1 -- 172.21.15.120:0/3191733522 wait complete. 2024-09-10T10:29:37.479 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:29:37.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.577+0000 7fb1fcfed640 1 Processor -- start 2024-09-10T10:29:37.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.577+0000 7fb1fcfed640 1 -- start start 2024-09-10T10:29:37.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.578+0000 7fb1fcfed640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb1f80c72e0 0x7fb1f806d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.578+0000 7fb1fcfed640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f80c4b80 con 0x7fb1f8151b30 2024-09-10T10:29:37.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.578+0000 7fb1fcfed640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f806ce50 con 0x7fb1f8157a20 2024-09-10T10:29:37.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.578+0000 7fb1fcfed640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb1f80c45c0 con 0x7fb1f80c72e0 2024-09-10T10:29:37.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.578+0000 7fb1f659c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fb1f8151b30 0x7fb1f80c60c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:45580/0 (socket says 172.21.15.120:45580) 2024-09-10T10:29:37.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.578+0000 7fb1f659c640 1 -- 172.21.15.120:0/3109641095 learned_addr learned my addr 172.21.15.120:0/3109641095 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:37.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.578+0000 7fb1effff640 1 --2- 172.21.15.120:0/3109641095 >> v2:172.21.15.134:3300/0 conn(0x7fb1f80c72e0 0x7fb1f806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.579+0000 7fb1f5d9b640 1 -- 172.21.15.120:0/3109641095 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1913778553 0 0) 0x7fb1e8001110 con 0x7fb1f8151b30 2024-09-10T10:29:37.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.579+0000 7fb1f5d9b640 1 -- 172.21.15.120:0/3109641095 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1e4003520 con 0x7fb1f8151b30 2024-09-10T10:29:37.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.579+0000 7fb1effff640 1 -- 172.21.15.120:0/3109641095 >> v1:172.21.15.134:6789/0 conn(0x7fb1f8157a20 legacy=0x7fb1f8161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.579+0000 7fb1effff640 1 -- 172.21.15.120:0/3109641095 >> v1:172.21.15.120:6789/0 conn(0x7fb1f8151b30 legacy=0x7fb1f80c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.579+0000 7fb1effff640 1 -- 172.21.15.120:0/3109641095 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f80c8bc0 con 0x7fb1f80c72e0 2024-09-10T10:29:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1effff640 1 --2- 172.21.15.120:0/3109641095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1f80c72e0 0x7fb1f806d980 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00097a0 tx=0x7fb1e002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=220f8995cbe5ebf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1f5d9b640 1 -- 172.21.15.120:0/3109641095 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0037070 con 0x7fb1f80c72e0 2024-09-10T10:29:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1f5d9b640 1 -- 172.21.15.120:0/3109641095 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb1e0032440 con 0x7fb1f80c72e0 2024-09-10T10:29:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1f5d9b640 1 -- 172.21.15.120:0/3109641095 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00327f0 con 0x7fb1f80c72e0 2024-09-10T10:29:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1fcfed640 1 -- 172.21.15.120:0/3109641095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1f80c72e0 msgr2=0x7fb1f806d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1fcfed640 1 --2- 172.21.15.120:0/3109641095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1f80c72e0 0x7fb1f806d980 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00097a0 tx=0x7fb1e002a230 comp rx=0 tx=0).stop 2024-09-10T10:29:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1fcfed640 1 -- 172.21.15.120:0/3109641095 shutdown_connections 2024-09-10T10:29:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1fcfed640 1 --2- 172.21.15.120:0/3109641095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1f80c72e0 0x7fb1f806d980 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1fcfed640 1 -- 172.21.15.120:0/3109641095 >> 172.21.15.120:0/3109641095 conn(0x7fb1f8067950 msgr2=0x7fb1f8067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1fcfed640 1 -- 172.21.15.120:0/3109641095 shutdown_connections 2024-09-10T10:29:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.580+0000 7fb1fcfed640 1 -- 172.21.15.120:0/3109641095 wait complete. 2024-09-10T10:29:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1fcfed640 1 Processor -- start 2024-09-10T10:29:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1fcfed640 1 -- start start 2024-09-10T10:29:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1fcfed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1f80c72e0 0x7fb1f81f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1fcfed640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1f8151b30 0x7fb1f81dfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1fcfed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1f8157a20 0x7fb1f80caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1fcfed640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f806d3c0 con 0x7fb1f80c72e0 2024-09-10T10:29:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1fcfed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f80c79c0 con 0x7fb1f8157a20 2024-09-10T10:29:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1fcfed640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f806d540 con 0x7fb1f8151b30 2024-09-10T10:29:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1f659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1f80c72e0 0x7fb1f81f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1f659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1f80c72e0 0x7fb1f81f2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46868/0 (socket says 172.21.15.120:46868) 2024-09-10T10:29:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.582+0000 7fb1f659c640 1 -- 172.21.15.120:0/4219287827 learned_addr learned my addr 172.21.15.120:0/4219287827 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1effff640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1f8151b30 0x7fb1f81dfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1f6d9d640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1f8157a20 0x7fb1f80caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1f659c640 1 -- 172.21.15.120:0/4219287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1f8151b30 msgr2=0x7fb1f81dfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1f659c640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1f8151b30 0x7fb1f81dfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1f659c640 1 -- 172.21.15.120:0/4219287827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1f8157a20 msgr2=0x7fb1f80caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1f659c640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1f8157a20 0x7fb1f80caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1f659c640 1 -- 172.21.15.120:0/4219287827 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f81ee810 con 0x7fb1f80c72e0 2024-09-10T10:29:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1f659c640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1f80c72e0 0x7fb1f81f2580 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800c760 tx=0x7fb1e800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1eeffd640 1 -- 172.21.15.120:0/4219287827 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8017070 con 0x7fb1f80c72e0 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1eeffd640 1 -- 172.21.15.120:0/4219287827 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb1e8005ab0 con 0x7fb1f80c72e0 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1fcfed640 1 -- 172.21.15.120:0/4219287827 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f806c8c0 con 0x7fb1f80c72e0 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1eeffd640 1 -- 172.21.15.120:0/4219287827 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8007070 con 0x7fb1f80c72e0 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.583+0000 7fb1fcfed640 1 -- 172.21.15.120:0/4219287827 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f81eed90 con 0x7fb1f80c72e0 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.584+0000 7fb1fcfed640 1 -- 172.21.15.120:0/4219287827 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f806de40 con 0x7fb1f80c72e0 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.585+0000 7fb1eeffd640 1 -- 172.21.15.120:0/4219287827 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb1e801a070 con 0x7fb1f80c72e0 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.585+0000 7fb1eeffd640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb1d806be30 0x7fb1d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.585+0000 7fb1eeffd640 1 -- 172.21.15.120:0/4219287827 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fb1e808a800 con 0x7fb1f80c72e0 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.585+0000 7fb1effff640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb1d806be30 0x7fb1d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.585+0000 7fb1effff640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb1d806be30 0x7fb1d806e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0034000 tx=0x7fb1e002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.589+0000 7fb1eeffd640 1 -- 172.21.15.120:0/4219287827 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb1f806de40 con 0x7fb1f80c72e0 2024-09-10T10:29:37.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.790+0000 7fb1fcfed640 1 -- 172.21.15.120:0/4219287827 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1f80c9d30 con 0x7fb1f80c72e0 2024-09-10T10:29:37.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.790+0000 7fb1eeffd640 1 -- 172.21.15.120:0/4219287827 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6962 (secure 0 0 0) 0x7fb1f80c4b80 con 0x7fb1f80c72e0 2024-09-10T10:29:37.791 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:37.791 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":18,"btime":"2024-09-10T10:27:26:951215+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/73921293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":73921293},{"type":"v1","addr":"172.21.15.134:6837","nonce":73921293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:26:41.113510+0000","modified":"2024-09-10T10:27:26.951208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4303,"mds_2":4288,"mds_3":4284,"mds_4":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":4,"incarnation":17,"state":"up:active","state_seq":14,"addr":"172.21.15.120:6836/2787145148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2787145148},{"type":"v1","addr":"172.21.15.120:6836","nonce":2787145148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":3,"incarnation":15,"state":"up:active","state_seq":13,"addr":"172.21.15.120:6839/1710396335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1710396335},{"type":"v1","addr":"172.21.15.120:6839","nonce":1710396335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":11,"addr":"172.21.15.120:6837/3208935627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3208935627},{"type":"v1","addr":"172.21.15.120:6837","nonce":3208935627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6834/231686968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":231686968},{"type":"v1","addr":"172.21.15.134:6834","nonce":231686968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6835/1434442356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":1434442356},{"type":"v1","addr":"172.21.15.134:6835","nonce":1434442356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4281,4284,4288,4303,4306]},"id":1}]} 2024-09-10T10:29:37.792 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 18 2024-09-10T10:29:37.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.792+0000 7fb1fcfed640 1 -- 172.21.15.120:0/4219287827 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb1d806be30 msgr2=0x7fb1d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.792+0000 7fb1fcfed640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb1d806be30 0x7fb1d806e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0034000 tx=0x7fb1e002e000 comp rx=0 tx=0).stop 2024-09-10T10:29:37.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.792+0000 7fb1fcfed640 1 -- 172.21.15.120:0/4219287827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1f80c72e0 msgr2=0x7fb1f81f2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.792+0000 7fb1fcfed640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1f80c72e0 0x7fb1f81f2580 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800c760 tx=0x7fb1e800cc20 comp rx=0 tx=0).stop 2024-09-10T10:29:37.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.792+0000 7fb1fcfed640 1 -- 172.21.15.120:0/4219287827 shutdown_connections 2024-09-10T10:29:37.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.792+0000 7fb1fcfed640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fb1d806be30 0x7fb1d806e2f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.792+0000 7fb1fcfed640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1f8157a20 0x7fb1f80caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.793+0000 7fb1fcfed640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1f8151b30 0x7fb1f81dfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.793+0000 7fb1fcfed640 1 --2- 172.21.15.120:0/4219287827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1f80c72e0 0x7fb1f81f2580 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.793+0000 7fb1fcfed640 1 -- 172.21.15.120:0/4219287827 >> 172.21.15.120:0/4219287827 conn(0x7fb1f8067950 msgr2=0x7fb1f8156810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:37.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.793+0000 7fb1fcfed640 1 -- 172.21.15.120:0/4219287827 shutdown_connections 2024-09-10T10:29:37.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.793+0000 7fb1fcfed640 1 -- 172.21.15.120:0/4219287827 wait complete. 2024-09-10T10:29:37.807 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:29:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.904+0000 7fe589d9f640 1 Processor -- start 2024-09-10T10:29:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.904+0000 7fe589d9f640 1 -- start start 2024-09-10T10:29:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.905+0000 7fe589d9f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe5840cd210 0x7fe5840ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.905+0000 7fe589d9f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe5840cd5f0 0x7fe58406c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.905+0000 7fe589d9f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5840ca900 con 0x7fe58406c830 2024-09-10T10:29:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.905+0000 7fe589d9f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe5840c8800 con 0x7fe5840cd210 2024-09-10T10:29:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.905+0000 7fe589d9f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe5840c7910 con 0x7fe5840cd5f0 2024-09-10T10:29:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.905+0000 7fe588d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe5840cd210 0x7fe5840ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.905+0000 7fe588d9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe5840cd210 0x7fe5840ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46882/0 (socket says 172.21.15.120:46882) 2024-09-10T10:29:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.905+0000 7fe588d9d640 1 -- 172.21.15.120:0/1091231786 learned_addr learned my addr 172.21.15.120:0/1091231786 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.906+0000 7fe583fff640 1 --2- 172.21.15.120:0/1091231786 >> v2:172.21.15.134:3300/0 conn(0x7fe5840cd5f0 0x7fe58406c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.906+0000 7fe588d9d640 1 -- 172.21.15.120:0/1091231786 >> v2:172.21.15.134:3300/0 conn(0x7fe5840cd5f0 msgr2=0x7fe58406c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.906+0000 7fe588d9d640 1 --2- 172.21.15.120:0/1091231786 >> v2:172.21.15.134:3300/0 conn(0x7fe5840cd5f0 0x7fe58406c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.906+0000 7fe588d9d640 1 -- 172.21.15.120:0/1091231786 >> v1:172.21.15.134:6789/0 conn(0x7fe58406c830 legacy=0x7fe584072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.906+0000 7fe588d9d640 1 -- 172.21.15.120:0/1091231786 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5840c7bf0 con 0x7fe5840cd210 2024-09-10T10:29:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.906+0000 7fe583fff640 1 --2- 172.21.15.120:0/1091231786 >> v2:172.21.15.134:3300/0 conn(0x7fe5840cd5f0 0x7fe58406c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.906+0000 7fe588d9d640 1 --2- 172.21.15.120:0/1091231786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5840cd210 0x7fe5840ca3c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe5840c96b0 tx=0x7fe57802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df23a5473de4d28b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe5837fe640 1 -- 172.21.15.120:0/1091231786 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578037070 con 0x7fe5840cd210 2024-09-10T10:29:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe5837fe640 1 -- 172.21.15.120:0/1091231786 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe57802ae50 con 0x7fe5840cd210 2024-09-10T10:29:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe5837fe640 1 -- 172.21.15.120:0/1091231786 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578032880 con 0x7fe5840cd210 2024-09-10T10:29:37.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe589d9f640 1 -- 172.21.15.120:0/1091231786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5840cd210 msgr2=0x7fe5840ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe589d9f640 1 --2- 172.21.15.120:0/1091231786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5840cd210 0x7fe5840ca3c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe5840c96b0 tx=0x7fe57802a300 comp rx=0 tx=0).stop 2024-09-10T10:29:37.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe589d9f640 1 -- 172.21.15.120:0/1091231786 shutdown_connections 2024-09-10T10:29:37.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe589d9f640 1 --2- 172.21.15.120:0/1091231786 >> v2:172.21.15.134:3300/0 conn(0x7fe5840cd5f0 0x7fe58406c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe589d9f640 1 --2- 172.21.15.120:0/1091231786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5840cd210 0x7fe5840ca3c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe589d9f640 1 -- 172.21.15.120:0/1091231786 >> 172.21.15.120:0/1091231786 conn(0x7fe58406cd20 msgr2=0x7fe58406d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:37.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe589d9f640 1 -- 172.21.15.120:0/1091231786 shutdown_connections 2024-09-10T10:29:37.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.907+0000 7fe589d9f640 1 -- 172.21.15.120:0/1091231786 wait complete. 2024-09-10T10:29:37.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.908+0000 7fe589d9f640 1 Processor -- start 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.908+0000 7fe589d9f640 1 -- start start 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe589d9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5840cd210 0x7fe584153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe589d9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5840cd5f0 0x7fe5841509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe589d9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe584150f10 0x7fe58414f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe589d9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5840c8490 con 0x7fe584150f10 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe589d9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5840c81b0 con 0x7fe5840cd210 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe589d9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe5840731f0 con 0x7fe5840cd5f0 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe58959e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe584150f10 0x7fe58414f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe588d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5840cd210 0x7fe584153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe58959e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe584150f10 0x7fe58414f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46888/0 (socket says 172.21.15.120:46888) 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe58959e640 1 -- 172.21.15.120:0/3589555507 learned_addr learned my addr 172.21.15.120:0/3589555507 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.909+0000 7fe583fff640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5840cd5f0 0x7fe5841509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe58959e640 1 -- 172.21.15.120:0/3589555507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5840cd5f0 msgr2=0x7fe5841509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe58959e640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5840cd5f0 0x7fe5841509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe58959e640 1 -- 172.21.15.120:0/3589555507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5840cd210 msgr2=0x7fe584153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe58959e640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5840cd210 0x7fe584153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe58959e640 1 -- 172.21.15.120:0/3589555507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe584150520 con 0x7fe584150f10 2024-09-10T10:29:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe58959e640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe584150f10 0x7fe58414f400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00c790 tx=0x7fe57c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe5817fa640 1 -- 172.21.15.120:0/3589555507 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c003b70 con 0x7fe584150f10 2024-09-10T10:29:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe5817fa640 1 -- 172.21.15.120:0/3589555507 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe57c003d10 con 0x7fe584150f10 2024-09-10T10:29:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe5817fa640 1 -- 172.21.15.120:0/3589555507 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00aab0 con 0x7fe584150f10 2024-09-10T10:29:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe589d9f640 1 -- 172.21.15.120:0/3589555507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5840c3950 con 0x7fe584150f10 2024-09-10T10:29:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.910+0000 7fe589d9f640 1 -- 172.21.15.120:0/3589555507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5840c3c10 con 0x7fe584150f10 2024-09-10T10:29:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.911+0000 7fe589d9f640 1 -- 172.21.15.120:0/3589555507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5840728c0 con 0x7fe584150f10 2024-09-10T10:29:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.912+0000 7fe5817fa640 1 -- 172.21.15.120:0/3589555507 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe57c00ac50 con 0x7fe584150f10 2024-09-10T10:29:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.912+0000 7fe5817fa640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe56806be30 0x7fe56806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.912+0000 7fe5817fa640 1 -- 172.21.15.120:0/3589555507 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fe57c08b060 con 0x7fe584150f10 2024-09-10T10:29:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.912+0000 7fe588d9d640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe56806be30 0x7fe56806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.912+0000 7fe588d9d640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe56806be30 0x7fe56806e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe578034000 tx=0x7fe57802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:37.916+0000 7fe5817fa640 1 -- 172.21.15.120:0/3589555507 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe57c058f50 con 0x7fe584150f10 2024-09-10T10:29:38.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.092+0000 7fe589d9f640 1 -- 172.21.15.120:0/3589555507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5840c8c70 con 0x7fe584150f10 2024-09-10T10:29:38.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.093+0000 7fe5817fa640 1 -- 172.21.15.120:0/3589555507 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+19282 (secure 0 0 0) 0x7fe57c05d250 con 0x7fe584150f10 2024-09-10T10:29:38.094 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T10:29:38.095 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":78,"fsid":"7d0d0830-4370-41c3-9acc-b67b07024f68","created":"2024-09-10T10:26:18.018818+0000","modified":"2024-09-10T10:29:05.476388+0000","last_up_change":"2024-09-10T10:26:29.078139+0000","last_in_change":"2024-09-10T10:26:20.091201+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:26:30.118649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:26:38.936606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:26:40.094780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:26:43.814005+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e3b9c3c3-cf7d-4b33-913e-60e928439d8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6809","nonce":3664783490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6811","nonce":3664783490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6815","nonce":3664783490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3664783490},{"type":"v1","addr":"172.21.15.120:6813","nonce":3664783490}]},"public_addr":"172.21.15.120:6809/3664783490","cluster_addr":"172.21.15.120:6811/3664783490","heartbeat_back_addr":"172.21.15.120:6815/3664783490","heartbeat_front_addr":"172.21.15.120:6813/3664783490","state":["exists","up"]},{"osd":1,"uuid":"62c20a9d-e6ab-446d-998c-9ee46e46aa90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6801","nonce":3972761964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6803","nonce":3972761964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6807","nonce":3972761964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3972761964},{"type":"v1","addr":"172.21.15.120:6805","nonce":3972761964}]},"public_addr":"172.21.15.120:6801/3972761964","cluster_addr":"172.21.15.120:6803/3972761964","heartbeat_back_addr":"172.21.15.120:6807/3972761964","heartbeat_front_addr":"172.21.15.120:6805/3972761964","state":["exists","up"]},{"osd":2,"uuid":"1528f8e0-5e13-4ec7-9d98-0d94b7a505c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6817","nonce":2916410859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6819","nonce":2916410859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6823","nonce":2916410859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2916410859},{"type":"v1","addr":"172.21.15.120:6821","nonce":2916410859}]},"public_addr":"172.21.15.120:6817/2916410859","cluster_addr":"172.21.15.120:6819/2916410859","heartbeat_back_addr":"172.21.15.120:6823/2916410859","heartbeat_front_addr":"172.21.15.120:6821/2916410859","state":["exists","up"]},{"osd":3,"uuid":"74942f78-c1fc-4886-abf0-950722c5edc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6825","nonce":1224278950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6827","nonce":1224278950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6831","nonce":1224278950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1224278950},{"type":"v1","addr":"172.21.15.120:6829","nonce":1224278950}]},"public_addr":"172.21.15.120:6825/1224278950","cluster_addr":"172.21.15.120:6827/1224278950","heartbeat_back_addr":"172.21.15.120:6831/1224278950","heartbeat_front_addr":"172.21.15.120:6829/1224278950","state":["exists","up"]},{"osd":4,"uuid":"e71319d4-5318-4441-84a8-7665ae037596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6817","nonce":1874635282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6819","nonce":1874635282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6823","nonce":1874635282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1874635282},{"type":"v1","addr":"172.21.15.134:6821","nonce":1874635282}]},"public_addr":"172.21.15.134:6817/1874635282","cluster_addr":"172.21.15.134:6819/1874635282","heartbeat_back_addr":"172.21.15.134:6823/1874635282","heartbeat_front_addr":"172.21.15.134:6821/1874635282","state":["exists","up"]},{"osd":5,"uuid":"29e193cd-37af-43e5-bd0c-2f85ce0631e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6809","nonce":457724087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6811","nonce":457724087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6815","nonce":457724087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":457724087},{"type":"v1","addr":"172.21.15.134:6813","nonce":457724087}]},"public_addr":"172.21.15.134:6809/457724087","cluster_addr":"172.21.15.134:6811/457724087","heartbeat_back_addr":"172.21.15.134:6815/457724087","heartbeat_front_addr":"172.21.15.134:6813/457724087","state":["exists","up"]},{"osd":6,"uuid":"55523ec2-cf12-44b3-aeea-f1cedabc996a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6825","nonce":50133702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6827","nonce":50133702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6831","nonce":50133702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":50133702},{"type":"v1","addr":"172.21.15.134:6829","nonce":50133702}]},"public_addr":"172.21.15.134:6825/50133702","cluster_addr":"172.21.15.134:6827/50133702","heartbeat_back_addr":"172.21.15.134:6831/50133702","heartbeat_front_addr":"172.21.15.134:6829/50133702","state":["exists","up"]},{"osd":7,"uuid":"097b1149-3e3c-4e5a-890b-d4cc4361d51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6801","nonce":251960875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6803","nonce":251960875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6807","nonce":251960875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251960875},{"type":"v1","addr":"172.21.15.134:6805","nonce":251960875}]},"public_addr":"172.21.15.134:6801/251960875","cluster_addr":"172.21.15.134:6803/251960875","heartbeat_back_addr":"172.21.15.134:6807/251960875","heartbeat_front_addr":"172.21.15.134:6805/251960875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.143056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.423968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.640076+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.536160+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.146033+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.426011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.598992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:26:26.182929+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2c","osds":[1,7]},{"pgid":"3.34","osds":[3,1]}],"pg_upmap_items":[{"pgid":"2.1d","mappings":[{"from":6,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":6}]},{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":6}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":3,"to":6},{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2},{"from":3,"to":0}]},{"pgid":"4.1e","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:29:38.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 -- 172.21.15.120:0/3589555507 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe56806be30 msgr2=0x7fe56806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe56806be30 0x7fe56806e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe578034000 tx=0x7fe57802e000 comp rx=0 tx=0).stop 2024-09-10T10:29:38.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 -- 172.21.15.120:0/3589555507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe584150f10 msgr2=0x7fe58414f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe584150f10 0x7fe58414f400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00c790 tx=0x7fe57c00cc50 comp rx=0 tx=0).stop 2024-09-10T10:29:38.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 -- 172.21.15.120:0/3589555507 shutdown_connections 2024-09-10T10:29:38.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe56806be30 0x7fe56806e2f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe584150f10 0x7fe58414f400 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5840cd5f0 0x7fe5841509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 --2- 172.21.15.120:0/3589555507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5840cd210 0x7fe584153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 -- 172.21.15.120:0/3589555507 >> 172.21.15.120:0/3589555507 conn(0x7fe58406cd20 msgr2=0x7fe584070220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:38.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 -- 172.21.15.120:0/3589555507 shutdown_connections 2024-09-10T10:29:38.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.095+0000 7fe589d9f640 1 -- 172.21.15.120:0/3589555507 wait complete. 2024-09-10T10:29:38.108 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:29:38.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.206+0000 7fe0cb128640 1 Processor -- start 2024-09-10T10:29:38.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.206+0000 7fe0cb128640 1 -- start start 2024-09-10T10:29:38.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0cb128640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe0c4154d00 0x7fe0c40c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0cb128640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe0c415e040 0x7fe0c40c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0cb128640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c406c950 con 0x7fe0c4071010 2024-09-10T10:29:38.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0cb128640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c40707e0 con 0x7fe0c415e040 2024-09-10T10:29:38.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0cb128640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c40c90f0 con 0x7fe0c4154d00 2024-09-10T10:29:38.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0c3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe0c415e040 0x7fe0c40c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0c3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe0c415e040 0x7fe0c40c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46900/0 (socket says 172.21.15.120:46900) 2024-09-10T10:29:38.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0c3fff640 1 -- 172.21.15.120:0/930782387 learned_addr learned my addr 172.21.15.120:0/930782387 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:38.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0c3fff640 1 -- 172.21.15.120:0/930782387 >> v2:172.21.15.134:3300/0 conn(0x7fe0c4154d00 msgr2=0x7fe0c40c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:38.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0c8ec4640 1 --2- 172.21.15.120:0/930782387 >> v2:172.21.15.134:3300/0 conn(0x7fe0c4154d00 0x7fe0c40c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0c3fff640 1 --2- 172.21.15.120:0/930782387 >> v2:172.21.15.134:3300/0 conn(0x7fe0c4154d00 0x7fe0c40c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0c3fff640 1 -- 172.21.15.120:0/930782387 >> v1:172.21.15.134:6789/0 conn(0x7fe0c4071010 legacy=0x7fe0c40cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:38.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.207+0000 7fe0c3fff640 1 -- 172.21.15.120:0/930782387 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c4070a60 con 0x7fe0c415e040 2024-09-10T10:29:38.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.208+0000 7fe0c3fff640 1 --2- 172.21.15.120:0/930782387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c415e040 0x7fe0c40c8bb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40cdf50 tx=0x7fe0b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486fce61a6eedd95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:38.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.208+0000 7fe0c37fe640 1 -- 172.21.15.120:0/930782387 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8037070 con 0x7fe0c415e040 2024-09-10T10:29:38.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.208+0000 7fe0c37fe640 1 -- 172.21.15.120:0/930782387 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe0b8032440 con 0x7fe0c415e040 2024-09-10T10:29:38.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.208+0000 7fe0c37fe640 1 -- 172.21.15.120:0/930782387 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8032830 con 0x7fe0c415e040 2024-09-10T10:29:38.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.209+0000 7fe0cb128640 1 -- 172.21.15.120:0/930782387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c415e040 msgr2=0x7fe0c40c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.209+0000 7fe0cb128640 1 --2- 172.21.15.120:0/930782387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c415e040 0x7fe0c40c8bb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40cdf50 tx=0x7fe0b802a230 comp rx=0 tx=0).stop 2024-09-10T10:29:38.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.209+0000 7fe0cb128640 1 -- 172.21.15.120:0/930782387 shutdown_connections 2024-09-10T10:29:38.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.209+0000 7fe0cb128640 1 --2- 172.21.15.120:0/930782387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c415e040 0x7fe0c40c8bb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.209+0000 7fe0cb128640 1 --2- 172.21.15.120:0/930782387 >> v2:172.21.15.134:3300/0 conn(0x7fe0c4154d00 0x7fe0c40c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.209+0000 7fe0cb128640 1 -- 172.21.15.120:0/930782387 >> 172.21.15.120:0/930782387 conn(0x7fe0c4067950 msgr2=0x7fe0c4067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:38.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.209+0000 7fe0cb128640 1 -- 172.21.15.120:0/930782387 shutdown_connections 2024-09-10T10:29:38.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.209+0000 7fe0cb128640 1 -- 172.21.15.120:0/930782387 wait complete. 2024-09-10T10:29:38.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.210+0000 7fe0cb128640 1 Processor -- start 2024-09-10T10:29:38.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.210+0000 7fe0cb128640 1 -- start start 2024-09-10T10:29:38.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0cb128640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c4071010 0x7fe0c406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0cb128640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c4154d00 0x7fe0c41f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0cb128640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0c415e040 0x7fe0c41f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0cb128640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4070d40 con 0x7fe0c4154d00 2024-09-10T10:29:38.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0cb128640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c406c670 con 0x7fe0c4071010 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0cb128640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c40c75a0 con 0x7fe0c415e040 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c4154d00 0x7fe0c41f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c4154d00 0x7fe0c41f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46912/0 (socket says 172.21.15.120:46912) 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0c3fff640 1 -- 172.21.15.120:0/1384204612 learned_addr learned my addr 172.21.15.120:0/1384204612 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0c8ec4640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c4071010 0x7fe0c406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0c3fff640 1 -- 172.21.15.120:0/1384204612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0c415e040 msgr2=0x7fe0c41f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0c96c5640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0c415e040 0x7fe0c41f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0c3fff640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0c415e040 0x7fe0c41f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0c3fff640 1 -- 172.21.15.120:0/1384204612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c4071010 msgr2=0x7fe0c406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0c3fff640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c4071010 0x7fe0c406d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.211+0000 7fe0c3fff640 1 -- 172.21.15.120:0/1384204612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c41f4b70 con 0x7fe0c4154d00 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.212+0000 7fe0c3fff640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c4154d00 0x7fe0c41f2740 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe0b802a270 tx=0x7fe0b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:38.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.212+0000 7fe0c17fa640 1 -- 172.21.15.120:0/1384204612 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8037040 con 0x7fe0c4154d00 2024-09-10T10:29:38.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.212+0000 7fe0c17fa640 1 -- 172.21.15.120:0/1384204612 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe0b8042070 con 0x7fe0c4154d00 2024-09-10T10:29:38.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.212+0000 7fe0cb128640 1 -- 172.21.15.120:0/1384204612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c41f1ca0 con 0x7fe0c4154d00 2024-09-10T10:29:38.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.212+0000 7fe0c17fa640 1 -- 172.21.15.120:0/1384204612 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8030bd0 con 0x7fe0c4154d00 2024-09-10T10:29:38.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.212+0000 7fe0cb128640 1 -- 172.21.15.120:0/1384204612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c41f1f40 con 0x7fe0c4154d00 2024-09-10T10:29:38.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.213+0000 7fe0cb128640 1 -- 172.21.15.120:0/1384204612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c41550e0 con 0x7fe0c4154d00 2024-09-10T10:29:38.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.213+0000 7fe0c17fa640 1 -- 172.21.15.120:0/1384204612 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0b8047460 con 0x7fe0c4154d00 2024-09-10T10:29:38.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.213+0000 7fe0c17fa640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe0b006be30 0x7fe0b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.214+0000 7fe0c17fa640 1 -- 172.21.15.120:0/1384204612 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6986+0+0 (secure 0 0 0) 0x7fe0b80ac020 con 0x7fe0c4154d00 2024-09-10T10:29:38.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.214+0000 7fe0c8ec4640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe0b006be30 0x7fe0b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.214+0000 7fe0c8ec4640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe0b006be30 0x7fe0b006e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4006fd0 tx=0x7fe0b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:38.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.218+0000 7fe0c17fa640 1 -- 172.21.15.120:0/1384204612 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe0b8045360 con 0x7fe0c4154d00 2024-09-10T10:29:38.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.416+0000 7fe0cb128640 1 -- 172.21.15.120:0/1384204612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe0c415c040 con 0x7fe0c4154d00 2024-09-10T10:29:38.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.418+0000 7fe0c17fa640 1 -- 172.21.15.120:0/1384204612 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fe0b807a450 con 0x7fe0c4154d00 2024-09-10T10:29:38.419 INFO:teuthology.orchestra.run.smithi120.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:29:38.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.422+0000 7fe0cb128640 1 -- 172.21.15.120:0/1384204612 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe0b006be30 msgr2=0x7fe0b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.422+0000 7fe0cb128640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe0b006be30 0x7fe0b006e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4006fd0 tx=0x7fe0b4008040 comp rx=0 tx=0).stop 2024-09-10T10:29:38.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.422+0000 7fe0cb128640 1 -- 172.21.15.120:0/1384204612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c4154d00 msgr2=0x7fe0c41f2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.422+0000 7fe0cb128640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c4154d00 0x7fe0c41f2740 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe0b802a270 tx=0x7fe0b80046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.423+0000 7fe0cb128640 1 -- 172.21.15.120:0/1384204612 shutdown_connections 2024-09-10T10:29:38.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.423+0000 7fe0cb128640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7fe0b006be30 0x7fe0b006e2f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.423+0000 7fe0cb128640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0c415e040 0x7fe0c41f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.423+0000 7fe0cb128640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0c4154d00 0x7fe0c41f2740 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.423+0000 7fe0cb128640 1 --2- 172.21.15.120:0/1384204612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0c4071010 0x7fe0c406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.423+0000 7fe0cb128640 1 -- 172.21.15.120:0/1384204612 >> 172.21.15.120:0/1384204612 conn(0x7fe0c4067950 msgr2=0x7fe0c415c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:38.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.423+0000 7fe0cb128640 1 -- 172.21.15.120:0/1384204612 shutdown_connections 2024-09-10T10:29:38.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.423+0000 7fe0cb128640 1 -- 172.21.15.120:0/1384204612 wait complete. 2024-09-10T10:29:38.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:29:38.457 INFO:tasks.ceph.mds.e.smithi120.stdout:starting mds.e at 2024-09-10T10:29:38.457 INFO:tasks.ceph.mds.c.smithi120.stdout:starting mds.c at 2024-09-10T10:29:38.458 INFO:tasks.ceph.mds.a.smithi120.stdout:starting mds.a at 2024-09-10T10:29:38.468 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-10T10:29:38.469 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-10T10:29:38.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.542+0000 7f9ff656d640 1 Processor -- start 2024-09-10T10:29:38.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.542+0000 7f9ff656d640 1 -- start start 2024-09-10T10:29:38.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.543+0000 7f9ff656d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9ff0152760 0x7f9ff00c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.543+0000 7f9ff656d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff0158890 con 0x7f9ff015c480 2024-09-10T10:29:38.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.543+0000 7f9ff656d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff0158b00 con 0x7f9ff00c6cc0 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.543+0000 7f9ff656d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff00c8530 con 0x7f9ff0152760 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.543+0000 7f9feffff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9ff0152760 0x7f9ff00c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.543+0000 7f9ff4b0a640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f9ff015c480 0x7f9ff0163100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:45602/0 (socket says 172.21.15.120:45602) 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.543+0000 7f9ff4b0a640 1 -- 172.21.15.120:0/1415131694 learned_addr learned my addr 172.21.15.120:0/1415131694 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.544+0000 7f9feffff640 1 -- 172.21.15.120:0/1415131694 >> v1:172.21.15.134:6789/0 conn(0x7f9ff00c6cc0 legacy=0x7f9ff015bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.544+0000 7f9feffff640 1 -- 172.21.15.120:0/1415131694 >> v1:172.21.15.120:6789/0 conn(0x7f9ff015c480 legacy=0x7f9ff0163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.544+0000 7f9feffff640 1 -- 172.21.15.120:0/1415131694 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff0158de0 con 0x7f9ff0152760 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.544+0000 7f9fef7fe640 1 -- 172.21.15.120:0/1415131694 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1345475847 0 0) 0x7f9ff0158890 con 0x7f9ff015c480 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.544+0000 7f9feffff640 1 --2- 172.21.15.120:0/1415131694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 0x7f9ff00c59d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00097a0 tx=0x7f9fe002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a84bdf3f79a80d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.544+0000 7f9fef7fe640 1 -- 172.21.15.120:0/1415131694 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0037070 con 0x7f9ff0152760 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.544+0000 7f9fef7fe640 1 -- 172.21.15.120:0/1415131694 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9fe0032640 con 0x7f9ff0152760 2024-09-10T10:29:38.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.544+0000 7f9fef7fe640 1 -- 172.21.15.120:0/1415131694 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0032a30 con 0x7f9ff0152760 2024-09-10T10:29:38.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.545+0000 7f9ff656d640 1 -- 172.21.15.120:0/1415131694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 msgr2=0x7f9ff00c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.545+0000 7f9ff656d640 1 --2- 172.21.15.120:0/1415131694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 0x7f9ff00c59d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00097a0 tx=0x7f9fe002a2c0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.545+0000 7f9ff656d640 1 -- 172.21.15.120:0/1415131694 shutdown_connections 2024-09-10T10:29:38.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.545+0000 7f9ff656d640 1 --2- 172.21.15.120:0/1415131694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 0x7f9ff00c59d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.545+0000 7f9ff656d640 1 -- 172.21.15.120:0/1415131694 >> 172.21.15.120:0/1415131694 conn(0x7f9ff006e670 msgr2=0x7f9ff006ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.545+0000 7f9ff656d640 1 -- 172.21.15.120:0/1415131694 shutdown_connections 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.545+0000 7f9ff656d640 1 -- 172.21.15.120:0/1415131694 wait complete. 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.546+0000 7f9ff656d640 1 Processor -- start 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.546+0000 7f9ff656d640 1 -- start start 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9ff656d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff00c6cc0 0x7f9ff014b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9ff656d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 0x7f9ff01ef220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9ff656d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ff015c480 0x7f9ff01ef760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9ff656d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0152b40 con 0x7f9ff0152760 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9ff656d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff01593a0 con 0x7f9ff00c6cc0 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9ff656d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff0152cc0 con 0x7f9ff015c480 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9fe7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 0x7f9ff01ef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9fe7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 0x7f9ff01ef220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46966/0 (socket says 172.21.15.120:46966) 2024-09-10T10:29:38.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9fe7fff640 1 -- 172.21.15.120:0/4273878550 learned_addr learned my addr 172.21.15.120:0/4273878550 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9ff4b0a640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ff015c480 0x7f9ff01ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9feffff640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff00c6cc0 0x7f9ff014b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9fe7fff640 1 -- 172.21.15.120:0/4273878550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ff015c480 msgr2=0x7f9ff01ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9fe7fff640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ff015c480 0x7f9ff01ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9fe7fff640 1 -- 172.21.15.120:0/4273878550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff00c6cc0 msgr2=0x7f9ff014b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9fe7fff640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff00c6cc0 0x7f9ff014b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9fe7fff640 1 -- 172.21.15.120:0/4273878550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff00cb5d0 con 0x7f9ff0152760 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9feffff640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff00c6cc0 0x7f9ff014b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9fe7fff640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 0x7f9ff01ef220 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800c430 tx=0x7f9fd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.547+0000 7f9fed7fa640 1 -- 172.21.15.120:0/4273878550 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8017070 con 0x7f9ff0152760 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.548+0000 7f9fed7fa640 1 -- 172.21.15.120:0/4273878550 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9fd80052d0 con 0x7f9ff0152760 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.548+0000 7f9ff656d640 1 -- 172.21.15.120:0/4273878550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff0147b80 con 0x7f9ff0152760 2024-09-10T10:29:38.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.548+0000 7f9fed7fa640 1 -- 172.21.15.120:0/4273878550 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8003850 con 0x7f9ff0152760 2024-09-10T10:29:38.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.548+0000 7f9ff656d640 1 -- 172.21.15.120:0/4273878550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff0148120 con 0x7f9ff0152760 2024-09-10T10:29:38.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.549+0000 7f9ff656d640 1 -- 172.21.15.120:0/4273878550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff0152e40 con 0x7f9ff0152760 2024-09-10T10:29:38.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.549+0000 7f9fed7fa640 1 -- 172.21.15.120:0/4273878550 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fd801a070 con 0x7f9ff0152760 2024-09-10T10:29:38.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.549+0000 7f9fed7fa640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9fd006be30 0x7f9fd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:38.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.549+0000 7f9fed7fa640 1 -- 172.21.15.120:0/4273878550 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7416+0+0 (secure 0 0 0) 0x7f9fd808a850 con 0x7f9ff0152760 2024-09-10T10:29:38.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.549+0000 7f9feffff640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9fd006be30 0x7f9fd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:38.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.550+0000 7f9feffff640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9fd006be30 0x7f9fd006e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0034000 tx=0x7f9fe002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:38.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.554+0000 7f9fed7fa640 1 -- 172.21.15.120:0/4273878550 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9ff0152e40 con 0x7f9ff0152760 2024-09-10T10:29:38.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:38.753+0000 7f9ff656d640 1 -- 172.21.15.120:0/4273878550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9ff0151bd0 con 0x7f9ff0152760 2024-09-10T10:29:39.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.420+0000 7f9fed7fa640 1 -- 172.21.15.120:0/4273878550 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f9ff0151bd0 con 0x7f9ff0152760 2024-09-10T10:29:39.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.422+0000 7f9ff656d640 1 -- 172.21.15.120:0/4273878550 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9fd006be30 msgr2=0x7f9fd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:39.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.422+0000 7f9ff656d640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9fd006be30 0x7f9fd006e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0034000 tx=0x7f9fe002e000 comp rx=0 tx=0).stop 2024-09-10T10:29:39.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.422+0000 7f9ff656d640 1 -- 172.21.15.120:0/4273878550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 msgr2=0x7f9ff01ef220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:39.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.422+0000 7f9ff656d640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 0x7f9ff01ef220 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800c430 tx=0x7f9fd800c8f0 comp rx=0 tx=0).stop 2024-09-10T10:29:39.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.423+0000 7f9ff656d640 1 -- 172.21.15.120:0/4273878550 shutdown_connections 2024-09-10T10:29:39.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.423+0000 7f9ff656d640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9fd006be30 0x7f9fd006e2f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:39.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.423+0000 7f9ff656d640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ff015c480 0x7f9ff01ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:39.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.423+0000 7f9ff656d640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ff0152760 0x7f9ff01ef220 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:39.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.423+0000 7f9ff656d640 1 --2- 172.21.15.120:0/4273878550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff00c6cc0 0x7f9ff014b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:39.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.423+0000 7f9ff656d640 1 -- 172.21.15.120:0/4273878550 >> 172.21.15.120:0/4273878550 conn(0x7f9ff006e670 msgr2=0x7f9ff0150f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:39.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.423+0000 7f9ff656d640 1 -- 172.21.15.120:0/4273878550 shutdown_connections 2024-09-10T10:29:39.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.423+0000 7f9ff656d640 1 -- 172.21.15.120:0/4273878550 wait complete. 2024-09-10T10:29:39.438 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:29:39.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.583+0000 7f9dc42a6640 1 Processor -- start 2024-09-10T10:29:39.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.583+0000 7f9dc42a6640 1 -- start start 2024-09-10T10:29:39.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc42a6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9dbc154df0 0x7f9dbc0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:39.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc42a6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc06d2e0 0x7f9dbc06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:39.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc42a6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dbc070390 con 0x7f9dbc15aa20 2024-09-10T10:29:39.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc42a6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9dbc06ce50 con 0x7f9dbc154df0 2024-09-10T10:29:39.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc42a6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9dbc1551d0 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc2042640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9dbc154df0 0x7f9dbc0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:39.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc2843640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f9dbc15aa20 0x7f9dbc15aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:45604/0 (socket says 172.21.15.120:45604) 2024-09-10T10:29:39.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc2843640 1 -- 172.21.15.120:0/2168325038 learned_addr learned my addr 172.21.15.120:0/2168325038 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-09-10T10:29:39.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc1841640 1 --2- 172.21.15.120:0/2168325038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc06d2e0 0x7f9dbc06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:39.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc2042640 1 -- 172.21.15.120:0/2168325038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc06d2e0 msgr2=0x7f9dbc06d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:39.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc2042640 1 --2- 172.21.15.120:0/2168325038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc06d2e0 0x7f9dbc06d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:39.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc2042640 1 -- 172.21.15.120:0/2168325038 >> v1:172.21.15.120:6789/0 conn(0x7f9dbc15aa20 legacy=0x7f9dbc15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:39.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc2042640 1 -- 172.21.15.120:0/2168325038 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc0b0f60 con 0x7f9dbc154df0 2024-09-10T10:29:39.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc1040640 1 -- 172.21.15.120:0/2168325038 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1356171135 0 0) 0x7f9dbc070390 con 0x7f9dbc15aa20 2024-09-10T10:29:39.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.584+0000 7f9dc1841640 1 --2- 172.21.15.120:0/2168325038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc06d2e0 0x7f9dbc06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:29:39.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.585+0000 7f9dc2042640 1 --2- 172.21.15.120:0/2168325038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc154df0 0x7f9dbc0c9700 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097a0 tx=0x7f9db802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a29e4cf605e36d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:39.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.585+0000 7f9dc1040640 1 -- 172.21.15.120:0/2168325038 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8037070 con 0x7f9dbc154df0 2024-09-10T10:29:39.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.585+0000 7f9dc1040640 1 -- 172.21.15.120:0/2168325038 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9db8032440 con 0x7f9dbc154df0 2024-09-10T10:29:39.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.585+0000 7f9dc1040640 1 -- 172.21.15.120:0/2168325038 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8032830 con 0x7f9dbc154df0 2024-09-10T10:29:39.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.585+0000 7f9dc42a6640 1 -- 172.21.15.120:0/2168325038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc154df0 msgr2=0x7f9dbc0c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:39.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.585+0000 7f9dc42a6640 1 --2- 172.21.15.120:0/2168325038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc154df0 0x7f9dbc0c9700 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097a0 tx=0x7f9db802a210 comp rx=0 tx=0).stop 2024-09-10T10:29:39.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.586+0000 7f9dc42a6640 1 -- 172.21.15.120:0/2168325038 shutdown_connections 2024-09-10T10:29:39.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.586+0000 7f9dc42a6640 1 --2- 172.21.15.120:0/2168325038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc06d2e0 0x7f9dbc06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:39.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.586+0000 7f9dc42a6640 1 --2- 172.21.15.120:0/2168325038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc154df0 0x7f9dbc0c9700 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:39.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.586+0000 7f9dc42a6640 1 -- 172.21.15.120:0/2168325038 >> 172.21.15.120:0/2168325038 conn(0x7f9dbc071720 msgr2=0x7f9dbc071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:39.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.586+0000 7f9dc42a6640 1 -- 172.21.15.120:0/2168325038 shutdown_connections 2024-09-10T10:29:39.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.586+0000 7f9dc42a6640 1 -- 172.21.15.120:0/2168325038 wait complete. 2024-09-10T10:29:39.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc42a6640 1 Processor -- start 2024-09-10T10:29:39.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc42a6640 1 -- start start 2024-09-10T10:29:39.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc42a6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc06d2e0 0x7f9dbc15e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc42a6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc154df0 0x7f9dbc0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc42a6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc15aa20 0x7f9dbc0af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc42a6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc06dc20 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc42a6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc0b1240 con 0x7f9dbc15aa20 2024-09-10T10:29:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc42a6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9dbc070910 con 0x7f9dbc154df0 2024-09-10T10:29:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc2042640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc06d2e0 0x7f9dbc15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc2042640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc06d2e0 0x7f9dbc15e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46980/0 (socket says 172.21.15.120:46980) 2024-09-10T10:29:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.587+0000 7f9dc2042640 1 -- 172.21.15.120:0/1821400118 learned_addr learned my addr 172.21.15.120:0/1821400118 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9dc2042640 1 -- 172.21.15.120:0/1821400118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc154df0 msgr2=0x7f9dbc0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9dc2042640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc154df0 0x7f9dbc0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9dc2042640 1 -- 172.21.15.120:0/1821400118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc15aa20 msgr2=0x7f9dbc0af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9dc2042640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc15aa20 0x7f9dbc0af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9dc2042640 1 -- 172.21.15.120:0/1821400118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc0aec60 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9dc2042640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc06d2e0 0x7f9dbc15e660 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097a0 tx=0x7f9db80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9db2ffd640 1 -- 172.21.15.120:0/1821400118 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8044070 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9db2ffd640 1 -- 172.21.15.120:0/1821400118 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9db8038040 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9db2ffd640 1 -- 172.21.15.120:0/1821400118 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8037050 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9dc42a6640 1 -- 172.21.15.120:0/1821400118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dbc0ae9c0 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.588+0000 7f9dc42a6640 1 -- 172.21.15.120:0/1821400118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dbc0b0700 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.589+0000 7f9dc42a6640 1 -- 172.21.15.120:0/1821400118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dbc0c90c0 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.589+0000 7f9db2ffd640 1 -- 172.21.15.120:0/1821400118 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9db8047070 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.590+0000 7f9db2ffd640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da006be30 0x7f9da006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.590+0000 7f9db2ffd640 1 -- 172.21.15.120:0/1821400118 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7416+0+0 (secure 0 0 0) 0x7f9db80ab7f0 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.590+0000 7f9dc1841640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da006be30 0x7f9da006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.590+0000 7f9dc1841640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da006be30 0x7f9da006e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0ce170 tx=0x7f9dac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.594+0000 7f9db2ffd640 1 -- 172.21.15.120:0/1821400118 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9db8079a60 con 0x7f9dbc06d2e0 2024-09-10T10:29:39.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:39.760+0000 7f9dc42a6640 1 -- 172.21.15.120:0/1821400118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9dbc0c94e0 con 0x7f9dbc06d2e0 2024-09-10T10:29:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.422+0000 7f9db2ffd640 1 -- 172.21.15.120:0/1821400118 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) ==== 181+0+0 (secure 0 0 0) 0x7f9db8079360 con 0x7f9dbc06d2e0 2024-09-10T10:29:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:29:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 -- 172.21.15.120:0/1821400118 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da006be30 msgr2=0x7f9da006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da006be30 0x7f9da006e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0ce170 tx=0x7f9dac008040 comp rx=0 tx=0).stop 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 -- 172.21.15.120:0/1821400118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc06d2e0 msgr2=0x7f9dbc15e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc06d2e0 0x7f9dbc15e660 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097a0 tx=0x7f9db80046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 -- 172.21.15.120:0/1821400118 shutdown_connections 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f9da006be30 0x7f9da006e2f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc15aa20 0x7f9dbc0af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc154df0 0x7f9dbc0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 --2- 172.21.15.120:0/1821400118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dbc06d2e0 0x7f9dbc15e660 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 -- 172.21.15.120:0/1821400118 >> 172.21.15.120:0/1821400118 conn(0x7f9dbc071720 msgr2=0x7f9dbc15aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 -- 172.21.15.120:0/1821400118 shutdown_connections 2024-09-10T10:29:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.424+0000 7f9dc42a6640 1 -- 172.21.15.120:0/1821400118 wait complete. 2024-09-10T10:29:40.440 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:29:40.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.544+0000 7f5844786640 1 Processor -- start 2024-09-10T10:29:40.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.544+0000 7f5844786640 1 -- start start 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5844786640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f583c1529e0 0x7f583c0c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5844786640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f583c070740 0x7f583c0c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5844786640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f583c0c4c30 0x7f583c0c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5844786640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f583c0c4550 con 0x7f583c1529e0 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5844786640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f583c0739c0 con 0x7f583c0c4c30 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5844786640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f583c073ca0 con 0x7f583c070740 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5842522640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f583c1529e0 0x7f583c0c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5842522640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f583c1529e0 0x7f583c0c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46988/0 (socket says 172.21.15.120:46988) 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5842522640 1 -- 172.21.15.120:0/4107741374 learned_addr learned my addr 172.21.15.120:0/4107741374 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5842522640 1 -- 172.21.15.120:0/4107741374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f583c070740 msgr2=0x7f583c0c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:40.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5842522640 1 --2- 172.21.15.120:0/4107741374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f583c070740 0x7f583c0c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5842d23640 1 --2- 172.21.15.120:0/4107741374 >> v2:172.21.15.134:3300/0 conn(0x7f583c0c4c30 0x7f583c0c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:40.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5842522640 1 -- 172.21.15.120:0/4107741374 >> v2:172.21.15.134:3300/0 conn(0x7f583c0c4c30 msgr2=0x7f583c0c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:40.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5842522640 1 --2- 172.21.15.120:0/4107741374 >> v2:172.21.15.134:3300/0 conn(0x7f583c0c4c30 0x7f583c0c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.545+0000 7f5842522640 1 -- 172.21.15.120:0/4107741374 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583c074130 con 0x7f583c1529e0 2024-09-10T10:29:40.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.546+0000 7f5842522640 1 --2- 172.21.15.120:0/4107741374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c1529e0 0x7f583c0c4010 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f583c06f370 tx=0x7f583802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02f3dd63a4fb77b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:40.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.546+0000 7f5841520640 1 -- 172.21.15.120:0/4107741374 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838036070 con 0x7f583c1529e0 2024-09-10T10:29:40.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.546+0000 7f5841520640 1 -- 172.21.15.120:0/4107741374 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5838031600 con 0x7f583c1529e0 2024-09-10T10:29:40.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.546+0000 7f5841520640 1 -- 172.21.15.120:0/4107741374 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58380319f0 con 0x7f583c1529e0 2024-09-10T10:29:40.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.547+0000 7f5844786640 1 -- 172.21.15.120:0/4107741374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c1529e0 msgr2=0x7f583c0c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:40.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.547+0000 7f5844786640 1 --2- 172.21.15.120:0/4107741374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c1529e0 0x7f583c0c4010 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f583c06f370 tx=0x7f583802a730 comp rx=0 tx=0).stop 2024-09-10T10:29:40.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.547+0000 7f5844786640 1 -- 172.21.15.120:0/4107741374 shutdown_connections 2024-09-10T10:29:40.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.547+0000 7f5844786640 1 --2- 172.21.15.120:0/4107741374 >> v2:172.21.15.134:3300/0 conn(0x7f583c0c4c30 0x7f583c0c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.547+0000 7f5844786640 1 --2- 172.21.15.120:0/4107741374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f583c070740 0x7f583c0c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.547+0000 7f5844786640 1 --2- 172.21.15.120:0/4107741374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c1529e0 0x7f583c0c4010 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.547+0000 7f5844786640 1 -- 172.21.15.120:0/4107741374 >> 172.21.15.120:0/4107741374 conn(0x7f583c06fc20 msgr2=0x7f583c070030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:40.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.547+0000 7f5844786640 1 -- 172.21.15.120:0/4107741374 shutdown_connections 2024-09-10T10:29:40.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.547+0000 7f5844786640 1 -- 172.21.15.120:0/4107741374 wait complete. 2024-09-10T10:29:40.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.548+0000 7f5844786640 1 Processor -- start 2024-09-10T10:29:40.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.548+0000 7f5844786640 1 -- start start 2024-09-10T10:29:40.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5844786640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c070740 0x7f583c149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:40.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5844786640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f583c0c4c30 0x7f583c1ede30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5844786640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f583c1529e0 0x7f583c1ee370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5844786640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f583c067cd0 con 0x7f583c070740 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5844786640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f583c0679f0 con 0x7f583c1529e0 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5844786640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f583c067fb0 con 0x7f583c0c4c30 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5842522640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c070740 0x7f583c149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5842522640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c070740 0x7f583c149f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47000/0 (socket says 172.21.15.120:47000) 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5842522640 1 -- 172.21.15.120:0/752254742 learned_addr learned my addr 172.21.15.120:0/752254742 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5842522640 1 -- 172.21.15.120:0/752254742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f583c0c4c30 msgr2=0x7f583c1ede30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5842522640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f583c0c4c30 0x7f583c1ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5842522640 1 -- 172.21.15.120:0/752254742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f583c1529e0 msgr2=0x7f583c1ee370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5842522640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f583c1529e0 0x7f583c1ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:40.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5842522640 1 -- 172.21.15.120:0/752254742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583c150540 con 0x7f583c070740 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5842522640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c070740 0x7f583c149f30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5838000f80 tx=0x7f5838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5832ffd640 1 -- 172.21.15.120:0/752254742 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838036050 con 0x7f583c070740 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.549+0000 7f5832ffd640 1 -- 172.21.15.120:0/752254742 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5838037040 con 0x7f583c070740 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.550+0000 7f5844786640 1 -- 172.21.15.120:0/752254742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583c1f1540 con 0x7f583c070740 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.550+0000 7f5832ffd640 1 -- 172.21.15.120:0/752254742 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838009510 con 0x7f583c070740 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.550+0000 7f5844786640 1 -- 172.21.15.120:0/752254742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f583c1f1b00 con 0x7f583c070740 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.550+0000 7f5844786640 1 -- 172.21.15.120:0/752254742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f583c1592a0 con 0x7f583c070740 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.551+0000 7f5832ffd640 1 -- 172.21.15.120:0/752254742 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5838044020 con 0x7f583c070740 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.551+0000 7f5832ffd640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f582006be30 0x7f582006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.551+0000 7f5832ffd640 1 -- 172.21.15.120:0/752254742 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6949+0+0 (secure 0 0 0) 0x7f58380aba60 con 0x7f583c070740 2024-09-10T10:29:40.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.552+0000 7f5841d21640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f582006be30 0x7f582006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:40.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.552+0000 7f5841d21640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f582006be30 0x7f582006e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f583c0c4270 tx=0x7f582c006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:40.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.556+0000 7f5832ffd640 1 -- 172.21.15.120:0/752254742 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f583804b020 con 0x7f583c070740 2024-09-10T10:29:40.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:40.730+0000 7f5844786640 1 -- 172.21.15.120:0/752254742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f583c1509f0 con 0x7f583c070740 2024-09-10T10:29:41.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.423+0000 7f5832ffd640 1 -- 172.21.15.120:0/752254742 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) ==== 169+0+0 (secure 0 0 0) 0x7f5838079ea0 con 0x7f583c070740 2024-09-10T10:29:41.425 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_data' removed 2024-09-10T10:29:41.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 -- 172.21.15.120:0/752254742 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f582006be30 msgr2=0x7f582006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:41.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f582006be30 0x7f582006e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f583c0c4270 tx=0x7f582c006c90 comp rx=0 tx=0).stop 2024-09-10T10:29:41.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 -- 172.21.15.120:0/752254742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c070740 msgr2=0x7f583c149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:41.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c070740 0x7f583c149f30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5838000f80 tx=0x7f5838004a40 comp rx=0 tx=0).stop 2024-09-10T10:29:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 -- 172.21.15.120:0/752254742 shutdown_connections 2024-09-10T10:29:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f582006be30 0x7f582006e2f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f583c1529e0 0x7f583c1ee370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f583c0c4c30 0x7f583c1ede30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 --2- 172.21.15.120:0/752254742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c070740 0x7f583c149f30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 -- 172.21.15.120:0/752254742 >> 172.21.15.120:0/752254742 conn(0x7f583c06fc20 msgr2=0x7f583c1e0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 -- 172.21.15.120:0/752254742 shutdown_connections 2024-09-10T10:29:41.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.426+0000 7f5844786640 1 -- 172.21.15.120:0/752254742 wait complete. 2024-09-10T10:29:41.443 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T10:29:41.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.548+0000 7f44f756d640 1 Processor -- start 2024-09-10T10:29:41.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.548+0000 7f44f756d640 1 -- start start 2024-09-10T10:29:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.549+0000 7f44f756d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f44f00ca050 0x7f44f00ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.549+0000 7f44f756d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f015aab0 con 0x7f44f0158360 2024-09-10T10:29:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.549+0000 7f44f756d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f015e7f0 con 0x7f44f015b570 2024-09-10T10:29:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.549+0000 7f44f756d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f44f015b9e0 con 0x7f44f00ca050 2024-09-10T10:29:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.549+0000 7f44f4b08640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f44f00ca050 0x7f44f00ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.549+0000 7f44f4b08640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f44f00ca050 0x7f44f00ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47008/0 (socket says 172.21.15.120:47008) 2024-09-10T10:29:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.549+0000 7f44f4b08640 1 -- 172.21.15.120:0/4203822699 learned_addr learned my addr 172.21.15.120:0/4203822699 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.549+0000 7f44f5309640 1 --1- 172.21.15.120:0/4203822699 >> v1:172.21.15.120:6789/0 conn(0x7f44f0158360 0x7f44f015dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:45612/0 (socket says 172.21.15.120:45612) 2024-09-10T10:29:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.550+0000 7f44f4b08640 1 -- 172.21.15.120:0/4203822699 >> v1:172.21.15.134:6789/0 conn(0x7f44f015b570 legacy=0x7f44f006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:29:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.550+0000 7f44f4b08640 1 -- 172.21.15.120:0/4203822699 >> v1:172.21.15.120:6789/0 conn(0x7f44f0158360 legacy=0x7f44f015dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.550+0000 7f44e7fff640 1 -- 172.21.15.120:0/4203822699 <== mon.0 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1432196521 0 0) 0x7f44f015aab0 con 0x7f44f0158360 2024-09-10T10:29:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.550+0000 7f44f4b08640 1 -- 172.21.15.120:0/4203822699 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f015ead0 con 0x7f44f00ca050 2024-09-10T10:29:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.550+0000 7f44f4b08640 1 --2- 172.21.15.120:0/4203822699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f00ca050 0x7f44f00ca4a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009870 tx=0x7f44e002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958b7d339b65f053 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.551+0000 7f44e7fff640 1 -- 172.21.15.120:0/4203822699 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0037070 con 0x7f44f00ca050 2024-09-10T10:29:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.551+0000 7f44e7fff640 1 -- 172.21.15.120:0/4203822699 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f44e0032440 con 0x7f44f00ca050 2024-09-10T10:29:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.551+0000 7f44e7fff640 1 -- 172.21.15.120:0/4203822699 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0032830 con 0x7f44f00ca050 2024-09-10T10:29:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.551+0000 7f44f756d640 1 -- 172.21.15.120:0/4203822699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f00ca050 msgr2=0x7f44f00ca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.551+0000 7f44f756d640 1 --2- 172.21.15.120:0/4203822699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f00ca050 0x7f44f00ca4a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009870 tx=0x7f44e002a2a0 comp rx=0 tx=0).stop 2024-09-10T10:29:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.551+0000 7f44f756d640 1 -- 172.21.15.120:0/4203822699 shutdown_connections 2024-09-10T10:29:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.551+0000 7f44f756d640 1 --2- 172.21.15.120:0/4203822699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f00ca050 0x7f44f00ca4a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.551+0000 7f44f756d640 1 -- 172.21.15.120:0/4203822699 >> 172.21.15.120:0/4203822699 conn(0x7f44f0071700 msgr2=0x7f44f0071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.551+0000 7f44f756d640 1 -- 172.21.15.120:0/4203822699 shutdown_connections 2024-09-10T10:29:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.551+0000 7f44f756d640 1 -- 172.21.15.120:0/4203822699 wait complete. 2024-09-10T10:29:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.552+0000 7f44f756d640 1 Processor -- start 2024-09-10T10:29:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.552+0000 7f44f756d640 1 -- start start 2024-09-10T10:29:41.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f756d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44f00ca050 0x7f44f015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f756d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f0158360 0x7f44f015d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f756d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44f015b570 0x7f44f0149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f756d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44f015b050 con 0x7f44f0158360 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f756d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44f015ad70 con 0x7f44f00ca050 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f756d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f44f00703e0 con 0x7f44f015b570 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f4b08640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f0158360 0x7f44f015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f4b08640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f0158360 0x7f44f015d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47024/0 (socket says 172.21.15.120:47024) 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f4b08640 1 -- 172.21.15.120:0/1801747450 learned_addr learned my addr 172.21.15.120:0/1801747450 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f4b08640 1 -- 172.21.15.120:0/1801747450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44f015b570 msgr2=0x7f44f0149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f4b08640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44f015b570 0x7f44f0149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f4b08640 1 -- 172.21.15.120:0/1801747450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44f00ca050 msgr2=0x7f44f015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f4b08640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44f00ca050 0x7f44f015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.553+0000 7f44f4b08640 1 -- 172.21.15.120:0/1801747450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f0155f10 con 0x7f44f0158360 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.554+0000 7f44f4b08640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f0158360 0x7f44f015d410 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f44e00090b0 tx=0x7f44e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.554+0000 7f44e5ffb640 1 -- 172.21.15.120:0/1801747450 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0037040 con 0x7f44f0158360 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.554+0000 7f44e5ffb640 1 -- 172.21.15.120:0/1801747450 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f44e0004230 con 0x7f44f0158360 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.554+0000 7f44f756d640 1 -- 172.21.15.120:0/1801747450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f00cc6e0 con 0x7f44f0158360 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.554+0000 7f44e5ffb640 1 -- 172.21.15.120:0/1801747450 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0030770 con 0x7f44f0158360 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.554+0000 7f44f756d640 1 -- 172.21.15.120:0/1801747450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f00ccca0 con 0x7f44f0158360 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.555+0000 7f44f756d640 1 -- 172.21.15.120:0/1801747450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f00c9970 con 0x7f44f0158360 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.555+0000 7f44e5ffb640 1 -- 172.21.15.120:0/1801747450 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44e0030910 con 0x7f44f0158360 2024-09-10T10:29:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.556+0000 7f44e5ffb640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f44d406be30 0x7f44d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:29:41.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.556+0000 7f44e5ffb640 1 -- 172.21.15.120:0/1801747450 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6303+0+0 (secure 0 0 0) 0x7f44e00ac060 con 0x7f44f0158360 2024-09-10T10:29:41.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.556+0000 7f44f5309640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f44d406be30 0x7f44d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:29:41.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.556+0000 7f44f5309640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f44d406be30 0x7f44d406e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f44ec007fd0 tx=0x7f44ec007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:29:41.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.560+0000 7f44e5ffb640 1 -- 172.21.15.120:0/1801747450 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44e007a970 con 0x7f44f0158360 2024-09-10T10:29:41.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:41.726+0000 7f44f756d640 1 -- 172.21.15.120:0/1801747450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f44f00cb610 con 0x7f44f0158360 2024-09-10T10:29:42.119 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:29:42.122 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:29:42.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.442+0000 7f44e5ffb640 1 -- 172.21.15.120:0/1801747450 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v82) ==== 178+0+0 (secure 0 0 0) 0x7f44e007a270 con 0x7f44f0158360 2024-09-10T10:29:42.443 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_data_ec' removed 2024-09-10T10:29:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.445+0000 7f44f756d640 1 -- 172.21.15.120:0/1801747450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f44d406be30 msgr2=0x7f44d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.445+0000 7f44f756d640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f44d406be30 0x7f44d406e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f44ec007fd0 tx=0x7f44ec007920 comp rx=0 tx=0).stop 2024-09-10T10:29:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.445+0000 7f44f756d640 1 -- 172.21.15.120:0/1801747450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f0158360 msgr2=0x7f44f015d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:29:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.445+0000 7f44f756d640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f0158360 0x7f44f015d410 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f44e00090b0 tx=0x7f44e00046c0 comp rx=0 tx=0).stop 2024-09-10T10:29:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.445+0000 7f44f756d640 1 -- 172.21.15.120:0/1801747450 shutdown_connections 2024-09-10T10:29:42.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.445+0000 7f44f756d640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.120:6832/3379691116,v1:172.21.15.120:6833/3379691116] conn(0x7f44d406be30 0x7f44d406e2f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:42.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.445+0000 7f44f756d640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44f015b570 0x7f44f0149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:42.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.445+0000 7f44f756d640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f0158360 0x7f44f015d410 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:42.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.445+0000 7f44f756d640 1 --2- 172.21.15.120:0/1801747450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44f00ca050 0x7f44f015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:29:42.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.445+0000 7f44f756d640 1 -- 172.21.15.120:0/1801747450 >> 172.21.15.120:0/1801747450 conn(0x7f44f0071700 msgr2=0x7f44f00c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:29:42.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.446+0000 7f44f756d640 1 -- 172.21.15.120:0/1801747450 shutdown_connections 2024-09-10T10:29:42.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T10:29:42.446+0000 7f44f756d640 1 -- 172.21.15.120:0/1801747450 wait complete. 2024-09-10T10:29:42.460 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T10:29:42.460 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:29:42.460 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.469 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:29:42.469 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:29:42.469 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.521 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:29:42.522 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-10T10:29:42.522 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.571 INFO:tasks.ceph.mds.e:Stopped 2024-09-10T10:29:42.571 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:29:42.571 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.582 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:29:42.582 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:29:42.583 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.634 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:29:42.635 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-10T10:29:42.635 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.686 INFO:tasks.ceph.mds.f:Stopped 2024-09-10T10:29:42.686 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T10:29:42.687 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T10:29:42.687 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.717 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T10:29:42.717 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T10:29:42.717 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.791 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T10:29:42.791 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T10:29:42.792 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.860 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T10:29:42.861 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T10:29:42.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.934 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T10:29:42.935 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T10:29:42.935 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:42.972 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T10:29:42.972 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T10:29:42.973 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:43.038 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T10:29:43.038 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T10:29:43.038 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:43.111 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T10:29:43.111 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T10:29:43.111 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:43.175 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T10:29:43.175 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T10:29:43.176 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T10:29:43.176 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:43.203 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T10:29:43.203 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T10:29:43.203 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:43.243 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T10:29:43.243 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T10:29:43.243 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T10:29:43.244 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:43.255 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T10:29:43.255 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T10:29:43.255 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:43.294 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T10:29:43.295 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T10:29:43.295 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:29:43.346 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T10:29:43.346 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T10:29:43.347 DEBUG:teuthology.orchestra.run.smithi120:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v MDS_SLOW_METADATA_IO | egrep -v MDS_TRIM | head -n 1 2024-09-10T10:29:43.388 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:29:43.388 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T10:29:43.839 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:29:43.840 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T10:29:43.929 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:29:43.929 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T10:29:43.985 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:29:43.985 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T10:29:44.033 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:29:44.034 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T10:29:44.282 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:29:44.282 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T10:29:44.390 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:29:44.390 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T10:29:44.448 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:29:44.448 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T10:29:44.505 INFO:tasks.ceph:Archiving mon data... 2024-09-10T10:29:44.506 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029/data/mon.a.tgz 2024-09-10T10:29:44.506 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-09-10T10:29:44.524 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.zTpdTdhQ7t 2024-09-10T10:29:44.524 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.zTpdTdhQ7t 2024-09-10T10:29:44.671 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.zTpdTdhQ7t 2024-09-10T10:29:44.755 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.zTpdTdhQ7t is 364KB 2024-09-10T10:29:44.850 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.zTpdTdhQ7t 2024-09-10T10:29:44.865 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029/data/mon.b.tgz 2024-09-10T10:29:44.865 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-10T10:29:44.883 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.L7fsj3T8Qs 2024-09-10T10:29:44.883 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.L7fsj3T8Qs 2024-09-10T10:29:45.061 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.L7fsj3T8Qs 2024-09-10T10:29:45.144 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.L7fsj3T8Qs is 503KB 2024-09-10T10:29:45.241 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.L7fsj3T8Qs 2024-09-10T10:29:45.256 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029/data/mon.c.tgz 2024-09-10T10:29:45.256 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-10T10:29:45.310 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.86OLTw8JqO 2024-09-10T10:29:45.311 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.86OLTw8JqO 2024-09-10T10:29:45.473 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.86OLTw8JqO 2024-09-10T10:29:45.557 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.86OLTw8JqO is 503KB 2024-09-10T10:29:45.655 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.86OLTw8JqO 2024-09-10T10:29:45.669 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T10:29:45.670 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T10:29:45.672 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T10:29:45.756 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T10:29:45.789 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T10:29:45.789 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T10:29:45.791 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029/remote/smithi120/crash 2024-09-10T10:29:45.792 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T10:29:45.824 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029/remote/smithi134/crash 2024-09-10T10:29:45.825 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T10:29:45.852 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T10:29:45.853 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:29:45.869 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:29:45.977 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T10:29:46.006 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T10:29:46.031 INFO:tasks.ceph:Compressing logs... 2024-09-10T10:29:46.031 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:29:46.049 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:29:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36889.log 2024-09-10T10:29:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37100.log 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36889.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.37100.log: 86.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.36889.log.gz 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36849.log 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose 88.3% -- /var/log/ceph/ceph-mds.b.log -- replaced with /var/log/ceph/ceph-client.admin.37100.log.gz 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36929.log 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36849.log.gz 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36929.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36929.log.gz 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36969.log 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T10:29:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36969.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.6.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36969.log.gz 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose/var/log/ceph/ceph-osd.5.log: -- /var/log/ceph/ceph-client.admin.37060.log 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.37060.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37060.log.gz 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37020.log 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mgr.x.log: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36809.log 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.37020.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37020.log.gz 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T10:29:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36809.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36809.log.gz 2024-09-10T10:29:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-09-10T10:29:46.133 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-mds.f.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-09-10T10:29:46.142 INFO:teuthology.orchestra.run.smithi134.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T10:29:46.186 INFO:teuthology.orchestra.run.smithi134.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T10:29:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T10:29:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54838.log 2024-09-10T10:29:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49250.log 2024-09-10T10:29:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54615.log 2024-09-10T10:29:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49408.log 2024-09-10T10:29:46.188 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54838.log.gz 2024-09-10T10:29:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56401.log 2024-09-10T10:29:46.188 INFO:teuthology.orchestra.run.smithi120.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.49250.log.gz 2024-09-10T10:29:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49408.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54615.log.gz 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42170.log 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51821.log 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49408.log.gz 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56401.log.gz 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45842.log 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51821.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42170.log.gz 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56089.log 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51821.log.gz 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46338.log 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46638.log 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.45842.log.gz 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56089.log.gz 2024-09-10T10:29:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46338.log: -- /var/log/ceph/ceph-client.admin.42560.log 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50265.log 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46338.log.gz 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46638.log.gz 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53854.log 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.50265.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42560.log.gz 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42556.log 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48824.log 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.50265.log.gz 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53854.log.gz 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42559.log 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48824.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42556.log.gz 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51900.log 2024-09-10T10:29:46.190 INFO:teuthology.orchestra.run.smithi120.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48824.log.gz 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45259.log 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42559.log: /var/log/ceph/ceph-client.admin.51900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54531.log 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42559.log.gz 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54833.log 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.51900.log.gz 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45259.log.gz 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42086.log 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54531.log.gz 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42380.log 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42086.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54833.log.gz 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48481.log 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42086.log.gz 2024-09-10T10:29:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43201.log 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42380.log.gz 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47880.log 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48481.log: /var/log/ceph/ceph-client.admin.43201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43613.log 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48481.log.gz 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47880.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43201.log.gz 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42549.log 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47880.log.gz 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46298.log 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43613.log.gz 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54363.log 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr: 89.3%/var/log/ceph/ceph-client.admin.46298.log: -- replaced with /var/log/ceph/ceph-client.admin.42549.log.gz 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36430.log 2024-09-10T10:29:46.192 INFO:teuthology.orchestra.run.smithi120.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46298.log.gz 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54363.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.36430.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54363.log.gz 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48105.log 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36430.log.gz 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43792.log 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42898.log 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48105.log.gz 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55505.log 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43792.log.gz 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43606.log 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42898.log: /var/log/ceph/ceph-client.admin.55505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54279.log 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42898.log.gz 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43606.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55505.log.gz 2024-09-10T10:29:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51979.log 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43606.log.gz 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50543.log 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54279.log.gz 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41876.log 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51979.log: /var/log/ceph/ceph-client.admin.50543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49008.log 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51979.log.gz 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41876.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.50543.log.gz 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51563.log 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49008.log: gzip -5 --verbose 87.8% -- /var/log/ceph/ceph-client.admin.57266.log -- replaced with /var/log/ceph/ceph-client.admin.41876.log.gz 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53559.log 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49008.log.gz 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57266.log: gzip 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51563.log.gz -5 2024-09-10T10:29:46.194 INFO:teuthology.orchestra.run.smithi120.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54837.log 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53559.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.57266.log.gz --verbose 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr: -- /var/log/ceph/ceph-client.admin.48884.log 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53716.log 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.53559.log.gz 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54837.log.gz 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43242.log 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48884.log.gz 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47582.log 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53716.log: /var/log/ceph/ceph-client.admin.43242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42894.log 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53716.log.gz 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47582.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43242.log.gz 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52862.log 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56695.log 2024-09-10T10:29:46.195 INFO:teuthology.orchestra.run.smithi120.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47582.log.gz 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42894.log.gz 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44149.log 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56695.log: gzip 90.7% -- replaced with /var/log/ceph/ceph-client.admin.52862.log.gz 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40577.log 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56695.log.gz 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49428.log 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41019.log 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44149.log.gz 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49428.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40577.log.gz 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42128.log 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49428.log.gz 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43476.log 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41061.log 2024-09-10T10:29:46.196 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42128.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41019.log.gz 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43476.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42128.log.gz 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45301.log 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43476.log.gz 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55865.log 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41061.log: /var/log/ceph/ceph-client.admin.45301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43612.log 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45301.log.gz 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42002.log 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41061.log.gz 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55865.log.gz 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40767.log 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43612.log: /var/log/ceph/ceph-client.admin.42002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42884.log 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43612.log.gz 2024-09-10T10:29:46.197 INFO:teuthology.orchestra.run.smithi120.stderr: 89.6%/var/log/ceph/ceph-client.admin.40767.log: -- replaced with /var/log/ceph/ceph-client.admin.42002.log.gz 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57040.log 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42884.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40767.log.gz 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47728.log 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42884.log.gz 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55506.log 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57040.log: /var/log/ceph/ceph-client.admin.47728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46977.log 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57040.log.gz 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43927.log 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47728.log.gz 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55506.log.gz 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40723.log 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48722.log 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46977.log.gz 2024-09-10T10:29:46.198 INFO:teuthology.orchestra.run.smithi120.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43927.log.gz 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48364.log 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42338.log 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40723.log.gz 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48722.log: /var/log/ceph/ceph-client.admin.48364.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48722.log.gz 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48603.log 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42212.log 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42338.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.48364.log.gz 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43456.log 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42338.log.gz 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48603.log.gz 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41799.log 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43456.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42212.log.gz 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50902.log 2024-09-10T10:29:46.199 INFO:teuthology.orchestra.run.smithi120.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43456.log.gz 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42561.log 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41799.log: /var/log/ceph/ceph-client.admin.50902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44390.log 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41799.log.gz 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42561.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50902.log.gz 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43202.log 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42561.log.gz 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40533.log 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54237.log 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43202.log.gz 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55509.log 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44390.log.gz 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40533.log.gz 2024-09-10T10:29:46.200 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54237.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54237.log.gz 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42891.log 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55509.log: /var/log/ceph/ceph.audit.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55509.log.gz 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56611.log 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53941.log 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42891.log: /var/log/ceph/ceph-client.admin.56611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43195.log 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42891.log.gz 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56611.log.gz 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56485.log 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53941.log.gz 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54153.log 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43609.log 2024-09-10T10:29:46.201 INFO:teuthology.orchestra.run.smithi120.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43195.log.gz 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56485.log: /var/log/ceph/ceph-client.admin.54153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56779.log 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56485.log.gz 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54153.log.gz 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45217.log 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56779.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43609.log.gz 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56569.log 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56779.log.gz 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56905.log 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45217.log: /var/log/ceph/ceph-client.admin.56569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57221.log 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45217.log.gz 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56905.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56569.log.gz 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46978.log 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56905.log.gz 2024-09-10T10:29:46.202 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46898.log 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57221.log.gz 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55166.log 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55168.log 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46978.log.gz 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44279.log 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46898.log.gz 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55168.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55166.log.gz 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40851.log 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56946.log 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44279.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55168.log.gz 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40851.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44279.log.gz 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43102.log 2024-09-10T10:29:46.203 INFO:teuthology.orchestra.run.smithi120.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40851.log.gz 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56527.log 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56946.log: /var/log/ceph/ceph-client.admin.43102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54321.log 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56946.log.gz 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43102.log.gz 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55171.log 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44522.log 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4%/var/log/ceph/ceph-client.admin.54321.log: -- replaced with /var/log/ceph/ceph-client.admin.56527.log.gz 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57350.log 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54321.log.gz 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55171.log.gz 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54836.log 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41918.log 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57350.log.gz 2024-09-10T10:29:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42044.log 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44522.log.gz 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41918.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54836.log.gz 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54447.log 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41918.log.gz 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54832.log 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54447.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42044.log.gz 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47097.log 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54447.log.gz 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41549.log 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr: 89.0%/var/log/ceph/ceph-client.admin.47097.log: -- replaced with /var/log/ceph/ceph-client.admin.54832.log.gz 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57125.log 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47097.log.gz 2024-09-10T10:29:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52737.log 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42897.log 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57125.log: /var/log/ceph/ceph-client.admin.52737.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57125.log.gz 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52737.log.gz 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40491.log 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56005.log 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42897.log.gz 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mds.a.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.41549.log.gz 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40621.log 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56005.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40491.log.gz 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43969.log 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56005.log.gz 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56737.log 2024-09-10T10:29:46.206 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55511.log 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43969.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40621.log.gz 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56737.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43969.log.gz 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50222.log 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56737.log.gz 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45363.log 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55511.log.gz 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47963.log 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.50222.log: /var/log/ceph/ceph-client.admin.45363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42896.log 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50222.log.gz 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42562.log 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45363.log.gz 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr: 88.3%/var/log/ceph/ceph-client.admin.42896.log: -- replaced with /var/log/ceph/ceph-client.admin.47963.log.gz 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42892.log 2024-09-10T10:29:46.207 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42562.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42896.log.gz 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56443.log 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42892.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42562.log.gz 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44065.log 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42892.log.gz 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47243.log 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44065.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56443.log.gz 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47160.log 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47243.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44065.log.gz 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57082.log 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47160.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.47243.log.gz 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56317.log 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47160.log.gz 2024-09-10T10:29:46.208 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55173.log 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56317.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44456.log 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57082.log.gz 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr: 87.7%/var/log/ceph/ceph-client.admin.55173.log: -- replaced with /var/log/ceph/ceph-client.admin.56317.log.gz 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43614.log 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55173.log.gz 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51183.log 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49946.log 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43614.log: /var/log/ceph/ceph-client.admin.51183.log: gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43614.log.gz 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr: -- /var/log/ceph/ceph-mon.a.log 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44456.log.gz 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51183.log.gz 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45500.log 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mon.a.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.49946.log.gz 2024-09-10T10:29:46.209 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43199.log 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56275.log 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43199.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45500.log.gz 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40232.log 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr: 88.2%/var/log/ceph/ceph-client.admin.56275.log: -- replaced with /var/log/ceph/ceph-client.admin.43199.log.gz 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52341.log 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56275.log.gz 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44191.log 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52341.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.41960.log 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40232.log.gz 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49789.log 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52341.log.gz/var/log/ceph/ceph-client.admin.41960.log: 2024-09-10T10:29:46.210 INFO:teuthology.orchestra.run.smithi120.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44191.log.gz 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43601.log 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41960.log.gz 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55707.log 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49789.log: /var/log/ceph/ceph-client.admin.43601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40977.log 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr: 91.1% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43601.log.gz 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49789.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55175.log 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55707.log: /var/log/ceph/ceph-client.admin.40977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56990.log 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55175.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40977.log.gz 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55707.log.gz 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56990.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55175.log.gz 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42558.log 2024-09-10T10:29:46.211 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.2.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56990.log.gz 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48582.log 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42296.log 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42558.log.gz 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.48582.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30030.log 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42296.log: 90.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53884.log 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48582.log.gz 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42296.log.gz 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph.tmp-client.admin.30030.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30030.log.gz 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54657.log 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40809.log 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54657.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53884.log.gz 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53985.log 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54657.log.gz 2024-09-10T10:29:46.212 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54195.log 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40809.log.gz 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47300.log 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53985.log.gz 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42895.log 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54195.log.gz 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47742.log 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47300.log.gz 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42895.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42895.log.gz 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.47742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46720.log 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mgr.y.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47742.log.gz 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52779.log 2024-09-10T10:29:46.213 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55791.log 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52779.log: gzip -5 --verbose 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46720.log.gz 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr: -- /var/log/ceph/ceph-client.admin.43196.log 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52779.log.gz 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55791.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55791.log.gz 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54489.log 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49148.log 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr: 88.3%/var/log/ceph/ceph-client.admin.54489.log: -- replaced with /var/log/ceph/ceph-client.admin.43196.log.gz 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52040.log 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54489.log.gz 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44237.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49148.log.gz 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.52040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56653.log 2024-09-10T10:29:46.214 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44237.log: 90.9% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44237.log.gz 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52040.log.gz 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40449.log 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53241.log 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56653.log.gz 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49864.log 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40449.log.gz 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54825.log 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49864.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53241.log.gz 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40347.log 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49864.log.gz 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56131.log 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54825.log: /var/log/ceph/ceph-client.admin.40347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56191.log 2024-09-10T10:29:46.215 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56131.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54825.log.gz 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40347.log.gz 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56359.log 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56131.log.gz 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55507.log 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56191.log: /var/log/ceph/ceph-client.admin.56359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40140.log 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55507.log: 87.6% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56191.log.gz 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56359.log.gz 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55507.log.gz 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41127.log 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55172.log 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40140.log: /var/log/ceph/ceph-client.admin.41127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56047.log 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55172.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40140.log.gz 2024-09-10T10:29:46.216 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41127.log.gz 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54027.log 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55172.log.gz 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53812.log 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56047.log: /var/log/ceph/ceph-client.admin.54027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55156.log 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53812.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56047.log.gz 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54027.log.gz 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40893.log 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53812.log.gz 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51325.log 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55156.log: /var/log/ceph/ceph-client.admin.40893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44107.log 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51325.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55156.log.gz 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40893.log.gz 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50661.log 2024-09-10T10:29:46.217 INFO:teuthology.orchestra.run.smithi120.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51325.log.gz 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52159.log 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44107.log: /var/log/ceph/ceph-client.admin.50661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36337.log 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52159.log: 88.8% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44107.log.gz 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50661.log.gz 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52159.log.gz 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43834.log 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49495.log 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.36337.log: /var/log/ceph/ceph-client.admin.43834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46080.log 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.49495.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36337.log.gz 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43834.log.gz 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55508.log 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49495.log.gz 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43241.log 2024-09-10T10:29:46.218 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46080.log: /var/log/ceph/ceph-client.admin.55508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54839.log 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43241.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55508.log.gz 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46080.log.gz 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43241.log.gz 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44023.log 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46382.log 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54839.log.gz 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50743.log 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44023.log.gz 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51126.log 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46382.log: /var/log/ceph/ceph-client.admin.50743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44717.log 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.51126.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.46382.log.gz 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50743.log.gz 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50461.log 2024-09-10T10:29:46.219 INFO:teuthology.orchestra.run.smithi120.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51126.log.gz 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45037.log 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.44717.log: /var/log/ceph/ceph-client.admin.50461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43876.log 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.0.45037.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50461.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44717.log.gz 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45037.log.gz 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51522.log 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41741.log 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.43876.log: /var/log/ceph/ceph-client.admin.51522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45499.log 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.41741.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43876.log.gz 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51522.log.gz 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48146.log 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41741.log.gz 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55749.log 2024-09-10T10:29:46.220 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45499.log: /var/log/ceph/ceph-client.admin.48146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55510.log 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55749.log: 90.9% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45499.log.gz 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48146.log.gz 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55749.log.gz 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44588.log 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54405.log 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55510.log: gzip/var/log/ceph/ceph-client.admin.44588.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45883.log 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54405.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55510.log.gz 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr: 91.0% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44588.log.gz 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44321.log 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54405.log.gz 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50742.log 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45883.log: /var/log/ceph/ceph-client.admin.44321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56821.log 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.50742.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45883.log.gz 2024-09-10T10:29:46.221 INFO:teuthology.orchestra.run.smithi120.stderr:gzip 87.8% -5 --verbose -- /var/log/ceph/ceph-client.admin.42254.log 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44321.log.gz 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50742.log.gz 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56821.log: /var/log/ceph/ceph-client.admin.42254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57308.log 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mds.e.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42254.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56821.log.gz 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54824.log 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56863.log 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.57308.log: /var/log/ceph/ceph-client.admin.54824.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57308.log.gz 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54824.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52438.log 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55371.log 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.56863.log: /var/log/ceph/ceph-client.admin.52438.log: 88.6% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56863.log.gz 2024-09-10T10:29:46.222 INFO:teuthology.orchestra.run.smithi120.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52438.log.gz 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50830.log 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54573.log 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55371.log: /var/log/ceph/ceph-client.admin.50830.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55371.log.gz 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50830.log.gz 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36076.log 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40681.log 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54573.log: /var/log/ceph/ceph-client.admin.36076.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54573.log.gz 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36076.log.gz 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52821.log 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr: 87.8%/var/log/ceph/ceph-osd.3.log: -- replaced with /var/log/ceph/ceph-client.admin.40681.log.gz 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55174.log 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52821.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52821.log.gz 2024-09-10T10:29:46.223 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.55174.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55174.log.gz 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45042.log 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mds.c.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53261.log 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.0.45042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45042.log.gz 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54111.log 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.53261.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53261.log.gz 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52619.log 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54111.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54111.log.gz 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54069.log 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.52619.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52619.log.gz 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45175.log 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.54069.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54069.log.gz 2024-09-10T10:29:46.224 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46216.log 2024-09-10T10:29:46.225 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.45175.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45175.log.gz 2024-09-10T10:29:46.225 INFO:teuthology.orchestra.run.smithi134.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40935.log 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.46216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42551.log 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.40935.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46216.log.gz 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47640.log 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40935.log.gz 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56233.log 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.42551.log: /var/log/ceph/ceph-client.admin.47640.log: 87.8%/var/log/ceph/ceph-client.admin.56233.log: -- replaced with /var/log/ceph/ceph-client.admin.42551.log.gz 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47640.log.gz 2024-09-10T10:29:46.228 INFO:teuthology.orchestra.run.smithi120.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56233.log.gz 2024-09-10T10:29:46.242 INFO:teuthology.orchestra.run.smithi134.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T10:29:46.454 INFO:teuthology.orchestra.run.smithi120.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T10:29:46.837 INFO:teuthology.orchestra.run.smithi134.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T10:29:46.842 INFO:teuthology.orchestra.run.smithi120.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T10:29:46.850 INFO:teuthology.orchestra.run.smithi120.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T10:29:46.878 INFO:teuthology.orchestra.run.smithi120.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T10:29:46.879 INFO:teuthology.orchestra.run.smithi134.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T10:29:46.894 INFO:teuthology.orchestra.run.smithi134.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T10:29:46.936 INFO:teuthology.orchestra.run.smithi134.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T10:29:46.936 INFO:teuthology.orchestra.run.smithi120.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T10:29:46.938 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-10T10:29:46.938 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m0.844s 2024-09-10T10:29:46.938 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m3.380s 2024-09-10T10:29:46.938 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.129s 2024-09-10T10:29:46.939 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T10:29:46.939 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m0.877s 2024-09-10T10:29:46.939 INFO:teuthology.orchestra.run.smithi120.stderr:user 0m3.538s 2024-09-10T10:29:46.939 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m0.274s 2024-09-10T10:29:46.939 INFO:tasks.ceph:Archiving logs... 2024-09-10T10:29:46.940 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029/remote/smithi120/log 2024-09-10T10:29:46.941 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:29:47.711 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029/remote/smithi134/log 2024-09-10T10:29:47.712 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:29:48.236 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T10:29:48.245 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T10:29:48.245 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:29:48.248 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:29:48.329 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T10:29:48.338 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T10:29:48.338 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:29:48.341 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:29:48.353 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:29:48.356 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:29:48.358 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:29:48.358 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-10T10:29:48.358 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 -426us[ -433us] +/- 50ms 2024-09-10T10:29:48.358 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 +341us[ +341us] +/- 64ms 2024-09-10T10:29:48.358 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +243us[ +243us] +/- 78ms 2024-09-10T10:29:48.358 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:29:48.360 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:29:48.360 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-10T10:29:48.361 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 31 -398us[ -395us] +/- 50ms 2024-09-10T10:29:48.361 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 29 +353us[ +353us] +/- 64ms 2024-09-10T10:29:48.361 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 +254us[ +254us] +/- 78ms 2024-09-10T10:29:48.361 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:29:48.362 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T10:29:48.370 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T10:29:48.371 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T10:29:48.379 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:29:48.469 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:29:48.536 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:29:48.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:29:48.565 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:29:48.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:29:48.593 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T10:29:48.602 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T10:29:48.610 INFO:teuthology.task.internal:Duration was 596.303334 seconds 2024-09-10T10:29:48.610 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T10:29:48.618 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T10:29:48.619 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:29:48.621 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:29:48.657 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:29:48.673 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:29:48.805 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T10:29:48.806 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T10:29:48.806 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:29:48.866 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-09-10T10:29:48.866 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:29:48.890 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T10:29:48.890 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:29:48.908 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:29:48.955 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T10:29:48.956 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:29:48.974 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:29:49.102 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T10:29:49.112 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T10:29:49.112 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:29:49.137 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:29:49.163 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T10:29:49.172 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:29:49.180 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:29:49.201 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-09-10T10:29:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-09-10T10:29:49.257 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:29:49.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:29:49.272 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:29:49.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:29:49.311 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T10:29:49.321 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T10:29:49.322 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029/remote/smithi120 2024-09-10T10:29:49.322 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:29:49.359 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899029/remote/smithi134 2024-09-10T10:29:49.359 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:29:49.393 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T10:29:49.393 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:29:49.396 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:29:49.442 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T10:29:49.453 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T10:29:49.453 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T10:29:49.462 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T10:29:49.462 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:29:49.465 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:29:49.477 INFO:teuthology.orchestra.run.smithi120.stdout: 265325 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:29 /home/ubuntu/cephtest 2024-09-10T10:29:49.497 INFO:teuthology.orchestra.run.smithi134.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:29 /home/ubuntu/cephtest 2024-09-10T10:29:49.499 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T10:29:49.508 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T10:29:49.531 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} duration: 596.303334236145 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-10T10:29:49.531 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T10:29:49.639 INFO:teuthology.run:pass